Business Research Methods William G. Zikmund Chapter 12: Experimental Research

Copyright © 2000 by Harcourt, Inc. All rights reserved. Requests for permission to make copies of any part of the work should be mailed to the following address: Permissions Department, Harcourt, Inc., 6277 Sea Harbor Drive, Orlando, Florida

Copyr ight © 2000 by Harcourt, Inc. All rights reserved EXPERIMENT A RESEARCH INVESTIGATION IN WHICH CONDITIONS ARE CONTROLLED ONE INDEPENDENT VARIABLE IS MANIPULATED (SOMETIMES MORE THAN ONE) ITS EFFECT ON A DEPENDENT VARIABLE IS MEASURED TO TEST A HYPOTHESIS

Copyr ight © 2000 by Harcourt, Inc. All rights reserved Manipulation of the Independent Variable Selection of Dependent Variable Assignment of Subjects (or other Test Units) Control Over Extraneous Variables Basic Issues of Experimental Design

Copyr ight © 2000 by Harcourt, Inc. All rights reserved The experimenter has some degree of control over the independent variable. The variable is independent because its value can be manipulated by the experimenter to whatever he or she wishes it to be.

Copyr ight © 2000 by Harcourt, Inc. All rights reserved Experiment Treatment Alternative manipulations of the independent variable being investigated

Copyr ight © 2000 by Harcourt, Inc. All rights reserved Independent Variable The experimenter controls independent variable. The variable’s value can be manipulated by the experimenters to whatever they wish it to be.

Copyr ight © 2000 by Harcourt, Inc. All rights reserved MANIPULATION OF INDEPENDENT VARIABLE Classificatory Vs. Continuous Variables Experimental and Control Groups Treatment Levels More Than One Independent Variable

Copyr ight © 2000 by Harcourt, Inc. All rights reserved Experimental treatments are the alternative manipulations of the independent variable being investigated.

Copyr ight © 2000 by Harcourt, Inc. All rights reserved DEPENDENT VARIABLE its value is expected to be dependent on the experimenter’s manipulation criterion or standard by which the results are judged

Copyr ight © 2000 by Harcourt, Inc. All rights reserved DEPENDENT VARIABLE SELECTION –e.g... sales volume, awareness, recall MEASUREMENT

Copyr ight © 2000 by Harcourt, Inc. All rights reserved Dependent Variable Selection - e.g., sales volume, awareness, recall Measurement

Copyr ight © 2000 by Harcourt, Inc. All rights reserved The dependent variable is so called because its value is expected to be dependent on the experimenter’s manipulation; it is the criterion or standard by which the results are judged.

Copyr ight © 2000 by Harcourt, Inc. All rights reserved TEST UNITS - subjects or entities whose response to the experimental treatment are measured or observed.

Copyr ight © 2000 by Harcourt, Inc. All rights reserved Two Types of Experimental Error Constant Error Random Errors

Copyr ight © 2000 by Harcourt, Inc. All rights reserved TWO TYPES OF EXPERIMENTAL ERRORS CONSTANT ERRORS RANDOM ERRORS

Copyr ight © 2000 by Harcourt, Inc. All rights reserved FIELD VERSUS LABORATORY EXPERIMENTS

Copyr ight © 2000 by Harcourt, Inc. All rights reserved Controlling Extraneous Variables Elimination of Extraneous Variables Constancy of Conditions Order of Presentation Blinding Random Assignment

Copyr ight © 2000 by Harcourt, Inc. All rights reserved How May an Experimenter control for Extraneous Variation? Eliminate Extraneous Variables Hold Conditions Constant Randomization Matching Subjects

Copyr ight © 2000 by Harcourt, Inc. All rights reserved Establishing Control

Copyr ight © 2000 by Harcourt, Inc. All rights reserved DEMAND CHARACTERISTICS EXPERIMENTAL PROCEDURES THE INTENTIONALLY HINT TO SUBJECTS SOMETHING ABOUT THE EXPERIMENTER’S HYPOTHESIS

Copyr ight © 2000 by Harcourt, Inc. All rights reserved Demand Characteristics Guinea Pigs Hawthorne Effect

Copyr ight © 2000 by Harcourt, Inc. All rights reserved Field Vs. Laboratory Experiment

Laboratory ExperimentField Experiment Artificial-Low Realism Few Extraneous Variables High control Low Cost Short Duration Subjects Aware of Participation Natural-High Realism Many Extraneous Variables Low control High Cost Long Duration Subjects Unaware of Participation Co pyright © 2000 by Harcourt, Inc. All rights reserved

CONTROL GROUPS Isolates Extraneous Variation

Copyr ight © 2000 by Harcourt, Inc. All rights reserved WHEN IS AN EXPERIMENT INTERNAL VALIDITY? INTERNAL VALIDITY - THE ABILITY OF AN EXPERIMENT TO ANSWER THE QUESTION OF WHETHER THE EXPERIMENTAL TREATMENT WAS THE SOLE CAUSE OF CHANGES IN A DEPENDENT VARIABLE DID THE MANIPULATION DO WHAT IT WAS SUPPOSED TO DO?

Copyr ight © 2000 by Harcourt, Inc. All rights reserved FACTORS INFLUENCING INTERNAL VALIDITY History Maturation Testing Instrumentation Selection Mortality

Copyr ight © 2000 by Harcourt, Inc. All rights reserved ISOLATING EXTRANEOUS VARIATION WITH A CONTROL GROUP History Effects Maturation Effects Mortality Effects

Type of Extraneous VariableExample History - specific events in the Environment between the Before and After Measurement That Are beyond the Experimenter’s Control Maturation - subjects Change during the Course of the Experiment Testing - The Before Measure Alerts or Sensitizes Subject to Nature of Experiment or Second Measure A Major Employer Closes Its Plant in Test Market Area Subjects Become Tired Questionnaire about the Traditional Role of Women Triggers Enhanced Awareness of Women in an Experiment Copyright © 2000 by Harcourt, Inc. All rights reserved.

Instrument - Changes in Instrument Result in Response Bias Selection - Sample Selection Error Because of Differential Selection Comparison Groups Mortality - Sample Attrition; some Subjects Withdraw from Experiment New Questions about Women are Interpreted Differently from Earlier Questions. Control Group and Experimental Group Is Self-Selected Group Based on Preference for Soft Drinks Subjects in One Group of a Hair Dying Study Marry Rich Widows and move to Florida Copyright © 2000 by Harcourt, Inc. All rights reserved.

HOW CAN INTERNAL VALIDITY BE INCREASE? Copyright © 2000 by Harcourt, Inc. All rights reserved.

Copyr ight © 2000 by Harcourt, Inc. All rights reserved INCREASING INTERNAL VALIDITY Control Group Random Assignment Pre-testing and Post-testing Posttest Only

WHAT ARE THE DIFFERENT BASIC EXPERIMENTAL DESIGNS? Copyright © 2000 by Harcourt, Inc. All rights reserved.

Copyr ight © 2000 by Harcourt, Inc. All rights reserved Quasi-Experimental Designs One Shot Design (After Only) One Group Pretest-Posttest Static Group Design

Copyr ight © 2000 by Harcourt, Inc. All rights reserved ONE SHOT DESIGN (AFTER ONLY) XO 1

Copyr ight © 2000 by Harcourt, Inc. All rights reserved One Group Pretest-Posttest O 1 X O 2

Copyr ight © 2000 by Harcourt, Inc. All rights reserved Static Group Design Experimental GroupX O 1 Control Group O 2

Copyr ight © 2000 by Harcourt, Inc. All rights reserved Three Good Experimental Designs Pretest - Posttest Control Group Design Posttest Only Control Group Solomon Four Group Design

Copyr ight © 2000 by Harcourt, Inc. All rights reserved Pretest-Posttest Control Group Design Experimental Group R O 1 X O 2 Control Group R O 3 X O 4

Copyr ight © 2000 by Harcourt, Inc. All rights reserved Posttest Only Control Group Experimental Group R X O 1 Control Group R O 2

Copyr ight © 2000 by Harcourt, Inc. All rights reserved One-Shot Design Internal Validity Problems History –weak Maturation –weak Testing –not relevant Instrumentation –not relevant Selection –weak Mortality –weak

Copyr ight © 2000 by Harcourt, Inc. All rights reserved One-Group Pretest-Posttest Internal Validity Problems History –weak Maturation –weak Testing –weak Instrumentation –weak Selection –controlled Mortality –controlled

Copyr ight © 2000 by Harcourt, Inc. All rights reserved Static-Group Design Internal Validity Problems History –controlled Maturation –possible source of concern Testing –controlled Instrumentation –controlled Selection –weak Mortality –weak

Copyr ight © 2000 by Harcourt, Inc. All rights reserved Pretest-Posttest Control Internal Validity Problems History –controlled Maturation –controlled Testing –controlled Instrumentation –controlled Selection –controlled Mortality –controlled

Copyr ight © 2000 by Harcourt, Inc. All rights reserved Solomon Four-Group Design Internal Validity Problems History –controlled Maturation –controlled Testing –controlled Instrumentation –controlled Selection –controlled Mortality –controlled

Copyr ight © 2000 by Harcourt, Inc. All rights reserved Posttest-Only Control Internal Validity Problems History –controlled Maturation –controlled Testing –controlled Instrumentation –controlled Selection –controlled Mortality –controlled

Copyr ight © 2000 by Harcourt, Inc. All rights reserved Solomon four Group Design Experimental Group 1: R O 1 X O 2 Control Group 1: R O 3 O 4 Experimental Group 2: R X O 5 Control Group 2: R X O 6

Copyr ight © 2000 by Harcourt, Inc. All rights reserved ADVANCED EXPERIMENTAL DESIGNS ARE MORE COMPLEX COMPLETELY RANDOMIZED RANDOMIZED BLOCK DESIGN LATIN SQUARE FACTORIAL

Copyr ight © 2000 by Harcourt, Inc. All rights reserved Completely randomized design An experimental design that uses a random process to assign subjects (test units) and treatments to investigate the effects of only one independent variable.

Copyr ight © 2000 by Harcourt, Inc. All rights reserved Completely Randomized Designs Average minutes shopper spends in store Control: no music Experimental treatment: slow music Experimental treatment: fast music

Copyr ight © 2000 by Harcourt, Inc. All rights reserved INDEPENDENT VARIABLE A Group AGroup BGroup C Level 1Level 2Level 3

Copyr ight © 2000 by Harcourt, Inc. All rights reserved Completely Randomized Design With a pretest posttest Group ARO 1 X 1 O 2 Group ARO 3 X 2 O 4 Group ARO 5 X 3 O 6

Copyr ight © 2000 by Harcourt, Inc. All rights reserved Completely Randomized Design With a posttest Group ARX 1 O 1 Group BRX 2 O 2 Group CRX 3 O 3

Copyr ight © 2000 by Harcourt, Inc. All rights reserved Randomized block design An extension of the completely randomized design in which a single extraneous variable that might affect test units’ response to the treatment has been identified and the effects of this variable are isolated by blocking out its effects.

Copyr ight © 2000 by Harcourt, Inc. All rights reserved Randomized Block Design Independent Variables Control: no music Experimental treatment slow music Experimental treatment: fast music Mornings and afternoons Evening hours Blocking variable

Copyr ight © 2000 by Harcourt, Inc. All rights reserved Factorial design An experiment that investigates the interaction of two or more variables on a single dependent variable.

Copyr ight © 2000 by Harcourt, Inc. All rights reserved Independent Variable 1 No Music cart signs Slow MusicFast MusicNo Music Grocery cart signs Independent Variable 2

Copyr ight © 2000 by Harcourt, Inc. All rights reserved PriceRedGold $25Cell 1Cell 4 $30Cell 2Cell 5 $35Cell 3Cell 6 Factorial Design -- Roller Skates Package Design

Copyr ight © 2000 by Harcourt, Inc. All rights reserved EFFECTS Main effect The influence of a single independent variable on a dependent variable. Interaction effect The influence on a dependent variable by combinations of two or more independent variables.

Copyr ight © 2000 by Harcourt, Inc. All rights reserved 2 x 2 Factorial Design Men Women Ad A Ad B Main Effects of Gender Main Effects of Ad > >

Copyr ight © 2000 by Harcourt, Inc. All rights reserved Ad AAd B Women Men Believability Interaction Between Gender and Ad Copy

Copyr ight © 2000 by Harcourt, Inc. All rights reserved Level 1Level 2 Level 1 Level 2 Group A Group D Group C Group B INDEPENDENT VARIABLE 2 INDEPENDENT VARIABLE 1

Copyr ight © 2000 by Harcourt, Inc. All rights reserved Group ARO 1 X 11 O 2 Group BRO 3 X 21 O 4 Group CRO 5 X 12 O 6 Group DRO 7 X 22 O 8 2 x 2 Factorial with a Pretest Posttest

Copyr ight © 2000 by Harcourt, Inc. All rights reserved 2 x 2 Factorial Design with a Posttest Measure Group ARX 11 O 1 Group BRX 21 O 2 Group CRX 12 O 3 Group DRX 22 O 4

Copyr ight © 2000 by Harcourt, Inc. All rights reserved A Test Market Experiment on Pricing Sales in Units (thousands) Regular Price $ X 1 = X= Reduced Price $ X 2 = Cents-Off Coupon Regular Price X 1 = Test Market A, B, or C Test Market D, E, or F Test Market G, H, or I Test Market J, K, or L Mean Grand Mean

Copyr ight © 2000 by Harcourt, Inc. All rights reserved LATIN SQUARE DESIGN A balanced, two-way classification scheme that attempts to control or block out the effect of two or more extraneous factors by restricting randomization with respect to the row and column effects.

Copyr ight © 2000 by Harcourt, Inc. All rights reserved ABC 2BCA 3CAB Order of Usage SUBJECT

Copyr ight © 2000 by Harcourt, Inc. All rights reserved TEST MARKETING Controlled experimentation Not just trying somethingout But scientific testing

Copyr ight © 2000 by Harcourt, Inc. All rights reserved TEST MARKETING AN EXPERIMENTAL PROCEDURE THAT PROVIDES AN OPPORTUNITY TO TEST A NEW PRODUCT OR A NEW MARKETING PLAN UNDER REALISTIC MARKET CONDITIONS TO MEASURE SALES OR PROFIT POTENTIAL

Copyr ight © 2000 by Harcourt, Inc. All rights reserved FUNCTONS OF TEST MARKETING ESTIMATEOUTCOMES IDENTIFY AND CORRECTWEAKNESSES IN PLANS

Copyr ight © 2000 by Harcourt, Inc. All rights reserved A Lengthy and Costly Procedure $$$$$ Loss of Secrecy When not to Test? How Long Should a Test Last?

Copyr ight © 2000 by Harcourt, Inc. All rights reserved SELECTING A TEST MARKET POPULATION SIZE DEMOGRAPHIC COMPOSITION LIFESTYLE CONSIDERATIONS COMPETITIVE SITUATION MEDIA SELF-CONTAINED TRADING AREA OVERUSED MARKETS - SECRECY

Copyr ight © 2000 by Harcourt, Inc. All rights reserved CONTROL METHOD OF TEST MARKETING SMALL CITY LOW CHANCE OF BEING DETECTED DISTRIBUTION IS FORCED (GUARANTEED)

Copyr ight © 2000 by Harcourt, Inc. All rights reserved The advantages of using the control method of test marketing Reduced costs Shorter time period needed for reading test market results Increased secrecy from competitors No distraction of company salespeople from regular product lines

Copyr ight © 2000 by Harcourt, Inc. All rights reserved SOME PROBLEMS ESTIMATING SALES VOLUME OVERATTENTION UNREALISTIC STORE CONDITIONS READING COMPETIVE ENVIRONMENT INCORRECTLY INCORRECT VOLUME FORECASTS –ADJUSTED DATA –PENETRATION AND REPEAT PURCHASE RATE TIME LAPSE

Copyr ight © 2000 by Harcourt, Inc. All rights reserved High Tech Test Markets ElectricTestMarkets SimulatedTestMarkets Virtual-realitySimulated Test Markets