Chapter 9 Causal Research Designs and Test Markets

Types of Experiments Definitions – two broad types: Laboratory experiments – take place in an artificial setting where extraneous variables can be controlled. Field experiments – take place in a natural setting but extraneous variables cannot be controlled. 9-2

Field experiments – issues to consider Realism Control Time frame Cost Competitive reactions 9-3

Types of Variables in Experimental Designs Independent Variables Types of Variables in Experimental Designs Dependent Variables Variables manipulated by researcher Control Variables Measures of effects/outcomes Extraneous Variables Conditions that we measure to better understand the true effects Uncontrolled, unmeasured variables that may affect the dependent variable 9-4

Validity and Reliability Concerns Internal Validity Reliability External Validity 9-5

Experiments – Validity External validity – the extent to which a causal relationship is true for the defined target population. Internal validity – the extent to which the research design accurately identifies causal relationships. Validity – the extent to which the conclusions drawn from the experiment are true. Notes: 9-6

Threats to Internal Validity History Maturation Testing Instrumentation Selection Bias Statistical Regression Mortality Notes: 9-7

Threats to External Validity Treatment vs. Treatment Treatment vs. Testing Treatment vs. Selection Treatment vs. Setting Treatment vs. History Notes: 9-8

The Nature Of Experimentation Construct validity . . . the extent to which the variables under investigation are completely and accurately identified prior to hypothesizing any functional relationships. (Are we measuring what we think we are measuring?) 9-9

Threats to Construct Validity Inadequate measurement of the variables One measurement, one methodology bias Evaluation apprehension – respondent concern of other knowing their responses Demand characteristics – respondents trying to guess the desired response Diffusion of treatment – respondents causing bias by discussing the treatment with others. Notes: 9-10

Improving Experimental Validity Time order of the manipulation exposure. Including control groups Exclusion of non-similar test subjects. Matching extraneous variables. Randomization of test subjects to treatment groups. Notes: 9-11

Symbols Used in Experimental Research Designs X = exposure of an independent variable to a group of test subjects for which the effects are to be determined. O = observation or measurement of the dependent variable (outcome) for the test subjects. [R] = random assignment of test subjects to separate groups. EG = the experimental group of test subjects. CG = the control group of test subjects. → = movement through time shown as left-to-right movement. Vertical alignment of symbols = activities that occur simultaneously. Horizontal alignment of symbols = refer to a specific group of subjects. 9-12

Pre-Experimental Designs One-Shot Study (EG): X → O1 One-Group, Pretest-Posttest (EG): O1 → X1 → O2 Static Group Comparison (CG): O2 9-13

True Experimental Designs Posttest-Only, Control Group (Good) (EG): [R] X → O1 (CG): [R] O2 Pretest-Posttest, Control Group (Better) (EG): [R] O1 → X → O2 (CG): [R] O3 → O4 Solomon Four Group (Best) Design 1 (CG): [R] O3 → O4 Design 2 (EG): [R] X → O5 (CG): [R] O6 9-14

Quasi-Experimental Designs Non-Equivalent Control Group Group 1 O1 → X → O2 Group 2 O3 O4 Separate Sample, Pretest-Posttest Sample 1 O1 → (X) Sample 2 (X) → O2 9-15

Test Marketing . . . use of a controlled field experiment to collect information on market performance indicators. Pros: Is most accurate method of forecasting future sales. Provides an opportunity to pretest marketing mix variables (particular PRICE). Cons: Expensive. Exposes the new product or service to competitors. Takes time to conduct.     9-16

9-17

Criteria for Selecting Test Market Locations Representativeness: Do demographics match the total market? Degree of isolation: Phoenix and Tulsa are isolated markets – thus better; Los Angeles is not isolated. Ability to control distribution and promotion: Are there preexisting arrangements to distribute the new product in selected channels of distribution? Are local media prepared to test variations in promotional messages? 9-18