1. Market Intelligence Session 7 Experimental Research

2. Experiments Only way to test causal hypotheses Independent Variable = hypothesized cause – Usually manipulated by the researcher/manager – Example: Send a color or black and white brochure Dependent Variable = effect – Measured (observed) by researcher/manager – Example: New accounts secured 2

3. 3 key features of true experiments 1.Manipulation of a variable 2.Control/comparison group 3.Random assignment to groups 3

4. Can’t always run true experiment Sometimes can’t manipulate (or ethically manipulate) variable of interest (smoking) Sometimes can’t get a comparison group beforehand (all people affected by event) Can’t always randomly assign people to groups (which class people take) 4

5. Can’t always run true experiment Sometimes can’t manipulate (or ethically manipulate) variable of interest Sometimes can’t get a comparison group beforehand Can’t always randomly assign people to groups Solution: Correlational or Quasi-experimental designs – Goal: get as close to true experiment as possible 5

6. Types of designs Correlational (cross-lag panel) Quasi-experimental – One group posttest only – One group pretest-posttest – Nonequivalent control posttest only – Nonequivalent control pretest-posttest – Time series – Non-equivalent control time series True experiments – Posttest equivalent groups – Pretest-posttest equivalent groups 6

7. Notation (used across social sciences) X = Manipulation – No X means that group did not receive manipulation R = Random assignment to different experimental groups or conditions O n = Observation of DV at Time N – O 1 = before manipulation – O 2 = after manipulation 7

8. Types of designs Correlational (cross-lag panel) Quasi-experimental – One group posttest only – One group pretest-posttest – Nonequivalent control posttest only – Nonequivalent control pretest-posttest – Interrupted time series – Non-equivalent control time series True experiments – Posttest equivalent groups – Pretest-posttest equivalent groups 8

9. Correlational designs Examine correlations between 2 variables Most common: Cross lag panels – Examine correlations between 2 variables at 2 time points – Purpose: to see if evidence supports 1 causal direction more than the other – Notation: O 1 O 2 9

10. Key: Look at diagonal correlations 10

11. Example: Lefkowitz et al. (1972) 11.21.01.38.05.31.-.05 10 years TV Violence Aggression

12. Example: Lefkowitz et al. (1972) 12.21.38.05 -.05 10 years TV Violence Aggression

13. Example: Lefkowitz et al. (1972) 13.21.01.38.05.31 -.05 10 years TV Violence Aggression

14. Types of designs Correlational (cross-lag panel) Quasi-experimental – One group posttest only – One group pretest-posttest – Nonequivalent control posttest only – Nonequivalent control pretest-posttest – Interrupted time series – Non-equivalent control time series True experiments – Posttest equivalent groups – Pretest-posttest equivalent groups 14

15. One group posttest only XO 2 15

16. Types of designs Correlational (cross-lag panel) Quasi-experimental – One group posttest only – One group pretest-posttest – Nonequivalent control posttest only – Nonequivalent control pretest-posttest – Interrupted time series – Non-equivalent control time series True experiments – Posttest equivalent groups – Pretest-posttest equivalent groups 16

17. One group pretest-posttest O 1 XO 2 17

18. Types of designs Correlational (cross-lag panel) Quasi-experimental – One group posttest only – One group pretest-posttest – Nonequivalent control posttest only – Nonequivalent control pretest-posttest – Interrupted time series – Non-equivalent control time series True experiments – Posttest equivalent groups – Pretest-posttest equivalent groups 18

19. Nonequivalent control posttest only XO 2 O 2 19

20. Types of designs Correlational (cross-lag panel) Quasi-experimental – One group posttest only – One group pretest-posttest – Nonequivalent control posttest only – Nonequivalent control pretest-posttest – Interrupted time series – Non-equivalent control time series True experiments – Posttest equivalent groups – Pretest-posttest equivalent groups 20

21. Nonequivalent control pretest-posttest O 1 XO 2 O 1 O 2 21

22. Types of designs Correlational (cross-lag panel) Quasi-experimental – One group posttest only – One group pretest-posttest – Nonequivalent control posttest only – Nonequivalent control pretest-posttest – Interrupted time series – Non-equivalent control time series True experiments – Posttest equivalent groups – Pretest-posttest equivalent groups 22

23. Interrupted time series O 1 O 1 O 1 XO 2 O 2 O 2 23

24. 24

25. Removal of treatment 25

26. Nonequivalent control time series O 1 O 1 O 1 XO 2 O 2 O 2 O 1 O 1 O 1 O 2 O 2 O 2 26

27. Adding control condition 27

28. 28

29. Threats to internal validity Selection bias: people in different groups/conditions may be different (because groups occurred naturally) History: an event occurring around same time as manipulation that has nothing to do with manipulation Maturation: people change over time Testing: repeatedly testing can change responses Differential attrition: when attrition is related to condition No control/baseline: nothing to compare it to 29

30. Types of designs Correlational (cross-lag panel) Quasi-experimental – One group posttest only – One group pretest-posttest – Nonequivalent control posttest only – Nonequivalent control pretest-posttest – Interrupted time series – Non-equivalent control time series True experiments – Posttest equivalent groups – Pretest-posttest equivalent groups 30

31. True Experimental Designs Posttest equivalent groups RXO 2 RO 2 Pretest posttest equivalent groups RO 1 XO 2 RO 1 O 2 31

32. Breckenridge Brewery Ad Breckenridge Brewery wants to assess the efficacy of TV ad spots for its new Amber Ale. X: Two weeks of ads for Breckenridge Ale. O: Give survey on beer brands purchased over past week. 32

33. Match up the 8 designs Quasi-experimental – One group posttest only – One group pretest-posttest – Nonequivalent control posttest only – Nonequivalent control pretest-posttest – Time series – Non-equivalent control time series True experiments – Posttest equivalent groups – Pretest-posttest equivalent groups 33

34. Breckenridge Amber Ale O2 Mean = 1.3 Packs per Week X Two Weeks of Ads for Breckenridge Amber Ale X Two Weeks of Ads for Breckenridge Amber Ale O2 Mean = 0.5 Packs per Week Difference b/w cities = 0.8 Durham Chapel Hill Design? ______________

35. Breckenridge Amber Ale O2 Mean = 1.3 Packs per Week X Two Weeks of Ads for Breckenridge Amber Ale X Two Weeks of Ads for Breckenridge Amber Ale O2 Mean = 0.5 Packs per Week Difference b/w cities = 0.8 Durham Chapel Hill Design? Nonequivalent control posttest only

36. Mean Breckenridge consumption (packs per week) 36 Design? ______________

37. Mean Breckenridge consumption (packs per week) 37 Design? Interrupted Time series

38. Breckenridge Amber Ale O1 O2 Mean = 0.2 Packs per Week Mean = 1.3 Packs per Week X Two Weeks of Ads for Breckenridge Amber Ale X Two Weeks of Ads for Breckenridge Amber Ale O1 Mean = 0.3 Packs per Week O2 Mean = 0.5 Packs per Week Δ=1.1 Δ=0.2 ΔΔ=0.9 Experimental Group (Randomly Assigned) Control Group (Randomly Assigned) Design? ______________

39. Breckenridge Amber Ale O1 O2 Mean = 0.2 Packs per Week Mean = 1.3 Packs per Week X Two Weeks of Ads for Breckenridge Amber Ale X Two Weeks of Ads for Breckenridge Amber Ale O1 Mean = 0.3 Packs per Week O2 Mean = 0.5 Packs per Week Δ=1.1 Δ=0.2 ΔΔ=0.9 Experimental Group (Randomly Assigned) Control Group (Randomly Assigned) Design? Pretest-posttest equivalent groups

40. Breckenridge Amber Ale O2O2 O2O2 Mean = 0.16 Packs per Week X Two Weeks of Ads for Breckenridge Amber Ale X Two Weeks of Ads for Breckenridge Amber Ale Design? ______________

41. Breckenridge Amber Ale O2O2 O2O2 Mean = 0.16 Packs per Week X Two Weeks of Ads for Breckenridge Amber Ale X Two Weeks of Ads for Breckenridge Amber Ale Design? One group posttest only

42. Breckenridge Amber Ale O1 O2 Mean = 0.2 Packs per Week Mean = 1.3 Packs per Week X Two Weeks of Ads for Breckenridge Amber Ale X Two Weeks of Ads for Breckenridge Amber Ale O1 Mean = 0.3 Packs per Week O2 Mean = 0.5 Packs per Week Δ=1.1 Δ=0.2 ΔΔ=0.9 Durham Chapel Hill Design? ______________

43. Breckenridge Amber Ale O1 O2 Mean = 0.2 Packs per Week Mean = 1.3 Packs per Week X Two Weeks of Ads for Breckenridge Amber Ale X Two Weeks of Ads for Breckenridge Amber Ale O1 Mean = 0.3 Packs per Week O2 Mean = 0.5 Packs per Week Δ=1.1 Δ=0.2 ΔΔ=0.9 Durham Chapel Hill Design? Nonequivalent control pretest-posttest

44. Breckenridge Amber Ale O2 Mean = 1.3 Packs per Week X Two Weeks of Ads for Breckenridge Amber Ale X Two Weeks of Ads for Breckenridge Amber Ale O2 Mean = 0.5 Packs per Week Difference between groups=0.8 Experimental Group (Randomly Assigned) Control Group (Randomly Assigned) Design? ______________

45. Breckenridge Amber Ale O2 Mean = 1.3 Packs per Week X Two Weeks of Ads for Breckenridge Amber Ale X Two Weeks of Ads for Breckenridge Amber Ale O2 Mean = 0.5 Packs per Week Difference between groups=0.8 Experimental Group (Randomly Assigned) Control Group (Randomly Assigned) Design? Posttest equivalent groups

46. Mean Breckenridge consumption (packs per week) 46 Design? ______________

47. Mean Breckenridge consumption (packs per week) 47 Design? Nonequivalent control time series

48. Breckenridge Amber Ale O1O1 O1O1 O2O2 O2O2 Mean = 0.2 Packs per Week Mean = 1.3 Packs per Week X Two Weeks of Ads for Breckenridge Amber Ale X Two Weeks of Ads for Breckenridge Amber Ale Δ=1.1 Design? ______________

49. Breckenridge Amber Ale O1O1 O1O1 O2O2 O2O2 Mean = 0.2 Packs per Week Mean = 1.3 Packs per Week X Two Weeks of Ads for Breckenridge Amber Ale X Two Weeks of Ads for Breckenridge Amber Ale Δ=1.1 Design? One group pretest-posttest

50. Experiments - Factorial Designs 2 or more independent variables (manipulated and/or measured), each with two or more levels. – Type 1: 2 marketing mix variables Both variables manipulated Important for determining whether you need to coordinate marketing actions – Type 2: “tactical segmentation” (1 segment responds differently to a marketing mix variable than another segment) Segmenting variable is measured, marketing action is manipulated Important for determining whether you should segment for that particular marketing action 50

51. Experiments - Factorial Designs What to look for in factorial designs – Is there a main effect of A? – Is there a main effect of B? – Key: Is there an interaction between A and B? (interaction: effect of one IV on DV depends on level of another IV) Analysis – Eye-ball method – Analysis of Variance (ANOVA) 51

52. Type 1: 2 marketing mix variables Assume two of you manage the Oreo account at Kroger. – One manages advertising, one manages in store promotions like end-of-aisle display You have been asked to evaluate whether ads and/or end-of-aisle display would increase sales … Advertising End of Aisle

53. Oreo Promotion Experiment Kroger: Supporting a discount on Oreo cookies Factor A: Ads in local paper a1 = no ads a2 = ad in Thursday local paper Factor B: Display location b1 = regular shelf b2 = end aisle 53

54. OREO PROMOTION EXPERIMENT Scenario 1 (EXPENDITURES/CUSTOMER/2 WKS) 54

55. OREO PROMOTION EXPERIMENT Scenario 1 (EXPENDITURES/CUSTOMER/2 WKS) 55 Main effect of A? Main effect of B?

56. OREO PROMOTION EXPERIMENT Scenario 1 (EXPENDITURES/CUSTOMER/2 WKS) 56 Interaction? this diff vs. this diff

57. OREO PROMOTION EXPERIMENT Scenario 1 (EXPENDITURES/CUSTOMER/2 WKS) 57 Interaction? this diff vs. this diff

58. SALES OF OREOS 58

59. SALES OF OREOS 59 2 main effects, no interaction

60. SALES OF OREOS 60 Do they need to coordinate to make their decisions?

61. Oreo Promotion Experiment Scenario 2 (Expenditures/customer/2 wks) 61 0.950.70 0.75 0.825 0.800.775

62. Oreo Promotion Experiment Scenario 2 (Expenditures/customer/2 wks) 62 0.950.70 0.75 0.825 0.800.775 Main effect of A? Main effect of B?

63. Oreo Promotion Experiment Scenario 2 (Expenditures/customer/2 wks) 63 0.950.70 0.75 0.825 0.800.775 Interaction?

64. 64 SALES OF OREOS (Expenditures/customer/2 wks)

65. 65 “Cross-over” interaction SALES OF OREOS (Expenditures/customer/2 wks)

66. 66 SALES OF OREOS (Expenditures/customer/2 wks) Do they need to coordinate to make their decisions?

67. Oreo Promotion Experiment Scenario 3 (Expenditures/customer/2 wks) 67 1.30

68. Oreo Promotion Experiment Scenario 3 (Expenditures/customer/2 wks) 68 1.30 Main effect of A? Main effect of B?

69. Oreo Promotion Experiment Scenario 3 (Expenditures/customer/2 wks) 69 Interaction? 1.30

70. SALES OF OREOS (Expenditures/customer/2 wks) 70

71. SALES OF OREOS (Expenditures/customer/2 wks) 71 “fan effect” interaction

72. SALES OF OREOS (Expenditures/customer/2 wks) 72 Do they need to coordinate to make their decision?

73. Oreo Example No A x B interaction – Effect of changing A (Ads) is independent of level of B (Display Location). – Implies that Ad & Display decisions can be decoupled…they influence sales additively A x B interaction – Effect of changing A (ads) depends on level of B (display location), and/or vice-versa – Fan effect: Cannot decouple variables – Cross-over: Cannot decouple variables 73

74. Type 2: Tactical Segmentation Should groups be treated same or differently with respect to specific marketing decision variable? A is a controllable decision variable and B is a potential segmentation variable – Interaction means that segments respond differently to this marketing lever. – Example: coupons x urban/suburban Question: does marketing mix variable have bigger effect for segment A or B? Is coupon more effective in urban or suburban neighborhoods? 74

75. Interactions and segmentation 75

76. Interactions and segmentation 76 Coupons have a bigger effect in the suburbs

77. Tactical Segmentation Example - Dog Food 1 potential segmentation variable (Size of Dog) 2 decisions – Price: Hi v. Lo – Ad Theme: “Love between dog and owner” vs. “Dog’s Active Life” 77

78. Segmentation Example: Dog Food I (rated on 10 pt scale) Price Advertising 78

79. Segmentation Example: Dog Food I Price Advertising 79

80. 80

81. Implications of Contrast A variable that is an excellent basis for segmentation with respect to one decision about a marketing mix element may be a poor basis for segmentation with respect to another mix element For any given mix element decision, when evaluating alternative bases for segmentation, look for ones with big differences in sensitivity to mix variable. 81

82. Tactical Segmentation II Example - Dog Food 1 decision: Price (hi vs. lo) 2 potential segmentation variables – Size of Dog – Income of owner 82

83. Segmentation Example: Dog Food II 83 How to compare? ANOVA

84. Segmentation Example: Dog Food II 84 Eye-ball method: compare difference of differences

85. Segmentation Example: Dog Food II 85 Price

86. Garlic Chopper Prototype Breakout 86 Survey: what do you want to know? How to structure survey? What type(s) of scales to use? Can randomly assign to conditions – What would you want to manipulate/measure?

87. For next time IBM case (team assignment due) Guest lecture: Kevin Clark Due: Cola conjoint assignment Not due: “product line scenarios” Quiz 2 next Friday – Study guide will be on Sakai shortly 87