1 Experimental Research An Overview Professor Gary Merlo Chapter 11

2 The Nature of Experiments Purpose of experimental research is to allow the investigator to control the research situation so that causal relationships among variables may be evaluated. In experiment one variable, (the independent variable) is manipulated and its effect upon another variable (the dependent) is measured. Other variables that may have influence upon the dependent are eliminated or controlled.

Experiment A research investigation in which conditions are controlled One independent variable is manipulated (sometimes more than one) Its effect on a dependent variable is measured To test a hypothesis

Manipulation of the Independent Variable Selection of Dependent Variable Assignment of Subjects (or other Test Units) Control Over Extraneous Variables Basic Issues of Experimental Design

5 Unit Pricing Illustration Purpose of unit pricing is to avoid confusion due to price calculations. Some evidence that unit pricing has failed to change consumer’s purchasing habits. Current form of unit price display is a separate shelf tag for each item. This type of display may not facilitate price comparisons.

6 Unit Pricing Illustration (continued) A survey may result in bias –Who would admit they did not use unit pricing when shopping. Randomly assign different experimental conditions to stores, this would minimize the differences inherent in the stores. Results show, a change in the presentation of unit price information caused a change in average price paid. Controlled for effect that store image could cause by using only on chain store in the experiment.

7 Results of Unit Pricing Experiment Decrease in price regardless of method of pricing However, greater decrease using list

8 Basic Issues Decisions that must be made about four basic elements of an experiment: –A. Manipulation of Independent Variable (IV) Experimenter has some degree of control over IV IV is hypothesized to be the causal influence Experimental treatments are the alternative manipulations of independent variable being investigated Experimental and Control Groups: –In simplest form only two values of the independent variable are manipulated. –By holding control group at zero (unchanged) can increase measurement of independent variable on experimental group. –May want more than two groups in experiment to get better understanding of relationships. –May want to investigate relationships on changes on more than one independent variable at the same time.

9 Basic Issues (continued) B. Selection of Measurement of Dependent Variable –Dependent variable is the criterion or standard by which the results are judged. –Careful consideration of an appropriate dependent variable is required. –For example: in a test market the time period for the effects to become evident should be carefully considered. (Buy a loser once, may not retry)

10 Basic Issues (continued) C. Selection and Assignment of Test Units –Test units are the subjects or entities whose responses to the experimental treatment are measured or observed. –People are the most common test units Sample selection error may occur because of the procedure utilized to assign subjects or test units to either the experimental group or the control group. Random sampling error occurs if repetitions of the basic experiment sometimes favor on experimental condition and sometimes the other by chance. However, random is still the best method of assignment.

11 Basic Issues (continued) C. Selection and Assignment of Test Units (continued) –Matching respondents on the basis of pertinent background information is another technique for controlling assignment errors. Examples: age, income, education (whatever is hypothesized to influence the dependent variable) –Repeated measure of same subjects eliminates possibility of differences in subjects but potentially the exposure may cause other problems.

12 Basic Issues (continued) D. Control over Extraneous Variable –Need to understand the types of experimental error. –Errors in research can be classified into two basic categories. Random error (similar to sampling error) Constant error (extraneous variables or conditions of administration are allowed to have an influence on dependent variables. –Example: Subjects in experimental group are always administered the treatment in the morning, testing fuel efficiency using different size car engines. –Time of day in this case is the constant error. Uncontrolled extraneous variable. –Constant error distorts the results in a particular direction, results are erroneous difference masks the results of the experiment.

13 Basic Issues (continued) D. Control over Extraneous Variable –Demand Characteristics Design characteristics that unintentionally hint to subjects about the experimenter’s hypothesis. If participants recognize experimenter’s expectation (demand), they are likely to act in a manner consistent with the experimental treatment. Demand characteristic can be person administering experiment, body language. Guinea pig effect - subjects change normal behavior or attitudes in order to cooperate.

14 Basic Issues (continued) D. Control over Extraneous Variable –Hawthorne Effect Researchers were attempting to study the effects on productivity of various working conditions. Regardless of treatment, productivity increased. Demonstrates that people behave differently when they know they are part of an experiment. Often disguise purpose of experiment to eliminate this effect.

15 Basic Issues (continued) D. Control over Extraneous Variable –Establishing Control If brand image and preference may influence outcome of taste test, package experiment in plain packages. If eliminating of extraneous variable is not possible, maintain constant conditions. –May require several tests and take average to determine results. If the order presented could influence results, counterbalancing attempts to limit this effect.

16 Basic Issues (continued) D. Control over Extraneous Variable Blinding –Control’s subjects knowledge of whether or not they have been given a particular treatment. Double-blind designs –Neither the experimenter nor the respondent know who receives the experiment. –Used when researcher might influence outcome.

17 Ethical Issues in Experimentation Experimental subjects should be fully informed and receive accurate information. However, researchers intentionally use deception to avoid respondents knowing the brand. In such situations it is necessary to debrief subject.

18 Fundamental Questions in Experimentation Basic experimental designs a single independent variable is manipulated. Simultaneous manipulation of the several independent variable is referred to as factorial design.

19 Fundamental Questions in Experimentation (continued) Laboratory experiment –Almost complete control over the research setting. –Tachistoscope allows researcher to experiment with the visual impact of advertising, packaging, etc. by controlling the amount of time a visual image is exposed to a subject.

20 Fundamental Questions in Experimentation (continued) Field Experiment –Natural setting –Used to tune a marketing strategy Controlled store test –Products put into a stores in a number of small cities or into selected supermarket chains. –Hybrid between laboratory and field experiment.

Laboratory ExperimentField Experiment Artificial-Low Realism Few Extraneous Variables High control Low Cost Short Duration Subjects Aware of Participation Natural-High Realism Many Extraneous Variables Low control High Cost Long Duration Subjects Unaware of Participation C opyright © 2000 by Harcourt, Inc. All rights reserved.

22 Issues of Experimental Validity Internal Validity –Refers to the question of whether the experimental treatment was the sole cause of observed changes in the dependent variable. Extraneous Variables that may jeopardize internal validity. –History Specific events in the external environment between the first and second measurements that are beyond the experimenter’s control.

23 Issues of Experimental Validity continued) Cohort Effects –Change in dependent variable that occurs because the members of one experimental group experienced different historical situations than members of the other experimental groups. –Maturation Change in people over time, function of time rather than function of independent variable. (hunger, tired)

24 Issues of Experimental Validity continued) Testing Effects –Pretesting effects because the initial measurement or test alerts respondents to the nature of the experiment and respondents may act differently than they would have if no pretest measures were taken. –If identical measuring instrument is used more than once there may be a testing effect. May use alternate form of testing for second test. May result in instrument effect

25 Issues of Experimental Validity continued) Instrument decay or variation –Selection: sample bias resulting in differential selection of respondents for comparison groups. –Mortality: (sample attrition that takes place over time)

26 Issues of Experimental Validity continued) External Validity –Ability of the research to generalize the results of the experiment to the marketplace. Must have internal validity to have external –Example: Student surrogates used a subjects Trade-offs between internal and external validity –Natural experiments tend to have greater external validity. –Laboratory experiments have higher internal validity.

27 Classification of Experimental Designs Various types of experimental designs. If only one variable is manipulated, basic experimental design. Symbolism for Diagramming Experimental Designs –X = exposure of a group to an experiment treatment –O = observation of measurement 0 1 = first observation 0 2 = second observation R = random assignment of test units

28 Three Examples of Quasi- Experimental Design Not true experiments because they do not adequately control for problems associated with loss of external or internal validity.

29 X0101 Subjects participate because of voluntary self-selection or arbitrary assignment. Lacks control of extraneous influences. Under certain circumstances may be only alternative.

30 One-Shot Design Internal Validity Problems History –weak Maturation –weak Testing –not relevant Instrumentation –not relevant Selection –weak Mortality –weak C opyright © 2000 by Harcourt, Inc. All rights reserved.

31 X Measurement before and after but no control group. If change over time, lacks control. Lacks control of history No control for mortality Lacks control for testing Frequently used in marketing because of the time and cost = Experiment

32 One-Group Pretest-Posttest Internal Validity Problems History –weak Maturation –weak Testing –weak Instrumentation –weak Selection –controlled Mortality –controlled C opyright © 2000 by Harcourt, Inc. All rights reserved.

33 Experimental Group: X0101 Control Group: 0202 Lack of assurance that two groups were equal during start Mortality is not controlled = Results of Experiment Used frequently when before measurement is not possible

34 Static-Group Design Internal Validity Problems History –controlled Maturation –possible source of concern Testing –controlled Instrumentation –controlled Selection –weak Mortality –weak C opyright © 2000 by Harcourt, Inc. All rights reserved.

35 Improved Experimental Designs Experimental Group: Control Group: R R 0101 X Effect of the treatment is: ( ) - ( ) Effect of extraneous variables is same on both groups Example, both groups receive pretest, same for history, maturation, testing effects, instrument decay, and regression effects. However, interactive testing effect not controlled.

36 Experimental Group: Control Group: R R The effect of the experimental group is Used when a pretest is difficult or impossible X With posttest measurement, the effects of testing and instrument decay are eliminated. Assumptions about extraneous variables are made the same.

37 Experimental Group I: Control Group I: Experimental Group II: Control Group II: R R R R 0101 X X Controls for interactive testing effect Estimates Results of experiment Estimates testing effect because of repeated measure Measurement without pretest ( ) - ( ) - ( ) = Maximum Control of Extraneous Variables

38 Compromise Designs Often in marketing research true experimentation is not possible. Often must sacrifice random assignment

X Vulnerable to historical changes - population, attitudes, economic patterns, etc. Advantage is to determine temporary from permanent changes.

Degree of Change Time X

Degree of Change Time X

Degree of Change Time X

Degree of Change Time X

44 End of Chapter 11