Experimental Design

Talking While Driving: Are Cell Phone Conversations Different from Passenger Conversations? Research Problems Talking on a cellphone leads to distracted driving. But is it any more dangerous than talking to a passenger? funny-pictures.picphotos.net

Testing Web Protocols for Financial Markets Researchers designed a laboratory experiment to compare performance of two electronic protocols (FIX and SOAP). FIX performed better Image: RB: Technology has drastically changed financial services

SUBJECTS EXPERIMENTAL GROUP 1 EXPERIMENTAL GROUP 2 PERFORMANCE COMPARE FIX SOAP IV DV LAB

Essential Features of Experimental Research IVs Experimental groups Control groups Situational Task Instructional Manipulated

Experimental Graph Showing Results Within Each Condition 12–6

12–7 Main Effect The experimental difference in dependent variable Interaction Effect Differences in dependent variable means due to a specific combination of independent variables

12–8 Manipulation of the Independent Variable 1.Experimental treatment: the way an experimental variable is manipulated. Categorical variables: described by class or quality Continuous variables: described by quantity (level) 2.Experimental Group 3.Control Group Minimize Experimental Error

12–9 Designing an Experiment More than One Independent Variable Repeated Measures Treatment 1 Treatment 2 Treatment 3 Treatment 1 Treatment 2 Treatment 3

12–10 Designing an Experiment (cont’d) Sample Selection And Random Sampling Errors Systematic or nonsampling error Subject selection, experimental design, and unrecognized extraneous variables Overcoming sampling errors Randomization Matching Control over extraneous variables

12–11 Designing an Experiment (cont’d) Sample Selection And Random Sampling 1.Errors Experimental Confound 2.Extraneous variables

Controlling extraneous variables Confounds: Any uncontrolled extraneous variable What to do???

12–13 Demand Characteristics 1.Demand Characteristic  An experimental design element or procedure that unintentionally provides subjects with hints about the research hypothesis. 2.Demand Effect  Occurs when demand characteristics actually affect the dependent variable. 3.Hawthorne Effect  People will perform differently from normal when they know they are experimental subjects.

12–14 Reducing Demand Characteristics 1.Use an experimental disguise 2.Isolate experimental subjects 3.Use a “blind” experimental administrator 4.Administer only one experimental treatment level to each subject

12–15 Establishing Control Constancy of Conditions Subjects in all experimental groups are exposed to identical conditions except for the differing experimental treatments. Counterbalancing Attempts to eliminate the confounding effects of order of presentation by varying the order of presentation (exposure) of treatments to subject groups.

12–16 Ethical Issues in Experimentation Debriefing experimental subjects – Communicating the purpose of the experiment – Explaining the researcher’s hypotheses Attempts to interfere with a competitor’s test- marketing efforts – Such acts as changing prices or increasing advertising to influence (confound) competitors’ test-marketing results are ethically questionable.

Practical Experimental Design Issues Basic versus Factorial Experimental Designs Basic experimental designs Factorial experimental design Basic versus Factorial Experimental Designs Basic experimental designs Factorial experimental design IV DV IV IV… 1

Practical Experimental Design Issues Laboratory Experiment A situation in which the researcher has more complete control over the research setting and extraneous variables Laboratory Experiment A situation in which the researcher has more complete control over the research setting and extraneous variables 2

Practical Experimental Design Issues Field Experiments Research projects involving experimental manipulations that are implemented in a natural environment. Field Experiments Research projects involving experimental manipulations that are implemented in a natural environment. 3

The Hidden in Hidden Valley Ranch The company tested three new flavors, and all of them sold out within three days Not so fast…a competitor bought them all! Illustrates the risk associated with field tests

The Artificiality of Laboratory versus Field Experiments 12–21

Between-Subjects Designs vs Within-Subjects Design

repeated-measures designs

Within-Subjects Designs Main problem to solve  order effects Progressive Carry-over (harder to control) Sequence A-B may yield differ carryover than the sequence B-A

The Validity of Experimental Research External validity The extent to which research findings generalize to contexts other than those of the experiment 1.generalize to other populations 2.generalize to other environments 3.generalize to other times

The Validity of Experimental Research Statistical conclusion validity Proper statistical analyses and conclusions Construct validity Well-chosen and well-defined IVs and DVs

Internal validity Does the experimental manipulation truly cause changes in the specific outcome of interest? Internally valid studies… 1.have valid operational definitions. 2.have valid measurements. 3.have no confounds. The Validity of Experimental Research

Threats to Internal Validity Studies extending over time (may have pretests) 1.History 2.Maturation 3.Testing and instrumentation Participant problems 1.Subject selection 2.Attrition

Measuring dependent variables (DVs) DVs are any behaviors measured in an experiment Problems: Ceiling effects vs floor effects Solution: Task of moderate difficulty, determined through pilot testing Essential Features of Experimental Research

12–31 Diagramming Experimental Designs: Symbols Classification of Experimental Designs

12–32 Quasi-Experimental Designs Experimental designs that do not involve random allocation of subjects to treatment combinations.

One Shot Design (After Only): X O 1 One Group Pretest–Posttest O 1 X O 2 Static Group Design Experimental X O 1 Control O 2

12–34 Pretest–Posttest Control Group Design (Before–After with Control) ExperimentalR O 1 X O 2 ControlR O 3 X O 4 Posttest Only Control Group (After-Only with Control) ExperimentalR X O 1 ControlR O 2

Complex Experimental Designs Factorial Design An experiment that investigates the interaction of two or more independent variables on a single dependent variable.

12–36 Effects in Factorial Design Main effect The influence of a single independent variable on a dependent variable. Interaction effect The influence on a dependent variable by combinations of two or more independent variables. Interaction occurs if the effect of one treatment differs at various levels of the other treatment.

Graphic Illustration of Interaction Between Sex and Advertising Copy