Experimental Research
Take some action and observe its effects Take some action and observe its effects Extension of natural science to social science Extension of natural science to social science Best for limited and well defined concepts Best for limited and well defined concepts Useful for hypothesis testing - need theory Useful for hypothesis testing - need theory Focus on determining causation, not just description Focus on determining causation, not just description
Components of Experiment Three components: Three components: Independent and dependent variables Independent and dependent variables Effects of stimulus on some outcome variable Effects of stimulus on some outcome variable Pretesting and posttesting Pretesting and posttesting Ability to assess change before and after manipulation Ability to assess change before and after manipulation Experimental and control groups Experimental and control groups Comparison group that does not get stimulus Comparison group that does not get stimulus
Experimental and Control Groups Must be as similar as possible. Must be as similar as possible. Control group represents what the experimental group would have been like had it not been exposed to the experimental stimulus. Control group represents what the experimental group would have been like had it not been exposed to the experimental stimulus.
Selecting Subjects Probability sampling Probability sampling Randomization Randomization Most statistics used to analyze results assume randomization of subjects. Most statistics used to analyze results assume randomization of subjects. Randomization only makes sense if you have a reasonably large pool of subjects. Randomization only makes sense if you have a reasonably large pool of subjects.
Pre-Experimental Designs On-Shot Case Study On-Shot Case Study One Group Pretest- One Group Pretest- Posttest Design Static Group Static Group Comparison Comparison
True Experimental Design
Solomon Four-Group Design Classic Design may sensitize subjects Classic Design may sensitize subjects More complex experimental designs More complex experimental designs
Posttest-only Control Group Design Includes Groups 3 and 4 of the Solomon design. Includes Groups 3 and 4 of the Solomon design. With proper randomization, only these groups are needed to control the problems of internal invalidity and the interaction between testing and stimulus. With proper randomization, only these groups are needed to control the problems of internal invalidity and the interaction between testing and stimulus.
Other Design Considerations Double blind - no experimenter bias Double blind - no experimenter bias Subject selection - convenience or representative Subject selection - convenience or representative Generalizability vs. explanatory power Generalizability vs. explanatory power Comparability of experimental and control groups Comparability of experimental and control groups Probability sampling for representativeness Probability sampling for representativeness Randomization over matching for equivalence Randomization over matching for equivalence
Threats to Validity in Experiments History - intervening event History - intervening event Maturation - people change Maturation - people change Testing - respond to measures Testing - respond to measures Instrumentation - change measures Instrumentation - change measures Regression - Regress to mean Regression - Regress to mean Selection biases - incomparable groups Selection biases - incomparable groups Experimental mortality - Drop out of study Experimental mortality - Drop out of study Diffusion of treatment - contamination of control Diffusion of treatment - contamination of control Compensation - advantage control group Compensation - advantage control group Compensatory rivalry - control group competes harder Compensatory rivalry - control group competes harder Demoralization - control group may give up Demoralization - control group may give up + External Threats to Validity / Interactions with Stimulus + External Threats to Validity / Interactions with Stimulus
Quasi-Experimental Design
"Natural" Experiments Important social scientific experiments occur outside controlled settings and in the course of normal social events. Important social scientific experiments occur outside controlled settings and in the course of normal social events. Raise validity issues because researcher must take things as they occur. Raise validity issues because researcher must take things as they occur.
Time and Survey Design Extending logic of Experimentation to Surveys Extending logic of Experimentation to Surveys Static designs: Static designs: Cross-sectional study Cross-sectional study Longitudinal designs: Longitudinal designs: Trend studies Trend studies Cohort studies Cohort studies Panel studies Panel studies
Experimental Method Strengths: Isolation of the experimental variable over time. Isolation of the experimental variable over time. Experiments can be replicated several times using different groups of subjects. Experiments can be replicated several times using different groups of subjects.Weaknesses: Artificiality of laboratory setting. Artificiality of laboratory setting. Social processes that occur in a lab might not occur in a more natural social setting. Social processes that occur in a lab might not occur in a more natural social setting.