Experiments and Quasi-Experiments 1 Experiments and Quasi-Experiments

2 Introduction Experiment: using a controlled situation to observe a result Involves taking and observing action Great for hypothesis-testing Theory-full

The Classical Experiment 3 The Classical Experiment Involves three major pairs of components: Independent and dependent variables Pre-Testing and Post-Testing Experimental and Control groups Randomization

Variables, X and Y X = Independent Variable (IV), cause, influencer 4 Variables, X and Y X = Independent Variable (IV), cause, influencer Y = Dependent Variable (DV), effect, outcome

Control and Experimental Groups 5 Control and Experimental Groups Experimental group – exposed to whatever treatment, policy, initiative we are testing Control group – very similar to experimental group, except that they are NOT exposed

6 Selecting Subjects Decide on target population 1st– the group to which the results of your experiment will apply Cardinal rule – ensure that C and E groups are as similar as possible Randomization helps towards this

Hawthorne Effect Pointed to the necessity of control groups 7 Hawthorne Effect Pointed to the necessity of control groups IV: improved working conditions (better lighting) DV: improvement in employee satisfaction and productivity Workers were responding more to the attention than to the improved working conditions

8 Placebo We often don’t want people to know if they are receiving treatment or not We expose our control group to a “dummy” IV just so we are treating everyone the same Medical research: participants don’t know what they are taking Ensures that changes in DV actually result from IV and are not psychologically based

Pre-Testing and Post-Testing 9 Pre-Testing and Post-Testing First, subjects measured on DV prior to association with the IV (pre-tested) Next, subjects are exposed to the IV Third, subjects are remeasured in terms of the DV (post-tested) Difference?--must be the intervention!

Double-Blind Experiment 10 Double-Blind Experiment Subjects and experimenters do not know who is in the control and experimental groups

Experiments and Causal Inference 11 Experiments and Causal Inference Experimental design ensures: Cause precedes effect via taking posttest Empirical correlation exists via comparing pretest to posttest No spurious 3rd variable influencing correlation via posttest comparison between experimental and control groups, and via randomization

Internal Validity Threats (12) Conclusions drawn from experimental results may not reflect what went on in experiment History – external events may occur during the course of the experiment Maturation – people grow Testing – the process of testing and retesting

More Internal Validity Threats 13 More Internal Validity Threats 4. Instrumentation – Changes in the measurement process 5. Statistical regression – Extreme scores regress to the mean 6. Selection bias – the way in which subjects are chosen 7. Experimental mortality – subjects may drop out prior to completion of experiment 8. Causal time order – ambiguity about order of stimulus and DV – which caused which?

Last, Internal Validity Threats 14 Last, Internal Validity Threats 9. Diffusion/imitation of treatment – when E and C groups communicate, E group may pass on elements to C 10. Compensatory treatment – C group is deprived of something considered to be of value 11. Compensatory Rivalry – C group deprived of the stimulus may try to compensate by working harder 12. Demoralization – feelings of deprivation result in C group giving up

Construct Validity Threats 15 Construct Validity Threats Concerned with generalizing from experiment to actual causal processes in the real world Link construct and measures to theory Clearly indicate what constructs are represented by what measures Decide how much treatment is required to produce change in DV

External Validity Threats 16 External Validity Threats Significant for experiments conducted under carefully controlled conditions rather than more natural conditions But, this reduces internal validity threats! A conundrum! Suggestion – explanatory studies -> internal validity; applied studies -> external validity

Statistical Conclusion Validity Threats (Low Power) 17 Statistical Conclusion Validity Threats (Low Power) Problem is likely when using small samples With more cases, it is easier to see more differences

Quasi-Experimental Designs 18 Quasi-Experimental Designs When?—randomization not possible Quasi = “to a certain degree” or, in short, “like”