1. Chapter 11 Quasi-Experimental Designs ♣ ♣ Introduction   Nonequivalent Comparison Group Design   Time-Series Design   Regression Discontinuity Design    Back to Brief Contents

2. 11.0 Introduction Quasi-experimental design  Not meet all the requirements necessary for controlling the influence of EV (e.g.) random assignment  Not eliminate all threats to internal validity Threats to internal validity are ruled out by Tab 11.1 Tab 11.1  Identification and study of the threats  By including design elements as pretests or other control groups  By coherent pattern matching—making a complex prediction that few rival hypotheses can explain  Back to Chapter Contents

3. 11.1 Nonequivalent Comparison Group Design pre-test treatment posttest Exper. Gp Y 1 X Y 2 Control Gp Y 1 Y 2 Fig 11.1 Fig 11.1 This is the most common quasi-experimental design Threats frequently reveal themselves in the outcome Example Fig 11.2 11.3 Fig 11.211.3 Possible biases Tab 11.2 Tab 11.2  Back to Chapter Contents

4. 11.1 Nonequivalent Comparison Group Design Outcomes with Rival Hypotheses  Outcome I: First Increasing Treatment Effect  Outcome II: Increasing Treatment and Control Groups  Outcome III: Second Increasing Treatment Effect  Outcome IV: Crossover Effect Causal Inference from the Nonequivalent Comparison Group Design  Back to Chapter Contents

5. Outcome Patterns in the Nonequivalent Comparison Group Design I: Increasing treatment effect Fig 11.4 Fig 11.4 experimental control pretest posttest Threat: Selection-Maturation, Selection-History Eliminate Selection-Maturation : Matching Tab 11.3 Tab 11.3 Statistical Regression Techniques

6. II: Increasing treatment and control groups Fig 11.5Fig 11.5 experimental control pretest posttest  Selection-maturation is a threat

7. III: Increasing treatment effect Fig 11.6 Fig 11.6  Statistical regression is a threat  Selection-history is a threat control experimental pretest posttest

8. IV: Cross-over effect Fig 11.7 Fig 11.7  No possible rival hypotheses (history?) experimental control pretest posttest

9. 11.1 Nonequivalent Comparison Group Design Outcomes with Rival Hypotheses Causal Inference from the Nonequivalent Comparison Group Design  Well designed and executed: ≈ randomized design  Well designed P assigned to group: not self-select Reduce pretest difference: matching  Back to Chapter Contents

10. 11.2 Time-Series Design Interrupted Time-Series design Fig 11.8 Fig 11.8 Pre-response Treatment Post-response Y 1 Y 1 Y 1 Y 1 X Y 2 Y 2 Y 2 Y 2  Treatment effect revealed by a different pattern of pre and posttest responses  Back to Chapter Contents

11. 11.2 Time-Series Design Interrupted Time-Series design  Lawler & Hackman (1969) IV: employee participation in develop and implementation of incentive plan DV: absenteeism Fig 11.9 Fig 11.9  Vernon, Bedford, & Wyatt (1924) IV: introduce a rest period DV: productivity (output) Fig 11.11 Fig 11.11  One-group pretest-posttest Fig 11.10 Fig 11.10  Back to Chapter Contents

12. 11.2 Time-Series Design Interrupted Time-Series design  Possible patterns Fig 11.12 Fig 11.12  Statistical method Bayesian moving average model Tryon (1982), Crosbie (1993)  Threat: History (end)  Back to Chapter Contents

13. 11.3 Regression Discontinuity Design Used to determine if the special treatment some individuals receive has any effect Characteristics of the design Fig 11.13 Fig 11.13  All individuals are pretested  Individuals who score above some cutoff score receive the treatment  All individuals are posttested  Discontinuity in the regression line indicates a treatment effect Fig 11.14 11.15 Fig 11.1411.15  Back to Chapter Contents

14. 11.3 Regression Discontinuity Design Design Criteria that Must be Adhered to Tab 11.4 Tab 11.4  Assignment must be based on the cutoff score  Assignment cannot be a nominal variable as gender, or drug user or nonuser  Cutoff score should be at (or near) the mean  Experimenter should control group assignment  Relationship (linear, curvilinear, etc.) should be known  Participants must be from the same population  Back to Chapter Contents

15. 11.3 Regression Discontinuity Design Primary Threat in the Regression Discontinuity Design  Contemporaneous history effect — This is possible but not plausible  Attrition: any design  This is the more powerful of the quasi-experimental designs (end)  Back to Chapter Contents