Other Quasi-Experimental Designs
Design Variations Show specific design features that can be used to address specific threats or constraints in the context
Proxy Pretest Design l Pretest based on recollection or archived data l Useful when you weren’t able to get a pretest but wanted to address gain NO1XO2NO1O2NO1XO2NO1O2
Separate Pre-Post Samples l Groups with the same subscript come from the same context. l Here, N 1 might be people who were in the program at Agency 1 last year, with those in N 2 at Agency 2 last year. l This is like having a proxy pretest on a different group. N1ON1XON2ON2ON1ON1XON2ON2O
Separate Pre-Post Samples l Take random samples at two times of people at two nonequivalent agencies. l Useful when you routinely measure with surveys. l You can assume that the pre and post samples are equivalent, but the two agencies may not be. R1OR1XOR2OR2OR1OR1XOR2OR2O N N
Double-Pretest Design l Strong in internal validity l Helps address selection-maturation l How does this affect selection-testing? NOOXONOOONOOXONOOO
Switching Replications l Strong design for both internal and external validity l Strong against social threats to internal validity l Strong ethically NOXOONOOXONOXOONOOXO
Nonequivalent Dependent Variables Design (NEDV) l The variables have to be similar enough that they would be affected the same way by all threats. l The program has to target one variable and not the other. NO1XO1NO2O2NO1XO1NO2O2
NEDV Example l Only works if we can assume that geometry scores show what would have happened to algebra if untreated. l The variable is the control. l Note that there is no control group here.
NEDV Pattern Matching l Have many outcome variables. l Have theory that tells how affected (from most to least) each variable will be by the program. l Match observed gains with predicted ones. l If match, what does it mean?
NEDV Pattern Matching l A “ladder” graph. l What are the threats? r =.997
NEDV Pattern Matching l Single group design, but could be used with multiple groups (could even be coupled with experimental design). l Can measure left and right on different scales (e.g., right could be t-values). l How do we get the expectations?
Regression Point Displacement (RPD) l Intervene in a single site l Have many nonequivalent control sites l Good design for community-based evaluation N (n=1) OXO NOO
RPD Example l Comprehensive community-based AIDS education l Intervene in one community (e.g., county) l Have 29 other communities (e.g., counties) in state as controls l measure is annual HIV positive rate by county
RPD Example X Y
X Y Regression line
RPD Example X Y Regression line Treated community point
RPD Example X Y Regression pine Treated community point Posttest displacement