Advanced Research Designs u Designing Designs Designing Designs cause & effect. uEstablishing cause & effect. threats uMinimizing threats to validity? uConstructing uConstructing a design? good uWhat is a good design? u Relationship Among Pre-post Designs Relationship Among Pre-post Designs

Establishing Cause & Effect Temporal precedence Covariation of cause and effect No alternative explanationsCauseEffectthen time If X, then Y. If not X, then not Y. ProgramOutcome Causes? Alternative cause Alternative cause Alternative cause Alternative cause

Establishing Cause & Effect T ypical Outcome Evaluation u Is taken care of because you intervene before you measure outcome u Is taken care of because you control the intervention u Is the central issue of internal validity -- usually taken care of through your design Temporal Precedence Covariation of Cause & Effect No Alternative Explanations

Minimizing Threats to Validity. u By argument u By measurement or observation u By design u By analysis u By preventive action

Minimizing Threats to Validity by Argument By argument Simply say the threat isn’t reasonable.

Minimizing Threats by Measurement By measurement In some cases it is possible to measure that the threat didn’t happen.

Minimizing Threats by Design ROXOROXO Maturationthreat? How can the design be improved to address:

Minimizing Threats by Design Add a group to control for the threat. ROXOROOROXOROO To address a maturationthreat

Minimizing Threats by Analysis Mortality threat: Are your dropouts different from those retained in your study?

Minimizing Threats by Analysis --Mortality Threat example-- Show through analysis that the threat isn’t plausible. Pretest Means can show groups equivalent.

Minimizing Threats by Preventive Action By preventive action Rivalry?

Minimizing Threats by Preventive Action By preventive action Don’t allow the threat to happen in the first place. Rivalry Get program next semester

How Do You Construct a Design? u Time u Programs u Observations u Groups Start with basic design ( X O ), then expand across ???

Design Construction: Time XOXO Start with the simplest design. time. Expand across time.

Design Construction: Time time. Expand across time. OOXOOOXO OXOOOOXOOO

Design Construction: Programs programs. Expand across programs. OX1OOX2OOX3OOX1OOX2OOX3O Programvariations

Design Construction: Observations observations. Expand across observations. O 1 XO 1,2,3 O 1 O 1,2,3 Measurementvariations

Design Construction: Groups groups. Expand across groups. ROXOROXO ROOROO ROXOROXO ROOROO Add a replicate.

Design Construction: Groups & Time groups and time. Expand across groups and time. ROXOROONOROXOROONO Time Groups

Design Construction: Summary Overview XOXO Start

Design Construction: Summary Overview XOXO Start Expand (Brainstorm) ROXOOROOXONOXONOOROXOOROOXONOXONOO

Design Construction: Summary Overview XOXO Start Expand ROXOOROOXONOXONOOROXOOROOXONOXONOO ROXOROONOOROXOROONOO Contract (reality, cost, etc.)

What Is a Good Design? u Theory-grounded u Situational (adapted) u Feasible u Redundant u Efficient

Relationships Among Pre-Post Designs

Pre-Post Designs u Randomized experiment (R) u Nonequivalent group design (N) u Regression-Discontinuity design (C) O XO OO

The Main Issue u Selection threats u Are the groups similar before the treatment? u Related to assignment method in each design

Assignment Method Probability of Assignment to Treatment Given the Pretest Pretest

Assignment Method Probability of Assignment to Treatment Given the Pretest RE Pretest

Assignment Method Probability of Assignment to Treatment Given the Pretest RE RD Pretest

Assignment Method Probability of Assignment to Treatment Given the Pretest RE RD NEGD Pretest

Assignment Method Probability of Assignment to Treatment Given the Pretest RE RD NEGD Pretest

Pretest Equivalence Equivalent Maximally nonequivalent

Pretest Equivalence Equivalent Maximally nonequivalent RE

Pretest Equivalence Equivalent Maximally nonequivalent RDRE

Pretest Equivalence Equivalent Maximally nonequivalent RDNEGDRE

Assignment Rule Known 4 4 Regression-Discontinuity Randomized Experiment Nonequivalent Group Design