Designing Designs for Research

Designing Designs cause u How do you establish cause? minimize threats u How do you minimize threats to validity? construct u How do you construct a design? good u What is a good design?

How Do You Establish Cause? Temporal Precedence

How Do You Establish Cause? Temporal precedence CauseEffect then Time

How Do You Establish Cause? Temporal precedence CauseEffect then Time Why is this important?

How Do You Establish Cause? Temporal precedence CauseEffect then Time Why is this important? Cyclical functions Inflation Unemployment

How Do You Establish Cause? Temporal Precedence Covariation of Cause and EffectCauseEffectthenTime

How Do You Establish Cause? Temporal precedence Covariation of cause and effectCauseEffectthen time If X, then Y. If not X, then not Y.

How Do You Establish Cause? Temporal precedence Covariation of cause and effectCauseEffectthen time If X, then Y. If not X, then not Y. If program given, then outcome observed. If program not given, then outcome not observed.

How Do You Establish Cause? Temporal precedence Covariation of cause and effectCauseEffectthen time If X, then Y. If not X, then not Y. If program given, then outcome observed. If program not given, then outcome not observed. If more of program, then more of outcome observed. If less of program given, then less of outcome observed.

How Do You Establish Cause? Temporal precedence Covariation of cause and effect No alternative explanationsCauseEffectthen time If X, then Y. If not X, then not Y.

How Do You Establish Cause? Temporal precedence Covariation of cause and effect No alternative explanationsCauseEffectthen time If X, then Y. If not X, then not Y. ProgramOutcome causes?

How Do You Establish Cause? Temporal precedence Covariation of cause and effect No alternative explanationsCauseEffectthen time If X, then Y. If not X, then not Y. ProgramOutcome Causes? Alternative cause Alternative cause Alternative cause Alternative cause

In Typical Outcome Evaluation u Is taken care of because you intervene before you measure outcome u Is taken care of because you control the intervention u Is the central issue of internal validity -- usually taken care of through your design Temporal Precedence Covariation of Cause and Effect No Alternative Explanations

How Do You Minimize Threats to Validity? u By argument u By measurement or observation u By design u By analysis u By preventive action

How Do You Minimize Threats to Validity? By argument

How Do You Minimize Threats to Validity? By argument Simply say the threat isn’t reasonable.

How Do You Minimize Threats to Validity? By measurement

How Do You Minimize Threats to Validity? By measurement Measure that the threat didn’t happen.

How Do You Minimize Threats to Validity? By design ROXOROXO Maturationthreat?

How Do You Minimize Threats to Validity? By design Add a group to control for the threat. ROXOROOROXOROO To address a maturationthreat

How Do You Minimize Threats to Validity? By analysis Mortality threat : Are your dropouts different from those retained in your study?

How Do You Minimize Threats to Validity? By analysis Show through analysis that the threat isn’t plausible. Pretestmeans

How Do You Minimize Threats to Validity? By preventive action Rivalry?

How Do You Minimize Threats to Validity? By preventive action Don’t allow the threat to happen in the first place. Rivalry Get program next semester

How Do You Construct a Design? u Across time u Across programs u Across observations u Across groups Expand...

How Do You Construct a Design? Start with the simplest design.

How Do You Construct a Design? XOXO Start with the simplest design.

How Do You Construct a Design? time. Expand across time.

How Do You Construct a Design? time. Expand across time. OOXOOOXO OXOOOOXOOO

How Do You Construct a Design? programs. Expand across programs.

How Do You Construct a Design? programs. Expand across programs. OX1OOX2OOX3OOX1OOX2OOX3O Programvariations

How Do You Construct a Design? observations. Expand across observations.

How Do You Construct a Design? observations. Expand across observations. O 1 XO 1,2,3 O 1 O 1,2,3 Measurementvariations

How Do You Construct a Design? groups. Expand across groups. ROXOROOROXOROO

How Do You Construct a Design? groups. Expand across groups. ROXOROOROXOROOROXOROOROXOROO Add a replicate.

How Do You Construct a Design? groups and time. Expand across groups and time. ROXOROONOROXOROONO

How Do You Construct a Design? groups and time. Expand across groups and time. ROXOROONOROXOROONO

How Do You Construct a Design? groups and time. Expand across groups and time. ROXOROONOROXOROONO Time Groups

How Do You Construct a Design? XOXO Start

How Do You Construct a Design? XOXO Start Expand ROXOOROOXONOXONOOROXOOROOXONOXONOO

How Do You Construct a Design? XOXO Start Expand ROXOOROOXONOXONOOROXOOROOXONOXONOO ROXOROONOOROXOROONOO Contract

What Is a Good Design? u Theory-grounded u Situational (adapted) u Feasible u Redundant u Efficient