1. Designing Designs for Research

2. Designing Designs cause u How do you establish cause? minimize threats u How do you minimize threats to validity? construct u How do you construct a design? good u What is a good design?

3. How Do You Establish Cause? Temporal Precedence

4. How Do You Establish Cause? Temporal precedence CauseEffect then Time

5. How Do You Establish Cause? Temporal precedence CauseEffect then Time Why is this important?

6. How Do You Establish Cause? Temporal precedence CauseEffect then Time Why is this important? Cyclical functions Inflation Unemployment

7. How Do You Establish Cause? Temporal Precedence Covariation of Cause and EffectCauseEffectthenTime

8. How Do You Establish Cause? Temporal precedence Covariation of cause and effectCauseEffectthen time If X, then Y. If not X, then not Y.

9. How Do You Establish Cause? Temporal precedence Covariation of cause and effectCauseEffectthen time If X, then Y. If not X, then not Y. If program given, then outcome observed. If program not given, then outcome not observed.

10. How Do You Establish Cause? Temporal precedence Covariation of cause and effectCauseEffectthen time If X, then Y. If not X, then not Y. If program given, then outcome observed. If program not given, then outcome not observed. If more of program, then more of outcome observed. If less of program given, then less of outcome observed.

11. How Do You Establish Cause? Temporal precedence Covariation of cause and effect No alternative explanationsCauseEffectthen time If X, then Y. If not X, then not Y.

12. How Do You Establish Cause? Temporal precedence Covariation of cause and effect No alternative explanationsCauseEffectthen time If X, then Y. If not X, then not Y. ProgramOutcome causes?

13. How Do You Establish Cause? Temporal precedence Covariation of cause and effect No alternative explanationsCauseEffectthen time If X, then Y. If not X, then not Y. ProgramOutcome Causes? Alternative cause Alternative cause Alternative cause Alternative cause

14. In Typical Outcome Evaluation u Is taken care of because you intervene before you measure outcome u Is taken care of because you control the intervention u Is the central issue of internal validity -- usually taken care of through your design Temporal Precedence Covariation of Cause and Effect No Alternative Explanations

15. How Do You Minimize Threats to Validity? u By argument u By measurement or observation u By design u By analysis u By preventive action

16. How Do You Minimize Threats to Validity? By argument

17. How Do You Minimize Threats to Validity? By argument Simply say the threat isn’t reasonable.

18. How Do You Minimize Threats to Validity? By measurement

19. How Do You Minimize Threats to Validity? By measurement Measure that the threat didn’t happen.

20. How Do You Minimize Threats to Validity? By design ROXOROXO Maturationthreat?

21. How Do You Minimize Threats to Validity? By design Add a group to control for the threat. ROXOROOROXOROO To address a maturationthreat

22. How Do You Minimize Threats to Validity? By analysis Mortality threat : Are your dropouts different from those retained in your study?

23. How Do You Minimize Threats to Validity? By analysis Show through analysis that the threat isn’t plausible. Pretestmeans

24. How Do You Minimize Threats to Validity? By preventive action Rivalry?

25. How Do You Minimize Threats to Validity? By preventive action Don’t allow the threat to happen in the first place. Rivalry Get program next semester

26. How Do You Construct a Design? u Across time u Across programs u Across observations u Across groups Expand...

27. How Do You Construct a Design? Start with the simplest design.

28. How Do You Construct a Design? XOXO Start with the simplest design.

29. How Do You Construct a Design? time. Expand across time.

30. How Do You Construct a Design? time. Expand across time. OOXOOOXO OXOOOOXOOO

31. How Do You Construct a Design? programs. Expand across programs.

32. How Do You Construct a Design? programs. Expand across programs. OX1OOX2OOX3OOX1OOX2OOX3O Programvariations

33. How Do You Construct a Design? observations. Expand across observations.

34. How Do You Construct a Design? observations. Expand across observations. O 1 XO 1,2,3 O 1 O 1,2,3 Measurementvariations

35. How Do You Construct a Design? groups. Expand across groups. ROXOROOROXOROO

36. How Do You Construct a Design? groups. Expand across groups. ROXOROOROXOROOROXOROOROXOROO Add a replicate.

37. How Do You Construct a Design? groups and time. Expand across groups and time. ROXOROONOROXOROONO

38. How Do You Construct a Design? groups and time. Expand across groups and time. ROXOROONOROXOROONO

39. How Do You Construct a Design? groups and time. Expand across groups and time. ROXOROONOROXOROONO Time Groups

40. How Do You Construct a Design? XOXO Start

41. How Do You Construct a Design? XOXO Start Expand ROXOOROOXONOXONOOROXOOROOXONOXONOO

42. How Do You Construct a Design? XOXO Start Expand ROXOOROOXONOXONOOROXOOROOXONOXONOO ROXOROONOOROXOROONOO Contract

43. What Is a Good Design? u Theory-grounded u Situational (adapted) u Feasible u Redundant u Efficient