1. Statistical Analysis of the Two Group Post-Only Randomized Experiment

2. Analysis Requirements l Two groups l A post-only measure l Two distributions, each with an average and variation l Want to assess treatment effect l Treatment effect = statistical (i.e., nonchance) difference between the groups RXORORXORO

3. Statistical Analysis

4. Control group mean

5. Statistical Analysis Control group mean Treatment group mean

6. Statistical Analysis Control group mean Treatment group mean Is there a difference?

7. What Does Difference Mean?

8. Medium variability

9. What Does Difference Mean? Medium variability High variability

10. What Does Difference Mean? Medium variability High variability Low variability

11. What Does Difference Mean? Medium variability High variability Low variability The mean difference is the same for all three cases.

12. What Does Difference Mean? Medium variability High variability Low variability Which one shows the greatest difference?

13. What Does Difference Mean? l A statistical difference is a function of the difference between means relative to the variability. l A small difference between means with large variability could be due to chance. l Like a signal-to-noise ratio. Low variability Which one shows the greatest difference?

14. What Do We Estimate? Low variability

15. What Do We Estimate? Low variability Signal Noise

16. What Do We Estimate? Low variability Signal Noise Difference between group means =

17. What Do We Estimate? Low variability Signal Noise Difference between group means Variability of groups =

18. What Do We Estimate? Low variability Signal Noise Difference between group means Variability of groups = = X T - X C SE(X T - X C ) __ __

19. What Do We Estimate? Low variability Signal Noise Difference between group means Variability of groups = X T - X C SE(X T - X C ) = = t-value __ __

20. What Do We Estimate? l The t-test, one-way analysis of variance (ANOVA) and a form of regression all test the same thing and can be considered equivalent alternative analyses. l The regression model is emphasized here because it is the most general. Low variability

21. Regression Model for t-Test or One-Way ANOVA y i =  0 +  1 Z i + e i

22. Regression Model for t-Test or One-Way ANOVA y i = outcome score for the i th unit  0 =coefficient for the intercept  1 =coefficient for the slope Z i =1 if i th unit is in the treatment group 0 if i th unit is in the control group e i =residual for the i th unit y i =  0 +  1 Z i + e i where:

23. In Graph Form...

24. 0 (Control) 1 (Treatment) ZiZi

25. In Graph Form... 0 (Control) 1 (Treatment) YiYi ZiZi

26. In Graph Form... 0 (Control) 1 (Treatment) YiYi ZiZi

27. In Graph Form... 0 (Control) 1 (Treatment)  0 is the intercept y-value when z=0. YiYi ZiZi

28. In Graph Form... 0 (Control) 1 (Treatment)  0 is the intercept y-value when z=0.  1 is the slope. YiYi ZiZi

29. Why Is  1 the Mean Difference? 0 (Control) 1 (Treatment)  0 is the intercept y-value when z=0.  1 is the slope. YiYi ZiZi

30. Why Is  1 the Mean Difference? 0 (Control) 1 (Treatment) Intuitive Explanation: Because slope is the change in y for a 1-unit change in x. YiYi ZiZi Change in y Unit change in x (i.e., z)

31. Why Is  1 the Mean Difference? 0 (Control) 1 (Treatment) Since the 1-unit change in x is the treatment- control difference, the slope is the difference between the posttest means of the two groups. YiYi ZiZi Change in y

32. Why  1 Is the Mean Difference in y i =  0 +  1 Z i + e i

33. Why  1 Is the Mean Difference in First, determine effect for each group: y i =  0 +  1 Z i + e i

34. Why  1 Is the Mean Difference in First, determine effect for each group: For control group (Z i = 0): y i =  0 +  1 Z i + e i

35. Why  1 Is the Mean Difference in First, determine effect for each group: For control group (Z i = 0): y i =  0 +  1 Z i + e i y C =  0 +  1 (0) + 0

36. Why  1 Is the Mean Difference in First, determine effect for each group: For control group (Z i = 0): y i =  0 +  1 Z i + e i y C =  0 +  1 (0) + 0 e i averages to 0 across the group.

37. Why  1 Is the Mean Difference in First, determine effect for each group: For control group (Z i = 0): y i =  0 +  1 Z i + e i y C =  0 +  1 (0) + 0 y C =  0 e i averages to 0 across the group.

38. Why  1 Is the Mean Difference in First, determine effect for each group: For control group (Z i = 0): For treatment group (Z i = 1): y i =  0 +  1 Z i + e i y C =  0 +  1 (0) + 0 y C =  0 e i averages to 0 across the group.

39. Why  1 Is the Mean Difference in First, determine effect for each group: For control group (Z i = 0): For treatment group (Z i = 1): y i =  0 +  1 Z i + e i y C =  0 +  1 (0) + 0 y C =  0 y T =  0 +  1 (1) + 0 e i averages to 0 across the group.

40. Why  1 Is the Mean Difference in First, determine effect for each group: For control group (Z i = 0): For treatment group (Z i = 1): y i =  0 +  1 Z i + e i y C =  0 +  1 (0) + 0 y C =  0 y T =  0 +  1 (1) + 0 e i averages to 0 across the group.

41. Why  1 Is the Mean Difference in First, determine effect for each group: For control group (Z i = 0): For treatment group (Z i = 1): y i =  0 +  1 Z i + e i y C =  0 +  1 (0) + 0 y C =  0 y T =  0 +  1 (1) + 0 y T =  0 +  1 e i averages to 0 across the group.

42. Why  1 Is the Mean Difference in y i =  0 +  1 Z i + e i

43. Why  1 Is the Mean Difference in Then, find the difference between the two groups: y i =  0 +  1 Z i + e i

44. Why  1 Is the Mean Difference in Then, find the difference between the two groups: y i =  0 +  1 Z i + e i y T =  0 +  1 yTyT treatment

45. Why  1 Is the Mean Difference in Then, find the difference between the two groups: y i =  0 +  1 Z i + e i y C =  0 y T =  0 +  1 y T - y C = controltreatment

46. Why  1 Is the Mean Difference in Then, find the difference between the two groups: y i =  0 +  1 Z i + e i y C =  0 y T =  0 +  1 y T - y C = (  0 +  1 ) controltreatment

47. Why  1 Is the Mean Difference in Then, find the difference between the two groups: y i =  0 +  1 Z i + e i y C =  0 y T =  0 +  1 y T - y C = (  0 +  1 ) -  0 controltreatment

48. Why  1 Is the Mean Difference in Then, find the difference between the two groups: y i =  0 +  1 Z i + e i y C =  0 y T =  0 +  1 y T - y C = (  0 +  1 ) -  0 controltreatment y T - y C =  0 +  1 -  0

49. Why  1 Is the Mean Difference in Then, find the difference between the two groups: y i =  0 +  1 Z i + e i y C =  0 y T =  0 +  1 y T - y C = (  0 +  1 ) -  0 controltreatment y T - y C =  0 +  1 -  0 

50. Why  1 Is the Mean Difference in Then, find the difference between the two groups: y i =  0 +  1 Z i + e i y C =  0 y T =  0 +  1 y T - y C = (  0 +  1 ) -  0 controltreatment y T - y C =  0 +  1 -  0 y T - y C =  1 

51. Conclusions l t-test, one-way ANOVA and regression analysis all yield same results in this case. l The regression analysis method utilizes a dummy variable for treatment. l Regression analysis is the most general model of the three.