1. The analysis of individual and average causal effects: Basic principles and some applications EUROPEAN SCIENCE FOUNDATION Programme ‘Quantitative Methods in the Social Sciences’ (QMSS) Seminar: ‘Theory-Driven Evaluation and Intervention Studies in the Social Sciences’, 28-29 September 2006, Nicosia, Cyprus Rolf Steyer Institute of Psychology Department of Methodology and Evaluation Research Email: rolf.steyer@uni-jena.de Rolf Steyer Institute of Psychology Department of Methodology and Evaluation Research Email: rolf.steyer@uni-jena.de Rolf Steyer Institute of Psychology Department of Methodology and Evaluation Research Email: rolf.steyer@uni-jena.de Rolf Steyer Institute of Psychology Department of Methodology and Evaluation Research Email: rolf.steyer@uni-jena.de Rolf Steyer University of Jena (Germany) Institute of Psychology Department of Methodology and Evaluation Research Email: rolf.steyer@uni-jena.de www.uni-jena.de/svw/metheval

2. 2 Overview Individual and average causal effects (Neyman, Rubin) Motivation: The Simpson Paradox Pre-Post Design with Control Group for the Analysis of Intervention Effects Designs for the Analysis of Individual Causal Effects - Example with an Intervention - Example with Method Effects Conclusions

3. 3 The Simpson Paradox

4. 4

5. 5

6. 6

7. 7

8. 8 The single-unit trial Sample a person u, register her assignment to one of the treatment conditions and observe her outcome y. In this single-unit trial U, X, and Y have a joint distribution u1u1 treatment y1y1 y2y2 y3y3 y4y4 y1y1 y2y2 y3y3 y4y4 control u2u2 treatment control............

9. 9 Individual and average causal effects (Neyman, Rubin) Define the true-outcome variables as follows:  0 (u) := E(Y 0  U = u) and  1 (u) := E (Y 1  U = u)  1-0 (u) :=  1 (u)   0 (u) = individual causal effect of unit u ACE 1-0 = E (  1-0 ) = E(  1 ) – E(  0 ) =: average causal effect

10. 10 Individual, conditional and average causal effects (Neyman, Rubin)

11. 11 Bias Theorem

12. 12 Three Design Types Between-Group Designs Pre-post Designs (not used at all in the Neyman-Rubin tradition) Between-Group Designs with Pre-Post Measures (only the between group comparisons are used in the Neyman-Rubin tradition)

13. 13 Utilizing Pre-Post Designs for the Analysis of Individual Effects Pre-post Designs and Between-Group Designs with Pre-Post Measures can be used to analyze not only average but also individual causal effects. The crucial asssumption is that the individual pretest distribution is the same as the individual posttest (outcome variable) distribution under control (no treatment).

14. 14 Theorem 1 (Sufficient conditions for unbiasedness of the (conditional) prima facie effects)

15. 15 Theorem 2

16. 16 Standard research questions in the analysis of causal effects

17. 17 Generalization to J + 1 Treatment Conditions

18. 18 Average effects in the analysis of causal effects

19. 19 Types of covariates

20. 20 Example (using EffectLite): Intelligence training

21. 21 Pre-Post Design with Control Group for the Analysis of Intervention Effects

22. 22 Between-Group Design, no Pretest Y 0 = τ 0 +  0 Y 1 =  0 +  1  τ 0 +  1

23. 23 Pre-Post Design, no Control Group

24. 24 Identified Individual Effects Model with pretests Y 11 and Y 21

25. 25 Identified Individual Effects Model with pretests Y 11 and Y 21

26. 26 Design for the Analysis of Method Effects

27. 27 Introducing Individual Method Effects

28. 28 Introducing Individual Method Effects τ 22  τ 12 = τ 21  τ 11 = IME 2-1

29. 29 Introducing Individual Method Effects τ 22  τ 12 = τ 21  τ 11 = IME 2-1

30. 30 An identifíed Individual-Method-Effects Model τ 22  τ 12 = τ 21  τ 11 = IME 2-1

31. 31 Model in treatment group Treatment

32. 32 Model in control group

33. 33 Model in treatment group (t-values)

34. 34 Model in control group (t-values)

35. 35 Correlation Matrix of ETA in Control group IQ ICE IME -------- -------- -------- IQ 1.00 ICE -0.65 1.00 IME -0.37 0.43 1.00

36. 36 Correlation Matrix of ETA in Experimental Group IQ ICE IME -------- -------- -------- IQ 1.00 ICE -0.52 1.00 IME -0.26 0.15 1.00

37. 37 The effects of negativ item formulation

38. 38 The effects of negativ item formulation

39. 39 The effects of negativ item formulation (standardized)

40. 40 Summary and Conclusion We can analyze individual (and average) causal effects in Pre-post Designs The causal interpretation rests on assumptions These assumptions can be tested Latent variables can be constructed from true-scores Not a single path in our SEM models represented a causal effect

41. 41 Want More? Steyer, R. & Partchev, I.. (2006). Causal Effects in Experiments and Quasi-Experiments: Theory (Chapters 1 -5 are available at www.causal- effects.de) Symposium on causality in Jena July 7 to 9, 2006 with Tom Cook, Steve West, Don Rubin … (videos available: see www.uni-jena.de/svw/metheval Online video of workshop on the analysis of causal effects (same home page) Software „EffectLite“ (see: www.statlite.com)

42. 42 Thanks to: Sven Hartenstein Ulf Kröhne Benjamin Nagengast Ivailo Partchev Steffi Pohl