1. Statistical Analysis of the Regression-Discontinuity Design

2. Analysis Requirements l Pre-post l Two-group l Treatment-control (dummy-code) COXOCOOCOXOCOO

3. Assumptions in the Analysis l Cutoff criterion perfectly followed. l Pre-post distribution is a polynomial or can be transformed to one. l Comparison group has sufficient variance on pretest. l Pretest distribution continuous. l Program uniformly implemented.

4. The Curvilinearilty Problem 1009080706050403020100 80 70 60 50 40 30 20 pre p o s t e f f If the true pre-post relationship is not linear...

5. The Curvilinearilty Problem 1009080706050403020100 80 70 60 50 40 30 20 pre p o s t e f f and we fit parallel straight lines as the model...

6. The Curvilinearilty Problem 1009080706050403020100 80 70 60 50 40 30 20 pre p o s t e f f and we fit parallel straight lines as the model... The result will be biased.

7. The Curvilinearilty Problem 1009080706050403020100 80 70 60 50 40 30 20 pre p o s t e f f And even if the lines aren’t parallel (interaction effect)...

8. The Curvilinearilty Problem 1009080706050403020100 80 70 60 50 40 30 20 pre p o s t e f f And even if the lines aren’t parallel (interaction effect)... The result will still be biased.

9. Model Specification l If you specify the model exactly, there is no bias. l If you overspecify the model (add more terms than needed), the result is unbiased, but inefficient l If you underspecify the model (omit one or more necessary terms, the result is biased.

10. Model Specification y i =  0 +  1 X i +  2 Z i For instance, if the true function is

11. Model Specification y i =  0 +  1 X i +  2 Z i For instance, if the true function is And we fit: y i =  0 +  1 X i +  2 Z i + e i

12. Model Specification y i =  0 +  1 X i +  2 Z i For instance, if the true function is: And we fit: y i =  0 +  1 X i +  2 Z i + e i Our model is exactly specified and we obtain an unbiased and efficient estimate.

13. Model Specification y i =  0 +  1 X i +  2 Z i On the other hand, if the true function is

14. Model Specification y i =  0 +  1 X i +  2 Z i On the other hand, if the true model is And we fit: y i =  0 +  1 X i +  2 Z i +  2 X i Z i + e i

15. Model Specification y i =  0 +  1 X i +  2 Z i On the other hand, if the true function is And we fit: y i =  0 +  1 X i +  2 Z i +  2 X i Z i + e i Our model is overspecified; we included some unnecessary terms, and we obtain an inefficient estimate.

16. Model Specification y i =  0 +  1 X i +  2 Z i +  2 X i Z i +  2 Z i And finally, if the true function is 2

17. Model Specification y i =  0 +  1 X i +  2 Z i +  2 X i Z i +  2 Z i And finally, if the true model is And we fit: y i =  0 +  1 X i +  2 Z i + e i 2

18. Model Specification y i =  0 +  1 X i +  2 Z i +  2 X i Z i +  2 Z i And finally, if the true function is: And we fit: y i =  0 +  1 X i +  2 Z i + e i Our model is underspecified; we excluded some necessary terms, and we obtain a biased estimate. 2

19. Overall Strategy l Best option is to exactly specify the true function. l We would prefer to err by overspecifying our model because that only leads to inefficiency. l Therefore, start with a likely overspecified model and reduce it.

20. Steps in the Analysis 1.Transform pretest by subtracting the cutoff. 2.Examine the relationship visually. 3.Specify higher-order terms and interactions. 4.Estimate initial model. 5.Refine the model by eliminating unneeded higher-order terms.

21. Transform the Pretest l Do this because we want to estimate the jump at the cutoff. l When we subtract the cutoff from x, then x=0 at the cutoff (becomes the intercept). X i = X i - X c ~

22. Examine Relationship Visually Count the number of flexion points (bends) across both groups...

23. Examine Relationship Visually Here, there are no bends, so we can assume a linear relationship. Count the number of flexion points (bends) across both groups...

24. Specify the Initial Model l The rule of thumb is to include polynomials to (number of flexion points) + 2. l Here, there were no flexion points so... l Specify to 0+2 = 2 polynomials (i.E., To the quadratic).

25. y i =  0 +  1 X i +  2 Z i +  3 X i Z i +  4 X i +  5 X i Z i + e i The RD Analysis Model y i = outcome score for the i th unit  0 =coefficient for the intercept  1 =linear pretest coefficient  2 =mean difference for treatment  3 =linear interaction  4 =quadratic pretest coefficient  5 =quadratic interaction X i =transformed pretest Z i =dummy variable for treatment(0 = control, 1= treatment) e i =residual for the i th unit where: ~~~~ 22

26. Data to Analyze

27. Initial (Full) Model The regression equation is posteff = 49.1 + 0.972*precut + 10.2*group - 0.236*linint - 0.00539*quad + 0.00276 quadint Predictor Coef Stdev t-ratio p Constant 49.1411 0.8964 54.82 0.000 precut 0.9716 0.1492 6.51 0.000 group 10.231 1.248 8.20 0.000 linint -0.2363 0.2162 -1.09 0.275 quad -0.005391 0.004994 -1.08 0.281 quadint 0.002757 0.007475 0.37 0.712 s = 6.643 R-sq = 47.7% R-sq(adj) = 47.1%

28. Without Quadratic The regression equation is posteff = 49.8 + 0.824*precut + 9.89*group - 0.0196*linint Predictor Coef Stdev t-ratio p Constant 49.7508 0.6957 71.52 0.000 precut 0.82371 0.05889 13.99 0.000 group 9.8939 0.9528 10.38 0.000 linint -0.01963 0.08284 -0.24 0.813 s = 6.639 R-sq = 47.5% R-sq(adj) = 47.2%

29. Final Model The regression equation is posteff = 49.8 + 0.814*precut + 9.89*group Predictor Coef Stdev t-ratio p Constant 49.8421 0.5786 86.14 0.000 precut 0.81379 0.04138 19.67 0.000 group 9.8875 0.9515 10.39 0.000 s = 6.633 R-sq = 47.5% R-sq(adj) = 47.3%

30. Final Fitted Model