Jeffrey S. Napierala Prof. Glenn D. Deane Department of Sociology, SUNY Albany Prof. Donald J. Hernandez Department of Sociology, Hunter College & CUNY Graduate Center Research was supported by a Grant from the Annie E. Casey Foundation Please do not cite or distribute this research without consent from the Authors

Outline The origins of this method An example using children's reading performance A “hybrid” approach Standard Errors Results Conclusions

Origins The audience for policy research often has a diverse background in statistics. To accommodate those without any proficiency, the presentation must be simple and transparent. To satisfy those with advanced proficiency, current methods must be used. We want to have our cake and eat it too…

Origins At one end of the spectrum: Sample means/proportions are simple and easy to understand …but may not clearly translate into effective policies… Towards the other end: A standard regression approach has obvious advantages, but results can be difficult to explain to general audiences.

Origins Hernandez, Donald J. Double Jeopardy: How Third-Grade Reading Skills and Poverty Influence High School Graduation. Baltimore: Annie E. Casey Foundation

An Example… Policy Question: How much would income transfers increase the percentage of (3 rd Grade) students reading at the proficient level? Data: National Longitudinal Survey of Youth (NLSY) 4,060 children in 2369 families followed across about 30 years Dependent variable: Peabody Individual Achievement (PIAT) Reading Comprehension Test. Continuous variable ranging from Methods: Weighted, linear GEE models with a correction for clustering within families.

An Example: Model Specification Two models to highlight approach: 1. A “Base” model with just income and income squared predicting PIAT Reading Comprehension. 2. A “Full” model with controls for the 1) non-linear effect of mother’s education, 2) health insurance coverage, 3) whether a child attended head start or preschool, 4) the “quality” of their neighborhood, 5) race, 6) sex, and 7) year of interview.

A “hybrid” Approach Let’s utilize the power of a regression, but keep the presentation as simple and flexible as possible. 1. Create a statistical model of the outcome. 2. “Simulate” new outcomes using the relevant parameters of the model. 3. Meaningfully summarize the distributions before and after the “simulation” using simple statistics/tabulations. 4. Compare the summary statistics/tabulations.

A “hybrid” Approach: Standard Errors Using the common formula for the standard error of a proportion (with and without a Design Effect multiplier of 1.388). A Monte-Carlo approach to incorporate error from the sample regression and sample proportion. First, the effect(s) of covariates are added into the original score (the raw DV) by sampling from a normal distribution with a mean and s.d. from the regression. Rates (of reading proficiency) are computed then additional sampling error is introduced. After 5000 iterations, the S.E. of the distribution is computed from all the rates.

Also, to compare rates from the same group of children (before and after “simulations”) a “paired proportions” t-test is used (Altman 1997). Source: Altman, Douglas G Practical Statistics for Medical Research. London: Chapman & Hall. A “hybrid” Approach: Standard Errors

Results…

Conclusions The “hybrid” approach has a few notable advantages over other methods… The independent effect of income on reading proficiency is much (much) less than might be expected from looking at bivariate or univariate results. We expect that about 4% ( ; 95% C.I.) more kids in poverty would read proficiently if their families were given additional income to move them out of povery.

Thank You!