Human Capital Policies in Education: Further Research on Teachers and Principals 5 rd Annual CALDER Conference January 27 th, 2012

Estimating the Effect of Leaders on Public Sector Productivity: The Case of School Principals Gregory Branch, Eric Hanushek, and Steven Rivkin, January 2012

Questions Is there substantial variation in principal effectiveness? Does the variation in principal effectiveness differ by the share of low income students in a school? Do more effective principals make better personnel decisions? Are “effective” principals more likely to leave high poverty schools?

UTD Texas Schools Project Stacked panels of students and staff Annual student testing Student demographic characteristics Information on staff  Follow principals, teachers, and students in Texas public schools  Very large samples: 7,420 unique principals and 28,147 principal-year observations in

Principals Change Frequently Quartile Principal in 1 st year Principal with 6+ years Poverty lowest highest Math achievement worst best

Estimation of Variation in Principal Quality Non-random selection of principals and students Control for observed student characteristics and prior achievement Make principals comparable in terms of tenure

No Simple Solutions Alternative approaches to estimation 1. Fixed effects for principals 2. Fixed effects for principals and schools 3. Direct estimation of quality variation 4. Validation with teacher turnover analysis

Alternative Value-Added Estimates Principal Spell Fixed Effects Regress math score on lagged math score, student demographic variables, principal-by-spell fixed effects

Alternative Value-Added Estimates Principal Spell Fixed Effects Regress math score on lagged math score, student demographic variables, principal-by-spell fixed effects Principal Spell and School Fixed Effects

Fixed Effects Estimates (s.d.) (without school fixed effects) Poverty quartile First three years tenure Lowest nd rd 0.21 Greatest0.26 All0.21

Test Measurement Issues Random measurement error  Use Bayesian shrinkage estimator Basic Skills Tests  Reweight to allow for initial achievement

Measurement Effects (s.d.) First three years tenure Unadjusted0.21 Shrunk0.20 Re-weighted0.27 Shrunk/re-weighted0.24

Alternative Fixed Effects Estimates (s.d.) Poverty quartile Up to six years tenure Lowest nd rd 0.23 Greatest0.28 All0.22

Alternative Fixed Effects Estimates (s.d.) Poverty quartile Up to six years tenure With school fixed effects Lowest nd rd Greatest All

Why is variance higher in high poverty schools? Larger variation in underlying principal skills in high poverty schools Or Principal quality differences translate into larger differences in test scores in high poverty schools

Direct Estimates of Variance If principal changes and if principal effects outcomes, pattern of student growth should change If other school factors are uncorrelated with principal change (partially testable), can obtain lower bound estimate of principal effectiveness.

Direct Lower Bound Estimates Adjacent yearInterrupted year With student controls* Different principal *Student ethnicity, gender, SES, ELL, special education, mobility measures

Range of Alternative Estimates Principal Fixed Effect Principal and School Fixed Effect Direct Estimates TotalWithin school Standard Deviation

Added Analysis – Principal Quality 1.Better principals => better teacher transitions 2.High mobility of both best and worst in most disadvantaged schools 3.Substantial number of worst principals become principal elsewhere.

Summary Purposeful sorting complicates estimates of principal quality and quality of leavers Substantial variation in estimates of principal quality (fixed effects and direct) Higher variance in high poverty schools Not due to test measurement complications Effects are large