1. Understanding Student Achievement: The Value of Administrative Data Eric Hanushek Stanford University

2. Big Issues in School Policy Debates  Relating analysis to policy interests  Confidence in causation  Generalizability

3. Analytical designs  Random assignment experiments  Natural experiments  “Data solutions”  Trade-offs Credibility Expense Questions that can be addressed

4. UTD Texas Schools Project  Multiple cohorts followed 1993-2002  Annual achievement in grades 3-8 (TAAS math and reading)  Each cohort > 200,000 students in over 3,000 schools  Augmented with district data

5. Examples of Topics  Teacher quality variations  Charter schools  Not discussed School choice and mobility Special education Teacher mobility Racial composition Peer achievement

6. Existing Evidence on Teacher Quality  Substantial variation in teacher quality  Observable characteristics of teachers explain little of the variation  Salary and other factors affect teacher transition probabilities  No evidence on transitions and teacher quality

7. Questions Addressed  What is variation in teacher quality? Measurable characteristics?  Do urban schools lose their best teachers? Quality by transitions  Do districts hire the best teachers?

8. Basic model

9. Measurement Error and Calculation of Variance of Teacher Quality  Observe teachers in two years:  Correlation across years:

10. Estimated Variance in Teacher Quality Lonestar District Within district Within school and year unadjusted demographic controls unadjusted demographic controls Teacher-year variation 0.2100.1790.1090.104 Adjacent year correlation 0.5000.4190.4580.442 Teacher quality variance / (s.d.) 0.105 (0.32) 0.075 (0.27) 0.050 (0.22) 0.047 (0.22)

12. Conclusions on Teacher Quality  Very large differences among teachers Differences within schools much larger than between schools  Conventional measures not good index of quality (master’s degree, certification test)  Observable characteristics First year of experience Teacher-student race match  Common assumptions about market for teachers not correct Best do not leave Districts with advantages do not use them

13. Popularity of charter schools  3,000 charter schools  40 states plus DC since 1991  1 percent of total students  10 percent of size of private school market  7+ percent rate of closure

14. Evaluation issues  Most analysis of entry and participation  No reliable information on performance  Difficulty of selection issue  Very political

15. Evaluation approaches  Model selection process [Heckman (1979)]  Instrument for attendance [Neal(1997)]  Intake randomization [Howell and Peterson (2002)]

16. Difficulties with traditional approaches  Difficult to find factors affecting attendance but not achievement  Cannot handle treatment heterogeneity

17. Empirical framework  Mean differences in individual value-added Identify charter school from individual entry-exit Consider time varying effects associated with charter school movements  Heterogeneity across schools  Consumer responsiveness to quality

18. Charter enrollment 19972001 4 th grade0.2 %0.8% 7 th grade0.2%0.9%

19. Participation rates by race/ethnicity 19972001 Blacks0.8%2.2% Hispanics0.1%0.6% Whites0.0%0.4% Low income0.3%0.8%

20. Charters by vintage (analytical) 199719981999200020012002Total one 171070834347270

21. Charters by vintage (analytical) 199719981999200020012002Total one 171070834347270 two 2169697840214

22. Charters by vintage (analytical) 199719981999200020012002Total one 171070834347270 two 2169697840214 Three 021586873166 Four 0121586692 Five+ 0013172243

23. Charter school effect Charter-0.17 Age 1-0.33 Age 2-0.25 Age 3-0.08 Age 40.00 Age 5 or more0.02

24. Demographically Adjusted School Quality

25. Do parents make good decisions?  Parents cannot see value added  Considerable mobility/exiting  Models: Exit=f(quality, age, year, race, grade)

26. Parental Choice (linear probability of exit) Student characteristics Student + peer characteristics Student + peer characteristics + peer achievement School quality 0.0020.006 School quality x charter -0.152-0.142-0.138

27. Parental Choice (linear probability of exit) Student characteristics Student + peer characteristics Student + peer characteristics + peer achievement School quality 0.0020.006 School quality x charter -0.152-0.142-0.138 high income -0.187 low income -0.096

28. Conclusions on Charter Schools  Difficult start-up period  Mean performance regular ≈ charter after two years  Heterogeneity in both markets  Parents react to quality in charter market Low income reaction one half upper income

29. Administrative data  Pros Broader generalizability Understanding heterogeneity Perhaps less costly  Cons Requires structure (e.g., linearity, time pattern of achievement) Regulatory problems (confidentiality) Data quality issues

30. Papers on Teacher Quality and Charter Schools  www.hanushek.net or www.nber.org www.hanushek.netwww.nber.org Hanushek, Eric A., John F. Kain, Daniel M. O'Brien, and Steve G. Rivkin. 2005. "The market for teacher quality." National Bureau of Economic Research, Working Paper No. 11154, (February). Hanushek, Eric A., John F. Kain, Steve G. Rivkin, and Gregory F. Branch. 2005. "Charter school quality and parental decision making with school choice." National Bureau of Economic Research, (March).