1. Research on teacher pay-for-performance Patrick McEwan Wellesley College pmcewan@wellesley.edu (Also see Victor Lavy, “Using performance-based pay to improve the quality of teachers,” Future of Children, vol. 17, no. 1.)

2. Pay-for-degrees-and-years-of- experience Little evidence that degrees and experience are systematically correlated with student outcomes But, growing evidence that teachers vary in their ability to increase student test scores or “value-added” So what? Weak monetary incentives for current teachers to improve or maintain effectiveness Weak monetary incentives for highly-effective teachers to enter teaching profession

3. Pay-for-performance? An attempt to (partially) tie remuneration to objective measures of student performance, often test scores Many short-lived and poorly-researched experiences with “merit pay” in U.S. More international experiences, especially Israel, Kenya, Mexico, and Chile This presentation briefly summarizes extant research

4. Policy ingredients Incentives for individuals or groups? What outcome measure(s)? Unadjusted or adjusted student test score(s) Principal or peer evaluations What performance targets? Absolute targets vs. relative “tournaments” What reward? How much? “One-shot” bonus vs. permanent wage bump Pecuniary vs. non-pecuniary awards

5. Potential benefits of pay-for- performance Increased teacher effort Increased alignment between school and classroom activities and goals Improved recruitment and retention of highly-effective teachers

6. Potential pitfalls Inaccurate measurement of teachers’ or schools’ “value-added” School(s), teacher(s), peers, families, noise? Negative effects on motivation or cooperation “Gaming” behavior to win awards Focus on few or easier-to-affect outcomes Focus on “bubble” children close to proficiency thresholds Test coaching Strategic exclusions from testing Outright cheating Twinkies on test day!

7. Criteria for evaluating research (on performance incentives) Internal validity Does pay-for-performance cause short-term and longer-term increases in student test scores? Does pay-for-performance cause longer-term improvements in recruitment and retention of highly-effective teachers? Key question: What is the counterfactual? External validity Can causal research findings be generalized to new students, districts, and variations of the policy?

8. Group incentives: U.S. evidence Dallas (Ladd 1999) School-based bonus in early 1990s Ranked schools by aggregated outcomes, adjusting for “socioeconomic status” Awarded bonuses to top 20% of schools, $1000 per principal/teacher Test score pass rates increased more quickly than comparison cities, for whites and Latinos

9. Group incentives: other evidence Kenya (Glewwe et al. 2003) Of 100 primary schools 50 randomly assigned to participate in incentive program Participating schools ranked by test score levels and gains About 50% of participating schools receive awards (1/3 monthly salary per teacher) Small, short-run increases in test participation and test scores No test score differences after program end Increase in test prep, but no change in pedagogy

10. Group incentives: other evidence Israel (Lavy 2002) 62 schools competed for $1.5 million in awards 1/3 of schools received awards, based on relative gains in test 75% of school award going to teacher salary, rest to school fund Compared to “similar” group of comparison schools, participating schools showed gains in test scores

11. Individual incentives: U.S. evidence Not much (yet!) National sample (Figlio and Kenny 2007) Rough measures of merit pay usage are correlated with student achievement Small effects that could simply indicate that “good” schools also tend to implement merit pay

12. Individual incentives: other evidence Israel (Lavy 2004) Individual teachers, by subject, compete for bonuses Ranked according to SES-adjusted class performance, half received awards Mixed evidence of effects on test scores Mexico (McEwan and Santibanez 2005) “Teacher Ladder” awarded large, permanent wage increases for meeting evaluation criteria Including teacher test scores, student test scores, peer evaluations Weak evidence that incentives improved test scores

13. A summary Some evidence of short-term test score effects, but mixed and best evidence outside U.S. Some evidence of unintended consequences Little evidence on long-term impacts on teacher recruitment and retention Little evidence on policy costs, or relative cost- effectiveness

14. Necessary but not sufficient conditions to implement pay-for-performance School and teacher performance measures That include a range of outcomes That provide unbiased measures of schools’ or teachers’ contributions to outcomes That minimize “noise” That are (relatively) transparent, and perceived to be fair by participants