Portability of Teacher Effectiveness across School Settings Zeyu Xu, Umut Ozek, Matthew Corritore May 29, 2016 Bill & Melinda Gates Foundation Evaluation of the Intensive Partnership Sites initiative

Motivation  Redistributing effective teachers at the center of several education policy initiatives  Teacher is the most important school input related to student learning  The distribution of effective teachers is uneven (recruiting, who moves, and to where)  Key assumption: Teachers effectiveness is portable  Students face different challenges in learning  School culture, environment and working conditions may affect teacher learning, practices, efforts, burnout, etc.  Literature  Jackson (2010), Jackson & Bruegmann (2009), Goldhaber & Hansen (2010)  Sanders, Wright & Langevin (2009) › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016

Research Questions  Do teachers retain their effectiveness across schools  On average  Across schools with similar settings  Across schools with different settings (by the direction of the change)  Teacher effectiveness measured by  Value-added  Settings defined by  School performance levels  School poverty levels  Conditional on teachers switching schools › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016

Preview of Findings  Among teachers who changed schools, on average their VA was unchanged or slightly improved  The same conclusion holds regardless of the similarity/difference between the sending and receiving schools or the direction of the move  High-performing teachers’ VA dropped and low-performing teachers’ VA gained in post-move years  This pattern is mostly driven by regression to the within-teacher mean and has little to do with school moves  Despite this pattern, high VA teachers still performed at a higher level than low VA teachers in post-move years › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016

Organization  Data and samples  Methodology  Findings  Summary and discussion › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016

Data  North Carolina through  Elementary level (4 th and 5 th grade math and reading teachers, self- contained classrooms)  Secondary level (algebra I and English I teachers, “Algebra I”, “Algebra I-B”, “Integrated Math II”, “English I” classrooms)  Florida through  Elementary level (4 th and 5 th grade math and reading teachers, “core courses” in a given subject)  Secondary level (9 th and 10 th grade math and reading teachers, “core courses” in a given subject) › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016

Sample restrictions  Remove charter schools  Remove students and teachers who changed schools during a school year (about 2-4% of obs)  Remove students with missing values on covariates  Keep classrooms with 10~40 students  Remove classrooms with >50% special education students › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016

Sample sizes North CarolinaFlorida ElementarySecondaryElementarySecondary Math21,1194,99929,9899,101 Reading21,1193,77529,3549,681 › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016 Number of Unique Teachers in the Analytic Samples

Two-Stage Analysis  Estimate teacher-year value-added  Difference-in-differences analysis › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016

Estimate Teacher VA  Test scores standardized by year, grade and subject (mean=0, sd=1)  (X) Covariates include:  1) grade repetition, 2) FRPL, 3) sex, 4) race/ethnicity, 5) gifted, 6) special education, 7) student school mobility and 8) grade level.  Bias (no school FE)  Noise (EB adjustment)  Alternative model specifications (achievement levels model) › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016

DiD  Three groups: non-movers, movers to a similar school setting, movers to a different school setting  FGLS, se clustered at the teacher level  (Y) Year and (T) teacher FEs  (X) Teacher experience (0-2, 3-5, 6-12, 13 or more years of exp)  (S) School quality (average peer VA)  (C) Classroom characteristics (FRL %, mean pretest score, sd of pretest score)  (Post) Post-move years indicator  (DP, DN) Indicators for school setting differences › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016

Define School Settings  School performance  NC: % students performing at or above grade level  FL: School performance scores based on both levels and growth  Standardized by year and aggregated across all years  School poverty  % FRPL  Aggregated across all years in which a teacher taught in that school  Change in school setting measures  ∆ = Receiving school – Sending school  Similar setting = within half a SD around the mean of the ∆ distribution  DP = 1 if ∆ > 0.25 (performance) or ∆ > 0.15 (poverty)  DN = 1 if ∆ < (performance) or ∆ < (poverty) › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016

Alternative DiD Specs  Last pre-move year and first post-move year  Between- vs. within-district moves  Replace the post-move indicator with individual year dummies (I t-1, I t-2, I t-3 …; I t+1, I t+2, I t+3 ) › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016

Distribution of Movers › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016 By school performance setting change

Distribution of Movers › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016 By school poverty setting change

Mover Characteristics › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016 NC elementary school teachers, by mobility status

Pre-Post Change in VA (elem) › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016 North CarolinaFlorida MathReadingMathReading All By school perf. Higher to lower Similar Lower to higher By school poverty Higher to lower Similar Lower to higher

Pre-Post Change in VA (sec) › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016 North CarolinaFlorida MathReadingMathReading All By school perf. Higher to lower Similar Lower to higher By school poverty Higher to lower Similar Lower to higher

By Pre-Move VA › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016 Actual year of move “Pseudo” move

By Pre-Move VA › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016 Elementary math teachersElementary math teachers (pseudo move)

By Pre-Move VA › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016 Elementary reading teachersElementary reading teachers (pseudo move)

By Pre-Move VA › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016 Secondary math teachersSecondary math teachers (pseudo move)

By Pre-Move VA › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016 Secondary reading teachersSecondary reading teachers (pseudo move)

Adjacent Year Correlations › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016 Correlation North CarolinaFlorida MathReadingMathReading Y t-2, Y t (0.426, 0.535)(0.232, 0.362)(0.314, 0.443)(0.111, 0.260) Y t-1, Y t (0.256, 0.420)(0.182, 0.354)(0.231, 0.369)(0.061, 0.213) Y t-+1 Y t (0.381, 0.537)(0.175, 0.358)(0.363, 0.487)(0.115, 0.264)

Pre-Post Comparisons of VA › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016 North Carolina

Pre-Post Comparisons of VA › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016 Florida

Summary  Among teachers who changed schools, on average their VA was unchanged or slightly improved  The same conclusion holds regardless of the similarity/difference between the sending and receiving schools or the direction of the move  High-performing teachers’ VA dropped and low-performing teachers’ VA gained in post-move years  This pattern is mostly driven by regression to the within-teacher mean and has little to do with school moves  Despite this pattern, high VA teachers still performed at a higher level than low VA teachers in post-move years › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016

Discussion  Teacher effectiveness does not appear to be hurt by moving to schools with different settings.  Multiple years of VA estimates can be used with other teacher evaluation data to identify effective teachers, capturing persistent teacher performance better and reducing post-move year shrinkage.  All results take teacher school changes as given. › Introduction › Data and Samples › Methodology › Findings › Summary and Discussion May 29, 2016