Teacher Effectiveness in Urban Schools Richard Buddin & Gema Zamarro IES Research Conference, June 2010

2 Research Questions How does teacher quality vary across classrooms and across schools? How do teacher qualifications affect teacher effectiveness? –Traditional measures like experience and education –Teacher licensure test scores Are the best teachers concentrated in a few schools? Are teachers with single-subject credentials more effective in middle school than teacher with multi-subject credentials? Does the quality of high school ELA instruction spill over into math and visa versa?

3 Focus on Los Angeles Unified School District LAUSD is 2 nd largest district in the US –690,000 students –36,000 teachers –800 schools Eight years worth of student achievement data for students in grades 2 through 11 –Achievement scores in reading and math each year –Students records linked longitudinally –Teachers are matched with individual students Teacher licensure tests scores as well as traditional teacher credentials like experience and educational background

4 Teacher Licensure Test Scores Like other states, California relies on expert panels of teachers, administrators, and content experts to develop tests and set cut scores California requires passing scores on three tests as part of teacher certification –General aptitude –Reading pedagogy (elementary only) –Subject matter knowledge Credentialing commission receives licensure scores directly, but they are not reported to districts or teaching candidates –potential teachers know whether they pass or not –districts know if a candidate is certified or not

5 How Do Teaching Candidates Perform on California Licensure Tests?

6 Teacher Pass Rates Rise with College Grades

7 Empirical Issues Estimation methods control for prior learning and isolate the contribution of current teacher and school inputs to student learning No ideal method—more flexible models require more years data and involve restrictions that may offset benefits We used three types of models 1.Contemporaneous value-added: student achievement is a function of student and teacher fixed effects 2.Value-added gains: Similar but uses year-to-year gains as dependent variable 3.Lagged achievement: Current student achievement as a function of prior year achievement Similar results for all models for elementary, middle, and high schools—focus on elementary here

8 Low Scoring Schools Have Much Different Mix of Students and Teachers than High Scoring Schools More minority, English learner, and low income students in low scoring schools Weaker teacher credentials in low scoring schools –Less experience –Fewer advanced degrees –Lower licensure scores In past two years, LAUSD has cut young teaching staff –With fewer young teachers, most of the gap in teacher characteristics has disappeared –Unclear how this will affect achievement Characteristic Lowest Quartile Schools Highest Quartile Schools Student Characteristics Reading Percentile Math Percentile Black Hispanic LEP Parents not high school graduates Teacher Characteristics Years of Experience Experience < 3 yrs Black Hispanic Master's/Doctorate General Ability (standardized) Subject Matter (standardized) Reading Pedagogy (standardized)

9 Teacher Effects Are Much Larger than School Effects High quality teachers are spread across schools and not concentrated in a few schools ReadingMath #1. Student & Teacher Fixed Effects Student (  Student ) Teacher (  Teacher ) #2. Student & School Fixed Effects Student (  Student ) School (  School )

10 Teacher quality distribution has a large interquartile range in reading and math Students with a teacher at the 25th percentile of the teacher quality distribution score 9 to 12 percentile points lower than do similar students assigned to a teacher at the 75th percentile Some teachers get much better classroom results than others Teacher Quality Has A Large Effect on Student Test Scores, After Controlling for Student Characteristics

11 Do Traditional Measures of Teacher Qualifications Predict Classroom Performance? Teacher experience is weakly related to student achievement –A five year increase in experience increases achievement by less than one percentage point Teacher education level has no effect on student achievement Also, little evidence that student/teacher match affects classroom achievement –Race/ethnic match –Gender match –High income students with better educated teacher

12 Do Higher Scoring Teachers Have Better Student Achievement in Their Classrooms? Licensure scores are unrelated to student achievement –General aptitude, subject-matter, and reading pedagogy score have no significant effects on student achievement –Also looked at scores for each test separately and different tests at different grades, but the scores did not matter in either reading or math Examined whether teachers who initially failed licensure tests had lower student achievement in their classrooms –Initial failures were also insignificantly linked to achievement scores

13 Licensure tests and teacher performance are available only for teachers who pass the tests –Potential teachers who fall below the cut scores on the licensure tests might have worse classroom outcomes than teachers who ultimately surpass those cut scores –Raising the cut scores of current tests would not improve student achievement Stricter licensure screens might do a better job of differentiating teachers –Tighter screens might significantly reduce the pool of available teachers and discourage qualified individuals from pursuing teaching careers –Current licensure results provide no indication of how well teachers will do in the classroom Limitations of Licensure Test Results

14 Special Findings from Middle and High School Analysis Middle school results –Teachers with elementary school credentials had slightly better success than did teachers with secondary school credentials –Higher general knowledge or subject-matter skill (as measured on licensure exam) did not affect classroom effectiveness High school results –Achievement outcomes differ substantially from teacher to teacher, and the effects of a good teacher spill over from one subject to another –Even here, effectiveness was unrelated to licensure scores

15 How does teacher quality vary across classrooms and across schools? Considerable variation in teacher quality across classrooms and less variation across schools Traditional teacher quality measures and licensure test scores explain little of the learning gap between these schools The teacher quality gap between low- and high- scoring schools is only about one percentage point –Efforts to improve the teaching performance in low performing schools are unlikely to succeed, if they rely entirely on traditional measures of teacher quality –Reshuffling teachers with better qualifications to low performing schools is unlikely to close the achievement gap

16 Final Thought Teacher quality measures may be weakly tied to student achievement because teacher effort is inversely related to those measures Perhaps more experienced or smarter teachers are better able to teach, but they do not consistently perform in the classroom Perhaps the current compensation system provides too little incentive for the “best” teachers to deliver their best performance in the classroom on a consistent basis