Strategies for estimating the effects of teacher credentials Helen F. Ladd Based on joint work with Charles Clotfelter and Jacob Vigdor CALDER Conference, Oct. 4, 2007

Basic value-added model Definition: A it = a A it-1 + b TQ it + c X it + error it where A = student achievement (i.e. test score) ; and TQ = teacher qualifications X = control variables Justification: Education is a cumulative process a = estimate of persistence of knowledge from one year to the next. a =1 => complete persistence (no decay) a = 0 => no persistence (100 percent decay) b = estimate of the effects of the qualifications of the student’s teacher in year t on her achievement in year t. (Model assumes a and b are constant across years)

Three papers – NC data Cross sectional data – fifth graders “Teacher-Student Matching and the Assessment of Teacher Effectiveness” Longitudinal data – fourth and fifth graders, multiple cohorts of students “How and Why Do Teacher Credentials Matter for Student Achievement?”” Course-specific achievement in high school courses – multiple cohorts “Teacher Credentials and Student Achievement in High School: A Cross- Subject Analysis with Student Fixed Effects” Note. Student achievement is normalized by grade, year and subject so that the mean is 0 and the SD = 1.

Challenges for all three papers Data – Identification of each student’s teacher Elementary schools – EOG tests High schools – E0C tests In both cases, we start with proctor of the test but we keep the observation only if we are quite confident that the proctor is the relevant teacher. `(> 75 % match rate in both elementary schools and high schools) Middle schools – identification not feasible Estimation – Non-random sorting of teachers and students among class rooms. “Positive” sorting => upward biased coefficients of teacher credentials

Cross sectional model – 5 th graders Strategies to reduce bias of estimates: Add an extensive set of student covariates Rich set available in NC data – e.g. education level of parents, T.V. watching Include school fixed effects Rules out bias from teacher-student sorting across schools Restrict sample to schools with evenly balanced classroom Reduces bias from sorting across classrooms within schools.

Coefficients of teacher experience - Math (all coefficients are statistically significant.) Years of experience (Base = 0 years) Student covariates School fixed effects Restricted sample With school fixed effects > Observations60,656 25,711

Longitudinal – grades 4 and 5 Achievement levels (A it ) or achievement gains (A it - A i,t-1 ) Models 1-3 (of 5) No fixed effects 1. Levels (with prior year achievement). Upward biased coefficients because of teacher student matching; potential bias from lagged achievement With school fixed effects 2. Levels. Better but problem of matching within schools remains and potential bias from lagged achievement; direction of bias unclear (see earlier paper) 3. Gains. Downward bias from misspecified persistence variable

Longitudinal Data (cont.) Models 4 and 5 (preferred) Full use of the longitudinal aspect of the data With student fixed effects 4. Levels (but no lagged achievement). Lower bound estimates of teacher credentials 5.Gains. Upward bound estimates of teacher credentials

Teacher experience Coefficients from models 4 and 5 All are statistically significant Base= no experience Math Reading 1-2 years0.057 / / years0.072 / / years / / years0.082 / / years0.092 / / / / 0.089

High school cross-subject analysis Subjects – algebra 1, English I, biology, geometry, ELP Strategy – at least three test scores for every student; include student fixed effects Equivalent to estimating: (A is -A i *) = b (TQ is -TQ i *) + error terms. Where A* is the mean for the student. Consider one potentially problematic error term: (e is -e i *). Think of e as unmeasured student ability. Potential concern if ability for a given student differs by subject AND teachers are distributed in a systematic way by the relative ability of students Based on empirical tests reported in the paper, we have reasonable confidence in our approach.

Coefficients of teacher experience in high school courses Years of experience (base = 0 years) Model with student fixed effects More than Cf. rising coefficients with with teacher fixed effects