1 Concentration of Low-Performing Students (4 th grade Math, 2002)

2 Teacher Sorting in NYS Elementary Schools (Math 2002)

3 85 Percent of New York State Teachers Take First Job within 40 Miles of Home

4 Policy Challenge How do public schools:  Attract potentially excellent teachers to teaching, especially in traditionally difficult-to- staff schools?  Provide the skills and experiences that help potentially excellent teachers develop into excellent teachers in those schools?  Retain strong teachers, especially in traditionally difficult-to-staff schools?

5 Data Collection Program analysis  State documents, program documents, accreditation reports, interviews, surveys, course syllabi; 18 institutions that prepare most traditional route teachers for NYC schools  Alternate route programs that prepare about 25 percent of NYC teachers Surveys  Graduating program participants (2004), new NYC teachers (2005), second year and former teachers (2006) Administrative data  All NYC teachers ; rich measures of teacher qualifications, including certification exams and areas, teacher retention.  Student achievement ; value-added scores in math and ELA, grades 4-8 linked to teachers.  Data on schools and students

6 Examining Teacher Preparation Paths What is the relative effectiveness of alternatively certified teachers compared to those prepared in traditional college preparation programs?  Selection: alternative certification teachers appear to have stronger academic credentials  Preparation: traditional preparation teachers have an undergraduate teacher education program; alternative certification teachers a 7 week summer program

7 Data and NYC Context Achievement gain data from NYC for through – student gains in grades 4-8 linked to teacher. DEFINING PATHWAY - The route teachers follow into their first teaching job.  College Recommended  Individual Evaluation (transcript review)  NYC Teaching Fellows  Teach for America  Temporary License (uncertified)  Other (i.e. reciprocity, other Transitional B programs, provisional) Caveat: All teachers in New York must ultimately complete the same set of requirements to receive certification. A lot of overlap and a lot of variation within pathways. For example, this past year about 1/4 th of traditional did student teaching as teacher of record.

8 Entering NYC Teachers by Pathway: Teaching Fellows Replace Temp License

9 Average Certification Exam Scores First Taking: (2004; Passing= 220,SD=~30)

10 Characteristics of Students by Pathway of First-Year Teachers

11 Methodology School, Grade, and Year Fixed Effects A isgjt =  0 +  1 A isgj(t-1) +  2 K it +  2 C j +E j  1 +  1 P j +π g +π t +π s +ε isgjt Specification Checks  Interactions with Experience  Gain as the outcome  Gains with student fixed effect  Only those with three or fewer years of experience  School by year by grade fixed effects  two stages  Analysis of the importance of school and class by omitting school fixed effects and classroom controls Alternative Definitions of Pathway Subgroups of Students Incorporating Turnover

12 Results of Student, Peer and Experience Variables, Base Math Model with grade, year & school fixed effects (n=960,970) Student MeasuresClass Average MeasuresTeacher Experience Lag score0.659***Lag score0.154***20.048*** *** (0.00) (0.01) Lag sq0.001Std Dev *** *** (0.00) (0.01) Female-0.009*** *** *** (0.00) (0.01) Other Eth-0.088***Other Eth *** *** (0.01) (0.12)(0.01) Asian0.114***Hispanic-0.182***60.080*** *** (0.00) (0.03)(0.01) Hispanic-0.083***Black-0.230***70.068*** *** (0.00) (0.03)(0.01) Black-0.109***Asian-0.061*80.075*** *** (0.00) (0.03)(0.01) English Lang ***English0.042**90.070*** *** (0.00) (0.02)(0.01) Free lunch-0.046***Free lunch-0.057*** *** *** (0.00) (0.01) Lag Abs-0.004***Lag Abs-0.010*** ***> *** (0.00) (0.01) Lag Susp ***Lag Susp ** (0.00) (0.03)

13 Total Pathway value-added findings for math with school-fixed effects (relative to College Recommended) *= significant at 10% **= significant at 5% ***= significant at 1%

14 Specifications checks, Base Model for Math Gain <= 3 Years Gains with Stud FE Schl/Year /Grade FE Two Stage 2 Stage Exp<=3 IE-0.012**-0.028** **-0.015**-0.032** (0.005)(0.011)(0.006)(0.005)(0.006)(0.013) Fellows-0.023***-0.025*** ***-0.025***-0.031*** (0.008)(0.009) (0.007)(0.009)(0.010) TFA (0.015)(0.016)(0.017)(0.015)(0.016)(0.018) Temp-0.021***-0.026***-0.010**-0.017***-0.023***-0.034*** (0.004)(0.006)(0.004) (0.005)(0.007) Other-0.021***-0.025***-0.018**-0.015**-0.026***-0.039*** (0.008)(0.010)(0.008)(0.007)(0.009)(0.012)

15 Effects on Student Achievement by Pathway by Experience, Math, Grades 4-5 (Coll. Rec. 1 yr =0;) *Caution – small sample size for third-year TFA makes interpretation of results tricky

16 Effects on Student Achievement by Pathway by Experience, Math, Grades 6-8 (Coll. Rec. 1 yr =0;) *Caution – small sample size for third-year TFA makes interpretation of results tricky

17 How Important Are These Effects? Statistical significance Comparison to other effects  1 st -year Teaching Fellows vs. CR teachers—0.04 difference in student gains in grade 4-5 math Similar in magnitude to 80% of the gain attributed to 1 st year of teaching experience (0.048)  3rd year TF vs. CR teachers—0.056 difference in student gains in 6-8th grade math (0.091 minus 0.035) greater in magnitude than 1 st year of experience gain

18 How Important Are These Effects (con’t)? Conversion to scale score For example, 7th grade math exam in 2002 had mean score=670, and std dev = 48. The advantage of 3 rd year Teaching Fellow versus 3 rd year CR from prior example  equals about 2.7 scale points (48 x.056)  a 2.7 scale point increase moves past 2.5% of students at mean

19 Given the effectiveness of teachers improves with their first few years of experience, any such differences in attrition could affect differences in value-added across pathways. Is there differential attrition by pathway?

20 Grades 4-8 Teacher Attrition Following the 1 st Four Years of Teaching, Cohorts Adjusted for grade School and Year

21 Simulated Student Achievement Gains by Pathway Accounting for Differential Attrition, Math 4-5

22 Simulated Student Achievement Gains by Pathway Accounting for Differential Attrition, Math 6-8

23 Summary Reduced entry requirements are attracting a substantial number of teachers with strong qualifications. Currently these teachers appear to be more effective in math than ELA and more effective in middle school grades than in upper elementary. Alternate route teachers are not, on the whole, doing better than college recommended teachers, and are doing worse in some cases. However, few of the differences across pathways are statistically significant. Need to look at the effects of characteristics of pathway on selection and preparation to figure out how to maximize effectiveness…

24 Not satisfying yet… Methodological question about the social significance of effects  Estimating the size of teacher effects (separating from schools)  Measurement error Variation in effects and experiences within pathways - trying to get at this with larger study

25 Childhood Program Math Value-Added (Grades 4-8, programs with at least 50 observations, with school fixed effects and controls for teacher attributes)

26 Childhood Program ELA Value-Added (Grades 4-8, programs with at least 50 observations, with school fixed effects and controls for teacher attributes)

27 Relationship of Program Features to Student Math Achievement Constructed variables related to preparation, based on 1 st year teacher survey responses Variables/factors linked to 5 aspects of preparation  Program structure  Preparation for teaching math  Preparation in learning/development, including preparation in classroom management  Preparation to teach linguistically and culturally diverse students  Field experiences

28 Content-Specific Preparation  General opportunities to learn about the teaching of math (Genmath): includes 12 items related to opportunities to:  Learn typical difficulties students have with place value  Study, critique, or adapt math curriculum materials  Study or analyze student math work

29 Field Experiences Field-job congruence:  My experiences in schools were similar to my current job in terms of grade level  My experiences in schools were similar to my current job in terms of subject area Field-program coherence:  My program lacked a sense of coherence among courses and between courses and field experiences  What I learned in methods courses reflected what I observed in my field experiences or in my own classroom

30 Learning About Learning Learning factor:  Includes 8 items related to learning about child development and learners, including special education, classroom management. Relative emphasis on:  Develop strategies for handling student misbehavior  Develop specific strategies for teaching English language learners (those with limited English proficiency)

31 Opportunities Grounded in Practice Relative emphasis on opportunities to  Listen to an individual child read aloud for the purpose of assessing his/her reading achievement  Plan a guided reading lesson  Study or analyze student math work Relative opportunities to:  Review New York City reading curriculum  Review New York City mathematics curriculum

32 Value-Added of Preparation Program Features (Math, 2005 cohort) Controlling for  student prior scores  a rich set of student characteristics  a rich set of aggregate student characteristics at the classroom level  teacher attributes  pathway

33 Effect of Program Attributes on 4 th – 8 th math student achievement VariablesAll pathsCR only Opportunities grounded in practice0.07**0.061~ Opportunities to study NYC curriculum 0.041** Field experience/job congruence0.041**0.045* General Math0.064* Relative Emphasis on teaching ELL0.03*0.06** ** p<.01, * p<.05, ~ p<.10

34 With important caveats… Analysis depends on significant variation in experiences. Doesn’t mean other factors aren’t important; Opportunities to learn to teach math, for example, may be very important but not enough variation in this factor to be significant. Differences in quality of experience, rather than quantity, may matter most. Yet quality is hardest to measure in large scale studies. We are early in the process of this analysis.

35 Challenge to teacher educators The importance of creating opportunities for students to engage in practice-based activities (e.g., analyzing student work in math, conducting individual reading assessments, studying NYC curriculum).