1. Introduction to NYC Teacher Data Initiative Training for Schools Fall 2008

2. 2 2 Session Objectives for Principals >Become familiar with the new Teacher Data Reports >Consider ways to incorporate this new tool into school-wide instructional improvement activities >Plan for sharing Teacher Data Reports with teachers >Locate support resources

3. 3 NYC Teacher Data Initiative Objectives >Develop statistical model to isolate the impact that individual teachers have on student achievement on state tests from factors outside of teachers ’ control >Provide this teacher data to principals and teachers in an accessible form >Support schools and teachers to use data:  One of many tools for instructional improvement  NOT for teacher evaluation

4. 4 Why Consider Teacher Level Data? Student achievement matters to life outcomes 8 TH GRADE PROFICIENCY RATINGS ARE PREDICTIVE OF HIGH SCHOOL REGENTS DIPLOMAS

5. 5 Why Consider Teacher Level Data? Teachers matter to student achievement NYC “ Value-Added ” teacher data shows what research consistently says: differences in teacher effectiveness have a significant impact on student performance. Range of NYC Teacher Value-Added Scores 2006-07 5 th grade -0.50-0.250.000.250.50 Math ELA -.31 -.45.41.29 Portion of a Proficiency Level above or below Predicted Gain

6. 6 6 Teacher Data Initiative Provides a New Lens on Teacher Effectiveness Purpose: To contribute another lens through which to look at teacher contributions to student learning Rationale: Teachers make a big difference, and Value-Added data provides a lens to focus on what teachers bring to students rather than what students bring to the classroom Framing Question: How might the Teacher Data Reports fit into existing school plans for instructional improvement and professional development? Teacher Data should not be viewed as a silver bullet, big initiative, or accountability metric. Rather it is a new tool available to principals and teachers to incorporate into their larger instructional and professional development plans.

7. 7 7 We Use An Array of Instruments to Determine Teacher And School-wide Professional Development Needs Classroom observations Lesson plans Participation in professional development Quality of student work products Student performance on state assessments No one measure gives us the full story, but the various pieces come together to create a more reliable picture. Areas of convergence and dissonance in our observations are equally useful.

8. 8 8 Teacher Data Initiative draws on 10 years of city-wide data (test scores, student, teacher, and school characteristics) to predict individual student gains. The predicted gains are compared to the actual gains for each student to determine the teacher ’ s contributions or the “ Value-Added. ” The teacher ’ s contribution for each student is averaged, and then compared to other 4-8 ELA and Math teachers by rank ordering top 20%, middle 60%, and bottom 20%. How Teacher Data Works Predicted Score Mathematically isolates factors beyond teacher control e.g. prior year test scores, ELL status, class size, etc Teacher Contribution Factors within teacher control e.g. quality of instruction & high expectations Actual Test Score Student scores on ELA & Math tests

9. 9 9 How Teacher Data Works: The Model Value-Added for one student Proficiency rating 3 rd Grade 4 th Grade 3 - - 2- Predicted Gain Actual Value Added Baseline (Previous Year’s) Score Teacher A Teacher B Teacher E Teacher D Teacher C Least Gain Most Gain The “Value- Added” is the difference between the predicted and actual scores Value-Added is averaged for all students in a class The Value-Added is measured in proficiency ratings Teacher Data Reports order teachers from least to most gain to determine a percentile rank

10. 10 The Model Mathematically Factors In Measurable Characteristics To Calculate Predicted Gain Student characteristicsClassroom characteristicsSchool characteristics Prior year reading Prior year math Free or reduced price lunch Special education status English Language Learner status Number of suspensions and absences (prior-year) Student retained in grade Attended summer school New to school Race Gender Prior year teacher Average prior year reading and math Percent free or reduced price lunch Percent special education status Percent English Language Learner status Average number of suspensions and absences (prior) Percent of students retained in grade Percent attended summer school Class size Percent by race Percent by gender Average classroom characteristics Average class size Total tested by grade/subject Year starting and ending school Teacher Characteristics (used when comparing teachers to peer teachers) Years of experience Years teaching in the same grade and subject

11. 11 Interpreting Teacher Data Results: Is this Result Good? Teacher Data Reports put a teacher ’ s results in context of other teachers: 1. Citywide All teachers in same grade/subject across the city 2. Peer teachers Also same grade/subject from across the city Adjusted for teacher experience level overall and in grade/subject ONLY 20% of classrooms with similar student/classroom and school characteristics »Using classroom predicted gain, each classroom is assigned to a quintile (20%) for peer comparisons

12. 12 Interpreting Teacher Data Results: What Does the Range Indicate? The most likely result for any teacher is the result presented on each line of the report and highlighted on graphs with an arrow. However, there is some uncertainty with any statistical measurement. Teacher Data reports also provide a range within which we can be 95% confidant of the result. >Ranges are larger for smaller sample sizes or fewer years of data 0%25%50%75% 2007-08 55%85%  Last 3 years 58%78%  100% My percentile (0%-100%) 68% Range* 70% My percentile

13. 13 Analyze Sample Reports to Look for Trends and Consider Key Questions Key Questions >What is being taught? >How is it taught? >Are the students learning? >How are teachers learning? >How are resources invested? Potential Trends >Clumps of teachers scoring low with a particular subgroup >Individual teachers consistently low/high across many groups >Sizeable difference between math and ELA >Similar scores among all teachers on a team or in a grade

14. 14 Example: Think Through a Specific Trend School summary report reveals much higher scores on math than ELA What is being taught? Is our math curriculum stronger than ELA? Consult with schools using similar curriculum How are resources invested? Are more push-in resources allocated for math? Teacher report reveals high scores on everything except for ELL students What is being taught? Might ELL students require additional instruction? Analyze test items for trends in ELL responses How is it taught? Is teacher differentiating instruction? Analyze quantity and quality of math PD Request peer observations Consult with others who have high ELL score Does this teacher receive adequate ELL Support? Pair this teacher with ELL coach

15. 15 Teacher Data: Uses and Cautions Potential Uses >Look for strengths, areas for development, surprises and wonderings >Emphasize instructional improvement >Triangulate with other insight  Consider factors you know about the teachers or the classrooms that may not be measurable  Help teachers connect these results with insights from their periodic assessments, student work, and item analysis >Consider professional development approaches for individual teachers or groups >Consider implications for student class assignment >Consider implications for curriculum or instructional programs >Consider implications for staffing needs >Prioritizing principal observation and coaching >Inform principal and teacher goal setting Cautions >Information is not to be used for teacher evaluation >Avoid replacing principal judgment and other forms of information >Not all negative value-added results are bad and all positive results are good  Use the performance ranges to see how strong a positive result is or how weak a negative result is. >Remember to consider context that is not easily measured and not in the model for example:  Push-in/pull-out teachers  AIS services  Life events for teachers, students  School context >Gain the teacher ’ s permission before sharing the report with other teachers >Consider individual teacher information confidential and thus not sharable with parents

16. 16 Sharing Reports With Teachers: Tips Principals are expected to share reports with teachers >First, take time to get comfortable with the key concepts and the information on the reports. Consider holding a group session to explain key concepts and review a sample report >Use materials provided or adapt Before each individual meeting to share a report with a teacher, consider these questions: >What do you know about the teacher? >What outcome do you want from the meeting? >What are the key points you want discussed? >Do you want to use the teacher reflection tools provided? >Should you give the teacher time before the meeting to review the report on their own? >What is the best setting, time, and other logistics for the meeting? During individual meetings: >Ask teacher what strengths, areas of improvement, surprises, wonderings s/he sees >Validate where you agree and continue to question where there is dissonance >Triangulate with other insight from periodic assessments, observations, student work etc >Discuss how this fits in or modifies existing development goals for teacher >Be clear what your expectations are and how you will help support the teacher

17. 17 Teacher Conversations: Practice

18. 18 Reviewing Own Schools ’ Reports

19. 19 Next Steps to Consider How will you use Teacher Data Reports to improve instruction? How will you involve others within the school? How will you introduce Teacher Data to your staff? How will you share individual reports with teachers? What additional supports might you need?

20. 20 Support Resources Web Resources: >Principal Portal: http://intranet.nycboe.net/DOEPortal/Principals/default.htmhttp://intranet.nycboe.net/DOEPortal/Principals/default.htm >Teacher Page: http://schools.nyc.gov/Teachers/default.htmhttp://schools.nyc.gov/Teachers/default.htm Help Desk: >TeacherDataReports@schools.nyc.govTeacherDataReports@schools.nyc.gov >(212) 374-6646 Leadership Academy coaches, CSA instructors and key UFT Teacher Center staff are knowledgeable Talent Office Support: >Sandra Tacina: stacina@schools.nyc.govstacina@schools.nyc.gov >Amy McIntosh: amcintosh2@schools.nyc.govamcintosh2@schools.nyc.gov