1. www.engageNY.org 1 New York State Education Department Interpreting and Using Your New York State-Provided Growth Scores August 2012

2. www.engageNY.org 2 2 By the End of This Presentation….  You should be able to: –Explain what information is included in a teacher growth report –Answer some common questions about the information –Describe the differences between a teacher-level, school-level, and district-level report –Understand how districts, principals and teachers can use the information on the reports as one source of data that can help improve instruction.

3. www.engageNY.org 3 3 Evaluating Educator Effectiveness 2011-12 Student growth on state assessments (state- provided) Student learning objectives Growth 20% Student growth or achievement Options selected through collective bargaining Locally Selected Measures 20% Rubrics Sources of evidence: observations, visits, surveys, etc. Other Measures 60% 4-8 grade ELA and Math Teachers & their Principals

4. www.engageNY.org 4 4 Key Points about NYS Growth Measures –We are measuring student growth and not achievement  Allow teachers to achieve high ratings regardless of incoming levels of achievement of their students –We are measuring growth compared to similar students  Similar students: Up to three years of the same prior achievement, three student-level characteristics (economic disadvantage, SWD, and ELL status) Every educator has a fair chance to demonstrate effectiveness on these measures regardless of the composition of his/her class or school.

5. www.engageNY.org 5 5 Review of Terms  SGP (student growth percentile): –the result of a statistical model that calculates each student’s change in achievement between two or more points in time on a State assessment or other comparable measure and compares each student’s performance to that of similarly achieving students  Unadjusted and adjusted MGP (mean growth percentile): –the average of the student growth percentiles attributed to a given educator –For evaluation purposes, the overall adjusted MGP is used. This is the MGP that includes all a teacher or principal’s students and takes into account student demographics (ELL, SWD, and economic disadvantage status).

6. www.engageNY.org 6 6 MGPs and Statistical Confidence 87 Confidence Range Upper Limit Lower Limit MGP NYSED will provide a 95% confidence range, meaning we can be 95% confident that an educator’s “true” MGP lies within that range. Upper and lower limits of MGPs will also be provided. An educator’s confidence range depends on a number of factors, including the number of student scores included in his or her MGP and the variability of student performance in the classroom.

7. www.engageNY.org 7 7 Growth Ratings and Score Ranges 2011-12 Growth RatingDescriptionGrowth Score Range (2011–12) Highly Effective Well above state average for similar students 18–20 EffectiveResults meet state average for similar students 9–17 DevelopingBelow state average for similar students 3–8 IneffectiveWell below state average for similar students 0–2 The growth scores and ratings are based on an educator’s combined MGP. For detailed information see the webinar posted here: http://engageny.org/resource/using-growth-measures-for-educator- evaluation-in-2011-2012/ http://engageny.org/resource/using-growth-measures-for-educator- evaluation-in-2011-2012/

8. www.engageNY.org 8 8 First let’s look at a growth report about a teacher… Jane Eyre

9. www.engageNY.org 9 9 Teacher-level Report District X School #1 Jane Eyre Teacher 1D’s Growth Score and Growth Rating are listed here Teacher 1D has a higher adjusted MGP in Math than ELA Teacher 1D does not have any growth data reported for any of the subgroups because 16 student scores are required to report any data Jane’s MGP = 47 (this is what is used to determine the growth score and growth rating) Jane’s Upper Limit = 55 and Lower Limit = 39

10. www.engageNY.org 10 www.engageNY.org 10 Common Questions about Teacher-level Reports Will an educator’s adjusted MGP always be higher than the unadjusted since the adjusted MGP takes into account not just prior student achievement but also economic disadvantage, ELL, and SWD status? Team Order of finishOverall PercentileDivisionDivision FinishDivision Percentile L199A1 M266B199 N350B233 O4 A2 P517B31 Q61A31 Division B Teams Not necessarily. Division B Percentile Let’s take the following simplified example of a relay race.

11. www.engageNY.org 11 www.engageNY.org 11 Common Questions about Teacher-level Reports What is the difference between an educator’s MGP and the percent of an educator’s students above the state median? Student SGP Teacher MGP (Average of all SGPs) % Students above State Median of 50 20 30 55 60 Let’s take the following simplified example:

12. www.engageNY.org 12 www.engageNY.org 12 Common Questions about Teacher-level Reports What is the difference between an educator’s MGP and the percent of an educator’s students above the state median? Student SGP Teacher MGP (Average of all SGPs) % Students above State Median of 50 20 30 55 60 (20+30+55+55+60) divided by 5 = 44 Let’s take the following simplified example:

13. www.engageNY.org 13 www.engageNY.org 13 Common Questions about Teacher-level Reports What is the difference between an educator’s MGP and the percent of an educator’s students above the state median? Student SGP Teacher MGP (Average of all SGPs) % Students above State Median of 50 20Not > 50 30Not > 50 55Yes > 50 55Yes > 50 60Yes > 50 (20+30+55+55+60) divided by 5 = 44 3 out of 5 students above 50 = 60% Let’s take the following simplified example: Note: in NYSED reports, no MGP is calculated for less than 16 student scores.

14. www.engageNY.org 14 www.engageNY.org 14 School-level Report District X School #1 An adjusted MGP and associated confidence range will be reported for each subject and grade level within the school. 49 % of students at School #1 scored above the State median. The Growth Score and Growth Rating for the Principal of School #1 are listed here School #1 has scores broken out by subject for grades 4-6. 36% of the student scores are from economically disadvantaged students, and no scores from English language learners. Summary of Revised APPR Provisions Memo: http://engageny.org/wp-content/uploads/2012/03/nys-evaluation-plans-guidance-memo.pdf

15. www.engageNY.org 15 www.engageNY.org 15 School-level Report—Detailed View District X School #1 Teacher 1E Teacher 1D Teacher 1C Teacher 1B Teacher 1A Teacher 1F Teacher 1I Teacher 1J Teacher 1K Teacher 1L Teacher 1G Teacher 1H School #1 has 12 teachers who teach grades 4-8 ELA and Math Teacher 1B has the most student scores linked to him (43 scores) 43 student scores could not be linked to any of the teachers Each teacher receives an adjusted MGP and associated confidence range that are used to determine the growth rating and growth score Teachers 1E and 1G did not receive any growth data because they are linked to less than 16 student scores

16. www.engageNY.org 16 www.engageNY.org 16 Frequently Asked Questions Why do some teachers have a combined MGP but no subject-level MGPs?  Minimum of 16 student scores  Student must be continuously enrolled in a course that leads to an assessment for 195 calendar days for ELA or 203 calendar days for Math.

17. www.engageNY.org 17 www.engageNY.org 17 Frequently Asked Questions Why are some students unassigned? How did the State define “unassigned”? Students are considered unassigned if the district did not provide a valid teacher-of-record for that student, or if the student-teacher linkage relationship did not meet the continuous enrollment guidelines set forth in the APPR guidance http://engageny.org/resource/guidance-on-new-yorks-annual- professional-performance-review-law-and-regulations/. http://engageny.org/resource/guidance-on-new-yorks-annual- professional-performance-review-law-and-regulations/ To meet the continuous enrollment guidelines, a student needed to be linked to a teacher for 195 calendar days for ELA, or 203 calendar days for Math in 2011-12.

18. www.engageNY.org 18 www.engageNY.org 18 District-level View—Page 1 NY State Summary NYS Summary Data— Included on ALL District reports Number of student scores included in calculation of State MGP NY Statewide Adjusted MGP = 52 State Median = 50 District X Statewide about 50% of ELL, SWD, and economically disadvantaged students scored above the State median.

19. www.engageNY.org 19 www.engageNY.org 19 District X Summary Data District-level View—Page 1-2 District Summary District X Summary Data—continued on next page of report Number of student scores included in calculation of district-wide MGP District-wide Adjusted MGP District X

20. www.engageNY.org 20 www.engageNY.org 20 District-level View—Page 3 List of Schools District X has two schools that have grades 4-8 ELA and Math scores School #1 School #2 District X Principal of School #1 Growth Score = 14 Growth Rating = Effective Principal of School #2 Growth Score = 6 Growth Rating = Developing

21. www.engageNY.org 21 www.engageNY.org 21 Using Growth Score results  Beyond evaluation, growth score information can provide additional information to help teachers, principals and districts with instructional improvement. –Of course, these measures are only one of multiple sources of evidence to use for this purpose –The best insight comes from considering the results in the context of other information about a teacher, group of teachers, principal or group of schools.

22. www.engageNY.org 22 www.engageNY.org 22 Districts may want to: Analyze district-level information using these reflective questions: How much did our students grow, on average, compared to similar students? Is this higher, lower, or about what we would have expected? Why? How do our MGPs for each reported subgroup (ELL, SWD, economically disadvantaged students, high- and low-achieving students) compare to each other and to our overall MGP? Are there any patterns? Are the MGPs higher, lower, or about what we would have expected? Why? How do the MGPs compare by subject and across grade levels? Why might they be similar or different? What should we do to understand any surprises using other information and evidence? Do we have the right plans in place to aid in professional growth and learning for our educators?

23. www.engageNY.org 23 www.engageNY.org 23 Districts may want to: Convene principals to reflect upon their school growth results in context of other information about student learning and teacher effectiveness in their schools: –Use BOCES trainers and/or SED online resources to ensure basic understanding of the measures and what information is found on reports –Engage principals individually or in a group to reflect on questions about their school information in the context of other evidence of teacher effectiveness:  How much did the students of my teachers grow, on average, compared to similar students and how does this differ across teachers? Are there differences across grades or subjects?  How do my teachers’ MGPs differ across each reported subgroup? Do I see any patterns?

24. www.engageNY.org 24 www.engageNY.org 24 Districts may want to: Plan for communicating with teachers about their results:  How will teachers get general information about the growth measures? –District or school-level training or self-directed use of SED resources  How and when will teachers receive their individual reports? –Remember SED will have online access for individual educators later in the fall –Use BOCES trainers and/or SED online resources to ensure basic understanding of the measures and what information is found on reports

25. www.engageNY.org 25 www.engageNY.org 25 Principals may want to:  Consider the reflective questions in their school-level reports:  See the Principal’s Guide to Interpreting Growth Scores: http://engageny.org/wp- content/uploads/2012/06/Principals_Guide_to_Interpreting_Your _Growth_Score.pdf http://engageny.org/wp- content/uploads/2012/06/Principals_Guide_to_Interpreting_Your _Growth_Score.pdf  See the Sample Principal Report—Annotated: http://engageny.org/wp- content/uploads/2012/06/Principal_Sample_Growth_Report.pdf http://engageny.org/wp- content/uploads/2012/06/Principal_Sample_Growth_Report.pdf  Plan how teachers will get the information they need to understand their own growth reports

26. www.engageNY.org 26 www.engageNY.org 26 Teachers may want to:  Review materials from SED about growth measures  View the “Growth Model for Educator Evaluation 2011-12” Webinar: http://engageny.org/resource/growth-model-for-educator-evaluation-in-2011-2012/ http://engageny.org/resource/growth-model-for-educator-evaluation-in-2011-2012/  View the “Using Growth Measures for Educator Evaluation in 2011-12” Webinar: http://engageny.org/resource/using-growth-measures-for-educator-evaluation-in-2011- 2012/ http://engageny.org/resource/using-growth-measures-for-educator-evaluation-in-2011- 2012/  See the Teacher’s Guide to Interpreting Growth Scores: http://engageny.org/wp- content/uploads/2012/06/Teachers_Guide_to_Interpreting_Your_Growth_Score.pdf http://engageny.org/wp- content/uploads/2012/06/Teachers_Guide_to_Interpreting_Your_Growth_Score.pdf  See the Sample Teacher Report—Annotated: http://engageny.org/wp- content/uploads/2012/06/Teacher_Sample_Growth_Report.pdf http://engageny.org/wp- content/uploads/2012/06/Teacher_Sample_Growth_Report.pdf  Consider the following reflective questions:  How much did my students grow, on average, compared to similar students? Is this higher, lower, or about what I would have expected? Why?  How does this information about student growth align with information about my instructional practice received through observations or other measures? Why might this be?

27. www.engageNY.org 27 www.engageNY.org 27 For More Information… Please review our posted Guides for Interpreting Your Growth Scores: http://engageny.org/resource/resources-about-state-growth- measures/ And the guidance on NYS’s APPR Law and Regulations: http://engageny.org/resource/guidance-on-new-yorks-annual- professional-performance-review-law-and-regulations/

28. www.engageNY.org 28 www.engageNY.org 28