How Adequate Yearly Progress (AYP) Is Determined Using Data The New York State Education Department August 21, 2012

2 Accountability Measures At the elementary/middle level, New York State reports student proficiency in 1) English language arts (ELA), 2) mathematics, and 3) science. At the secondary level, New York State reports student proficiency in 1) ELA, 2) mathematics, and 3) graduation rate.

3 Accountability Groups For each accountability measure, New York State reports data on the following “accountability” groups: All Students American Indian or Alaska Native Students Black or African American Students Hispanic or Latino Students Asian or Native Hawaiian/Other Pacific Islander Students White Students Multiracial Students Students with Disabilities* Limited English Proficient (LEP) Students (also known as English Language Learners – ELLs)* Economically Disadvantaged Students* * Students are included in the Students with Disabilities, LEP Students, or Economically Disadvantaged Students group if they were reported to the Department in the Student Information Repository System (SIRS) as being a member of the group at any time during the reporting year.

4 Non-AYP Groups New York State also reports data on the following “non-AYP” groups (i.e., groups for which AYP is not determined but for whom data are used to make some accountability determinations): Not American Indian or Alaska Native Students Not Black or African American Students Not Hispanic or Latino Students Not Asian or Native Hawaiian/Other Pacific Islander Students Not White Students Not Multiracial Students General-Education Students English Proficient Students Not Economically Disadvantaged Students Male Students Female Students Migrant Students Not Migrant Students

5 Adequate Yearly Progress: Participation and Performance Schools and districts must meet pre- defined participation and performance criteria on New York State’s accountability measures to make Adequate Yearly Progress (AYP).

6 Grade Configurations for AYP Elementary/Middle Level: Participation and performance at the elementary/middle level are determined for students in grades 3 through 8 combined for ELA and math and grades 4 and 8 combined for science based on the school’s or district’s grade configuration. For example, a middle school that has only grades 6 through 8 will have the participation and performance calculated for their grades 6 through 8 students combined. Secondary Level: Participation for ELA and math at the secondary level is determined based on the school’s or district’s 2011–12 grade 12 enrollment. Performance for ELA and math at the secondary level is determined based on the school’s or district’s 2008 accountability cohort. Graduation rate at the secondary level is determined based on the school’s or district’s 2007 graduation-rate total cohort after 4 years and/or 2006 graduation-rate total cohort after 5 years. See “Secondary-Level Cohort Definitions” at for more information.

7 Participation Criterion

8 Participation Criterion Elementary/Middle Level F or an accountability group with 40 or more students enrolled during the test administration period to meet the participation criterion in English language arts (ELA) or mathematics, 95 percent of these students must have valid scores on an appropriate assessment. For an accountability group with 40 or more students enrolled during the test administration period to meet the participation criterion in science, 80 percent of these students must have valid scores on an appropriate assessment.

9 Elementary/Middle-Level Assessments That Can Be Used To Fulfill Participation Criterion AssessmentsEligible Students Grades 3–8 New York State Testing Program (NYSTP) Assessments in ELA and Mathematics All students (general education & students with disabilities) New York State Grade 4 Elementary-Level Science and Grade 8 Middle-Level Science Tests All students (general education & students with disabilities) Regents Living Environment, Physical Setting/Earth Science, Physical Setting/Chemistry, and Physical Setting/Physics Tests in Lieu of Grade 8 Middle-Level Science Test All students (general education & students with disabilities) New York State Alternate Assessments in ELA (Grades 3–8 Equivalent), Mathematics (Grades 3–8 Equivalent), and Science (Grades 4 and 8 Equivalent) Students with severe cognitive disabilities New York State English as a Second Language Achievement Tests (NYSESLAT) in Lieu of NYSTP in ELA (Grades 3–8) Students whose first language is NOT English and who have been in the United States (not including Puerto Rico) for less than one year

10 Participation Criterion Secondary Level For an accountability group with 40 or more 12 th graders to meet the participation criterion in English language arts (ELA) or mathematics, 95 percent of the 12 th graders must have valid scores on Regents examinations (or approved alternatives), Regents competency tests (or approved alternatives), or New York State Alternate Assessments.

11 Secondary-Level Assessments That Can Be Used To Fulfill Participation Criterion AssessmentsEligible Students Regents Examinations in Comprehensive English and Mathematics, and Approved Alternatives All students (general education & students with disabilities) Regents Competency Tests in Reading, Writing, and Mathematics, and Approved Alternatives Students with disabilities and students with a 504 plan that allows an RCT accommodation New York State Alternate Assessments in ELA and Mathematics (Secondary Level) Students with severe cognitive disabilities

12 “Safety Net” for Groups That Fail the Participation Criterion If the participation rate of an accountability group with 40 or more students falls below the required rate, the Department calculates a weighted average of the current year’s and the previous year’s participation rates. If the average participation rate equals or exceeds the required rate, the group fulfills the participation criterion. Sample calculation for group below 95 percent participation criterion: YearEnrollmentTestedRate Current605693% Previous757397% Weighted Average Calculation %

13 Medically Excused If a student in grades 3 through 8 is incapacitated by illness or injury during the entire test administration and make-up period for elementary/middle-level English language arts, mathematics, or science, the student is not counted in the numerator or the denominator when participation rates are calculated. To use this flexibility, the district must have on file documentation from a medical practitioner that the student was too incapacitated to be tested. This option is not applicable at the secondary level.

14 Performance Criterion: Performance Indices

15 Performance Index (PI) A Performance Index (PI) is a value from 0 to 200 that is assigned to an accountability group, indicating how that group performed on a required State test (or approved alternative) in English language arts, mathematics, or science. PIs are determined using the performance levels and equations on the following slides.

16 Performance Levels Elementary/Middle-Level ELA and Mathematics Student performance in elementary/middle-level ELA and mathematics is determined using a Performance Index (PI) calculation. This calculation uses six levels of student achievement: Level 1 On Track = Basic and On Track to Proficient Level 1 Not On Track = Basic and NOT On Track to Proficient Level 2 On Track = Basic Proficient and On Track to Proficient Level 2 Not On Track = Basic Proficient and NOT On Track to Proficient Level 3 = Proficient Level 4 = Advanced

17 Performance Index (PI) Formula Elementary/Middle-Level ELA and Mathematics PI = [(number of continuously enrolled tested students scoring at Level 1 On Track + Level 1 On Track + Level 2 On Track + Level 2 On Track + Level 2 NOT On Track + Level 3 + Level 3 + Level 4 + Level 4) ÷ number of continuously enrolled tested students]  100

18 Sample Elementary/Middle-Level ELA PI Calculation for a School with Grades 3, 4, and 5 Only Count of students performing at level: Student Grade Count on not on not track track track track TOTAL PI = [( ) ÷ 108]  100 = 165 Note: The methodology is the same regardless of how many grade levels (3-8) a school serves.

19 Performance Levels Elementary/Middle-Level Science and Secondary-Level ELA and Mathematics Student performance in elementary/middle-level science and secondary-level ELA and mathematics is also determined using a Performance Index (PI) calculation. This calculation uses four levels of student achievement: Level 1 = Basic Level 2 = Basic Proficient Level 3 = Proficient Level 4 = Advanced

20 Performance Index (PI) Formula Elementary/Middle-Level Science and Secondary-Level ELA and Mathematics Elementary/Middle Level Science: PI = [(number of continuously enrolled tested students scoring at Levels 2, 3, and 4 + the number scoring at Levels 3 and 4) ÷ number of continuously enrolled tested students]  100 Secondary Level ELA and Mathematics: PI = [(number of cohort members scoring at Levels 2, 3, and 4 + the number scoring at Levels 3 and 4) ÷ number of cohort members]  100

21 Sample Elementary/Middle-Level Science PI Calculation for a School with Grades 3, 4, and 5 Only Number of Test Number Students at Levels Grade of Students TOTAL PI = [( ) ÷ 108]  100 = 140 Note: The methodology is the same regardless of how many grade levels (3-8) a school serves.

22 Assessments That Can Be Used To Fulfill the Performance Criterion at the Elementary/Middle Level AssessmentEligible StudentsScores Grades 3–8 New York State Testing Program (NYSTP) Assessments in ELA and Mathematics All students (general education & students with disabilities) Level 4 Level 3 Level 2 On Track Level 2 Not On Track Level 1 On Track Level 1 Not On Track New York State Grade 4 Elementary-Level Science and Grade 8 Middle-Level Science Tests All students (general education & students with disabilities) Level 4 Level 3 Level 2 Level 1 Regents Living Environment, Physical Setting/Earth Science, Physical Setting/Chemistry, and Physical Setting/Physics Tests in Lieu of Grade 8 Middle-Level Science Test All students (general education & students with disabilities) Level 4 (85–100) Level 3 (65–84) Level 2 (55–64) Level 1 (0–54) New York State Alternate Assessments (NYSAA) in ELA (Grades 3–8 Equivalent), Mathematics (Grades 3–8 Equivalent), and Science (Grades 4 and 8 Equivalent) Students with severe cognitive disabilities Level 4 Level 3 Level 2 Level 1

23 Order of Precedence for Using Elementary/Middle-Level Assessment Scores for Performance Index Determinations Grades 3–8 English Language Arts and Mathematics: If a student has more than one applicable ELA or math score, the order of precedence for selecting a performance level for use in the PI calculation is: 1) NYSTP, and 2) NYSAA. Grade 8 Science: If an eighth-grader has more than one applicable science score, the order of precedence for selecting a performance level for use in the PI calculation is: 1) New York State Grade 8 Intermediate-Level Science Test for the current year, 2) NYSAA Grade 8 Equivalent in Science, 3) Regents science examination, and 4) New York State Grade 8 Intermediate-Level Science Test taken by the student in 7th grade in the previous year. NYSESLAT: At the elementary/middle level, if a district chooses to give the NYSTP ELA assessment to a LEP student who is eligible to take the NYSESLAT in lieu of the NYSTP, NYSED will count the student’s NYSTP ELA scores when computing the school’s and district’s accountability PI.

24 How Assessments Are Used To Fulfill the Performance Criterion in Secondary-Level ELA The highest score a student receives on an assessment, regardless of when it is taken, is counted in the PI calculations. Students who do not take an assessment are counted as performing at Accountability (Acc) Level 1. AssessmentsEligible Students Score/ Accountability Performance Level Regents Examination in Comprehensive English All students (general education & students with disabilities) 90–100 = Acc Level 4 75–89 = Acc Level 3 65–74 = Acc Level 2 0–64 = Acc Level 1 Approved Alternatives to Regents Examination in Comprehensive English All students (general education & students with disabilities) Pass = Acc Level 3 Fail = Acc Level 1 Regents Competency Tests in Reading and Writing (and Approved Alternatives) Students with disabilities Pass = Acc Level 1 Fail = Acc Level 1 New York State Alternate Assessment in ELA (Secondary Level) Students with severe cognitive disabilities Level 4 = Acc Level 4 Level 3 = Acc Level 3 Level 2 = Acc Level 2 Level 1 = Acc Level 1

25 How Assessments Are Used To Fulfill the Performance Criterion in Secondary-Level Math The highest score a student receives on an assessment, regardless of when it is taken, is counted in the PI calculations. Students who do not take an assessment are counted as performing at Accountability (Acc) Level 1. AssessmentsEligible Students Score/ Accountability Performance Level Regents Examinations in Mathematics All students (general education & students with disabilities) 90–100 = Acc Level 4 80–89 = Acc Level 3 65–79 = Acc Level 2 0–64 = Acc Level 1 Approved Alternatives to Regents Examinations in Mathematics All students (general education & students with disabilities) Pass = Acc Level 3 Fail = Acc Level 1 Regents Competency Test in Mathematics (and Approved Alternatives) Students with disabilities Pass = Acc Level 1 Fail = Acc Level 1 New York State Alternate Assessment in Mathematics (Secondary Level) Students with severe cognitive disabilities Level 4 = Acc Level 4 Level 3 = Acc Level 3 Level 2 = Acc Level 2 Level 1 = Acc Level 1

26 Order of Precedence for Using Secondary-Level Assessment Scores for Accountability Determinations Regents examination where accountability performance is Level 3 or 4 Passing score on an alternative to a Regents examination Regents examination where accountability performance is Level 2 Accountability performance Level 1 for any assessment (Regents examination, alternative to a Regents examination, pass or fail on an Regents Competency Test (RCT) or an alternative to the RCT, or NYSAA) New York State Alternate Assessment (NYSAA) – Any Level* *NYSAA counts toward secondary-level accountability ONLY if it was the only secondary-level assessment taken. If a student takes more than one assessment in a subject, regardless of when the assessments were taken, the assessment used to fulfill the graduation requirement will be chosen according to the precedence list below, with the top assessment on the list taking precedence over the next one, etc.

27 Performance Criterion: Effective AMOs, State Standards, Safe Harbor Targets, and Progress Targets

28 Meeting the Performance Criterion Using Effective AMOs, State Standards, Safe Harbor, and Progress Targets To meet the performance criterion in ELA and math, the Performance Index of a group with 30 or more students must be equal to or greater than the Effective Annual Measurable Objective (Effective AMO) or the group must make Safe Harbor. To meet the performance criterion in science, the Performance Index of a group with 30 or more students must be equal to or greater than the Effective AMO or the group must meet its Progress Target. To meet the performance criterion in graduation rate, the graduation rate of a group with 30 or more students must be equal to or greater than the State Standard or the group must meet its Progress Target.

29 Annual Measurable Objectives (AMOs) and State Standards The Annual Measurable Objective (AMO) is the PI value that signifies that an accountability group is making satisfactory progress toward pre- determined goals. These values increase from year to year and are different for ELA, mathematics, and science. AMOs are available at State Standard is the PI value that signifies minimally satisfactory performance in graduation rate. The graduation-rate state standard is currently 80%.

30 Confidence Intervals Were Used to Determine Effective AMOs Annual Measurable Objective A confidence interval is a range of points around an AMO for an accountability group of a given size that is considered to be not significantly different than the AMO. The four small squares below represent four schools with the same PI but with different numbers of tested students. The vertical lines represent the confidence interval for each school based on the number of students tested. The more students tested, the smaller the confidence interval.

31 An Effective AMO (EAMO) is the lowest PI that an accountability group of a given size can achieve in a subject for the group’s PI not to be considered significantly different from the AMO for that subject. If an accountability group's PI equals or exceeds the EAMO and the group meets its participation requirement, the group is considered to have made AYP. EAMOs are available at Effective AMOs

32 Safe Harbor for ELA and Math Safe Harbor is an alternate means to demonstrate AYP for accountability groups whose PI is less than their Effective AMO. The Safe Harbor Target calculation for ELA and math is: Safe Harbor Target = {Previous Year’s PI} + [(200 – {Previous Year’s PI})  0.10] For a group to make safe harbor in English or math, the Performance Index must be greater than or equal to the Safe Harbor Target.

33 Safe Harbor Target Calculations for Elementary/Middle-Level Groups With Fewer than 30 Students in the Previous Year For elementary/middle-level ELA and mathematics, if in the current year a district or school has an accountability group with 30 or more students but did not have 30 or more students in the previous year, student scores for the previous two years are combined to calculate a Safe Harbor Target for the current year. If in the combined years, there are still not 30 or more students with valid test scores in the group, the group is assigned a Safe Harbor Target of 20.

34 Safe Harbor Target Calculations for Cohorts With Fewer than 30 Members For secondary-level ELA and mathematics, if in the current year a district or school has an accountability group with 30 or more cohort members but did not have 30 or more cohort members in the previous year, student scores for the previous two cohorts are combined to calculate a Safe Harbor Target for the current year. If in the combined years, there are still not 30 or more cohort members in the group, the group is assigned a Safe Harbor Target of 20.

35 Science and Graduation-Rate Progress Targets Progress Targets are determined in science for groups whose PI is less than their EAMO in science and in graduation rate for groups whose graduation rate is less than the State Standard. The Progress Target in science is determined by adding one point to the previous year’s PI. The Progress Target for the 2007 four-year cohort is a 10% gap reduction (one percentage point minimum) over the previous year’s graduation rate Four-Year Cohort Progress Target = ((80 – 2006 Graduation-Rate Total Cohort Graduation Rate)  0.10) Graduation-Rate Total Cohort Graduation Rate The Progress Target for the 2006 five-year cohort is a 20% gap reduction (one percentage point minimum) over the previous year’s graduation rate Five-Year Cohort Progress Target = ((80 – 2005 Graduation-Rate Total Cohort Graduation Rate)  0.20) Graduation-Rate Total Cohort Graduation Rate

36 Progress Target Calculations for Elementary/Middle-Level Science Groups with Fewer than 30 Students in the Previous Year For elementary/middle-level science, if in the current year a district or school has an accountability group with 30 or more students but did not have 30 or more students in the previous year, student scores for the previous two years are combined to calculate a Progress Target for the current year. If in the combined years, there are still not 30 or more students with valid test scores in the group, the group is assigned a Progress Target of 1.

37 Progress Target Calculations for Cohorts with Fewer than 30 Members For secondary-level graduation rate, if in the current year a district or school has an accountability group with 30 or more cohort members but did not have 30 or more cohort members in the previous year, student scores for the previous two cohorts are combined to calculate a Progress Target for the current year. If in the combined years, there are still not 30 or more cohort members in the group, the group is assigned a Progress Target of 16.

38 Performance for Schools/Districts with Fewer Than 30 Students If a school or district has more than 0 but fewer than 30 students in the All Students group for performance for elementary/middle- and secondary-level ELA and mathematics and elementary/middle- level science, New York State combines the current year’s and the previous year’s data for all student groups with more than 0 students in the current year to determine new Performance Indices. If the new performance indices equal or exceed the EAMO, Safe Harbor Target, or Progress Target for the two-year combined group size, the group fulfills the performance criterion. Sample calculation: YearEnrollmentLevels PI Current = 41N/A Previous = 39N/A Combined  (( )  55) 145

39 Graduation Rate for Schools/Districts with Fewer Than 30 Graduation-Rate Total Cohort Members If a school or district has more than 0 but fewer than 30 graduation- rate total cohort members in the current year, the Department combines the data for the current year’s and the previous year’s graduation-rate total cohorts for all student groups with more than 0 students in the current year’s graduation-rate total cohort to determine new graduation rates. Sample calculation: YearCohort Enrollment Cohort Members with Local or Regents Diploma Graduation Rate Current2018N/A Previous2521N/A Combined453987%

40 Former Limited English Proficient (LEP) Students If the count of LEP students for performance is equal to or greater than 30, former LEP students are also included in the performance calculations. A former LEP student is one who was previously identified as LEP but reached proficiency in English by achieving a Level 3 or 4 on both the Listening and Speaking and the Reading and Writing portions of the New York State English as a Second Language Achievement Test (NYSESLAT) during one of the previous two school years.

41 Former Students with Disabilities If the count of students with disabilities for performance is equal to or greater than 30, former students with disabilities are also included in the performance calculations. A former student with disabilities is one who is not identified in the current school year as a student with disabilities but was previously identified as a student with disabilities in at least one of the previous two school years.

42 Graduation Rate Calculation The graduation rate for a group is determined by dividing the number of graduation-rate total cohort members who graduated with a local or Regents diploma by the number of graduation-rate total cohort members, and multiplying the result by 100. For example: Graduation-rate total cohort members = 178 Graduation-rate total cohort members with local or Regents diplomas = 146 Graduation rate = (146  178)  100 = or 82%

43 Making Adequate Yearly Progress (AYP)

44 Determining Adequate Yearly Progress (AYP) To determine AYP for an accountable group, calculations are made in the following order: 1.The participation rate for the group based on the current year’s data is determined. 2.If the participation rate for the group based on the current year’s data is below the required rate, the current and previous year’s data are combined to determine a participation rate. 3.If the participation rate criterion is not met, the group fails to make AYP, regardless of the performance. 4.If the participation rate criterion is met, the PI is compared to the Effective AMO for ELA, math, and science. For graduation rate, the graduation rate is compared with the State Standard. 5.If the PI is equal to or greater than the Effective AMO in ELA, math, and science, the group makes AYP. If the graduation rate is equal to or greater than the State Standard the group makes AYP in graduation rate. 6.If the criteria in #5 are not met, the PI is compared to the Safe Harbor Target (ELA and math) or Progress Target (science), or the graduation rate is compared to the Progress Target (graduation rate).

45 Determining Adequate Yearly Progress (AYP) (continued) 7.For ELA and math, if the PI is greater than or equal to the Safe Harbor Target, the group makes AYP. If the PI is less than Safe Harbor Target, the group does not make AYP. 8.For science, if the PI is greater than or equal to the Progress Target, the group makes AYP. If the PI is less than the Progress Target, the group does not make AYP. 9.For graduation rate, if the graduation rate is greater than or equal to the Progress Target, the group makes AYP. If the graduation rate is less than the Progress Target, the group does make AYP.

46 Accountability for Limited English Proficient (LEP) Students  All LEP students in grades K–12 must take the New York State English as a Second Language Achievement Test (NYSESLAT) annually.  LEP students in grades 3 through 8 enrolled in U.S. schools (not including Puerto Rico) for less than one year may take the NYSESLAT in lieu of the NYSTP in ELA to fulfill the testing requirement for accountability. The one-year window does not have to be 12 consecutive months. In addition, students enrolled anytime during a month, including July and August, are considered enrolled for that month.  Eligible students may be exempt from taking the NYSTP in ELA for the first year in which they are enrolled during the NYSTP ELA test administration period. Such students may not be exempt in subsequent years, even if they have been enrolled in a United States school for less than 12 months.  LEP students in grades 3 through 8 who did not take the ELA assessment, were enrolled in U.S. schools (not including Puerto Rico) for less than one year, have valid scores on both the NYSESLAT Reading/Writing and Speaking/Listening components, and were reported in the Student Information Repository System (SIRS) with a Program Service code of 0242 (NYSESLAT-eligible) will meet the ELA participation requirement.  NYSESLAT performance levels will not be used in calculating the Performance Index. LEP students meeting the criteria to use the NYSESLAT in lieu of the ELA will not be included in the Performance Index calculation.  Districts receiving Title III funding must identify each participating student in the SIRS.

47 Accountability for Students with Disabilities

48 New York State Alternate Assessment (NYSAA)  NYSAA performance levels are counted the same as general assessment (NYSTP, Regents, etc.) levels when determining PIs for English, mathematics, and science.  No Child Left Behind (NCLB) regulations allow a maximum of one percent of scores used in calculating the PI for each accountability measure for a district to be based on proficient and advanced proficient scores on the NYSAA, unless the district has a waiver to exceed the one percent.  To meet this requirement, districts that have more than one percent of their continuously enrolled tested students performing at Levels 3 and 4 on the NYSAA must have some of these students counted at Level 2 when determining PIs. If these students attend schools within the district, this reduction may impact the schools as well. Districts must report the performance levels as received by the students. NYSED will make the adjustments for PI calculations only.

49 Testing Ungraded Students with Disabilities  The CSE must determine that a student meets the criteria specified by the New York State Education Department.  Students must be administered the correct test for their age, as specified in the “Reporting Student Data in SIRS” at

50  Accountability: the Office of Accountability at accountinfo.mail.nysed.gov  The New York State Accountability Report and data collection and reporting for New York State: the Information and Reporting Services Office at  New York State assessments: the Office of Assessment Policy, Development and Administration at  Elementary and Secondary Education Act (ESEA): the United States Department of Education at Whom to Contact for Further Information