Understanding the Accountability Report & Report Card Timothy Johnson David Scalzo Erie 1 BOCES Data Analysis Team. All rights reserved.

Before We Start You will need to gather some resources from: NYSED site SEDDAS Portal Level 2 Data Warehouse Erie 1 BOCES Data Analysis Team. All rights reserved. Before We Start…

AOR and CIR for your District and Schools Accountability Report and Report Card for your District and Schools SEDDAS – IRSP Portal – ESEA Revised Performance Indices Report – released July Effective Annual Measurable Objectives on Results.pdf Ira Swartz Memo from January Revised.pdf Accountability Determinations FAQ April 2013 document 12AccountabilityDeterminationsFAQ.pdf Erie 1 BOCES Data Analysis Team. All rights reserved. Resources

Level 2 Verification Reports - SIRs HS Accountability Reports SIRs Elementary/Middle Level Accountability Reports SIRs 307 – Annual Dropout and Non-Completer Report SIRs 201 – Total Cohort Summary Report SIRs 202 – Total Cohort Assessment Summary Report Erie 1 BOCES Data Analysis Team. All rights reserved. Resources

Changes in NYS School Year End Reports Pre and Forward Erie 1 BOCES Data Analysis Team. All rights reserved.

Enrollment Average Class Size Demographic Factors Attendance and Suspensions Teacher Qualifications Teacher Turnover Rate Staff Counts Erie 1 BOCES Data Analysis Team. All rights reserved. AOR – Accountability & Overview Report

District Summary Elementary/Middle Level ELA, Math, Science participation and performance Secondary Level ELA, Math, Science participation and performance Graduation Rate Erie 1 BOCES Data Analysis Team. All rights reserved. AOR – Accountability & Overview Report

Breakdown of school accountability in a district Erie 1 BOCES Data Analysis Team. All rights reserved. Shows grade by grade breakdown of all subject grade level performance on state assessments AOR – Accountability & Overview Report

CIR – Comprehensive Information Report Regents Exams Regents Competency Tests NYSESLAT NAEP Cohort Performance after four years NYSAA High School Information High School Completers Non-completers Post-Secondary Plans. Erie 1 BOCES Data Analysis Team. All rights reserved.

Adequate Yearly Progress and Performance Indices Elementary/Middle Level ELA AYP-Participation-Performance Math AYP-Participation-Performance Science AYP-Participation-Performance Secondary Level ELA AYP-Participation-Performance Math AYP-Participation-Performance Un-weighted Combined ELA and Math Performance Indicators Graduation Rate AYP 4 year Graduation-Rate Total Cohort 5 year Graduation-Rate Total Cohort Erie 1 BOCES Data Analysis Team. All rights reserved. The New Accountability Report …

Enrollment Average Class Size Demographic Factors Attendance and Suspensions Teacher Qualifications Teacher Turnover Rate Staff Counts Erie 1 BOCES Data Analysis Team. All rights reserved. The New Report Card Report …

Contains brief description of NYSTP Assessments, NYSAA Assessment, NYSESLAT Tests, Secondary Level Cohorts Grade by grade breakdown of all subject grade level performance on state assessments for 3-8 Assessments Statewide NAEP Results, Participation Rates for NAEP, LEP, Students with Disabilities Total Cohort Results after 4 Years of Instruction in Secondary ELA, Mathematics, Global History and Geography, US History and Government, and Science. NYSAA results, Regents Exams, Regents Competency Tests, NYSESLAT Erie 1 BOCES Data Analysis Team. All rights reserved.

High School Completers High School Non-Completers Post-Secondary Plans of Completers Erie 1 BOCES Data Analysis Team. All rights reserved. The New Report Card Report …

Easy Part First… New Report Card is a repackaging of reports we have seen before. Profile Section = AOR District Profile Student Performance = AOR Overview of District Performance minus Summary plus CIS Assessment Breakdown Student Outcomes = CIS High School Information Erie 1 BOCES Data Analysis Team. All rights reserved.

Hard Part – Accountability Report In order to understand this report you must have an understanding of the changes to accountability as a result of the ESEA Waiver. The Ira Swartz memo from January 2013 summarizes this with three attachments Attachment A – Revised Method for Calculating PI Attachment B – How NYS Reset Annual Measurable Objectives (AMO) Attachment C – 4 year and 5 year Graduation Rates and Adequate Yearly Progress (AYP) Erie 1 BOCES Data Analysis Team. All rights reserved.

Methodology for Calculation of PI at the Elementary/Middle Level Look at Attachment A of Ira Swartz January 2013 memo PI for ELA and Math now gives districts credit for students in Level 1 and Level 2 who are on track to proficiency PI for calculating 4 & 8 Science remains the same because there is no way to project a track to proficiency without grade level assessments between the 4 th and 8 th grade tests. Note– this will improve a districts chances of making AYP for the Elem/Middle Level because of the formula changes. Erie 1 BOCES Data Analysis Team. All rights reserved.

Source: New Directons in Assessment, Data Use, and Accountability New York State Education Department Office of Accountability Ira Schwartz, Assistant Commissioner for DATAG Friday, July 13, 2012

PI = ([2(Count at Level 1 On Track) + Count at Level 2 Not On Track + 2(Count at Level 2 On Track) + 2(Count at Level 3) + 2(Count at Level 4)] ÷ [Count of Tested Students]) × 100 PI = ([2(0) + (146) + 2(4)+2(235)+2(121)]/[521] )x 100 PI = ([ ]/[521]) x 100 PI = (866/521) x 100 PI = 1.66x 100 = 166 Sample PI Calculation Erie 1 BOCES Data Analysis Team. All rights reserved.

Performance Level On Track to Proficiency Number of StudentsMultipierTotal 1 (Below Standards)No (Below Standards)Yes (Meeting Basic Standards)No (Meeting Basic Standards)Yes (Meeting Proficiency Standards)NA (Exceeding Proficiency Standards)NA Totals PI = 86600/521=166 Another Way to Look at PI… Erie 1 BOCES Data Analysis Team. All rights reserved.

Methodology for Calculation of PI at the Secondary Level The formula for calculating the PI at the Secondary Level remains the same. The rigor increases because the cut points for the Levels change (and are different) for ELA and Math. Erie 1 BOCES Data Analysis Team. All rights reserved.

Erie 1 BOCES Data Team. All rights reserved. Safe Harbor is an alternative way to meet performance criterion for accountability groups whose PIs are less than their EAMOs. The formula for is: PI + (200 – PI)x PIs which were reissued, are available in the ISRP portal. In the above example, the revised PI was 136 Note: Revised PI obtained from the IRSP Portal – ESEA Revised Performance Indices Report SH = ( )x0.10 = = = 142 Safe Harbor 138

Erie 1 BOCES Data Team. All rights reserved. If the SH Target matches the EAMO, it is because the formula would have yielded a result greater than the EAMO, as in the example below. The PI was 133 SH = ( )x0.10 = = 139.7> EAMO of 128 Note: Safe Harbor will no longer require schools and districts to meet the third academic indicator requirement (Science or Graduation Rate). Safe Harbor

The formula to calculate high school PI remains the same as follows: ((Level 2 + Level 3 + Level 4 + Level 3 + Level 4)/Cohort)*100 Districts get half credit for Level 2s and full credit for Level 3s & 4s. Note: Under the new scales there will be more 2s and fewer 3s and 4s. This lowers the PIs for most districts. Note: Under this methodology districts receive no credit in the PI for students who score between on a Regents exam or Pass a RCT exam. Erie 1 BOCES Data Analysis Team. All rights reserved. Cut Point Chart

Reset of the Annual Measurable Objectives (AMO) Look at Attachment B of Ira Swartz January 2013 memo Because of ESEA Waiver AMOs were reset to reflect student growth. These EAMOs (Effective Annual Measurable Objectives) are by group size – this is where the EAMO values came from on the Accountability Charts. The charts found at: MOs-for-AYPDeterminationBased-on Results.pdf Erie 1 BOCES Data Analysis Team. All rights reserved.

4 Year and 5 Year Cohort Graduation Rates and AYP The numbers in the total cohort groups change based on the following: One day enrollment changes the numbers from previous report cards. These numbers used to be based on 5 month enrollment. Incarcerated youth will not be counted as transfers. Ungraded SWD will be entered into a Cohort based upon when they first entered a 9 th grade program. Erie 1 BOCES Data Analysis Team. All rights reserved.

4 year and 5 year Graduation Rate to make AYP Its 4 year graduation rate cohort meets or exceeds the graduation goal; or its 4 year graduation rate cohort meets or exceeds the 4 year progress target. Its 5 year extended graduation rate cohort meets or exceeds the graduation goal; or its 5 year extended graduation rate cohort meets or exceeds the 5 year progress target. Erie 1 BOCES Data Analysis Team. All rights reserved.

Things to watch out for! Elementary/Middle Level Participation charts are based on Students enrolled during the test admin period. Performance charts are based on Tested Students Enrolled on BEDS day Erie 1 BOCES Data Analysis Team. All rights reserved.

Things to watch out for! Secondary Participation charts are based on 12 th Graders with valid test scores Performance charts are based on Cohort groups Erie 1 BOCES Data Analysis Team. All rights reserved.

Erie 1 BOCES Data Team. All rights reserved. Unweighted combined (or average) PIs for ELA and Math for the building or district, by subgroups. In this example, ( )/4 = = 162 Unweighted Combined ELA and Math PIs

Level 2 Charts can Help with Discrepancies Level 2 charts have different ways of displaying data that help in resolving data discrepancies. Level 2 charts also allow you to drill down to see individual students. When you see a students name it can help to determine if that student falls into one of the three reasons that may move students between cohorts. Erie 1 BOCES Data Analysis Team. All rights reserved.

Reasons for Data Surprises Elem/MS PI formula changes caused some districts to make AYP and others not to make AYP – students not on track to proficiency made the difference between districts making AYP at Elem/MS Level High School Levels range changes made it more difficult for student groups to make AYP at HS level Differentiated accountability changed the EAMO targets because they are based on the NCLB subgroups (buckets) Changes in Grad % for four and five year were caused by rule changes for placing students in cohorts Changing from 5 month enrollment to 1 day enrollment caused numbers of students districts were held accountable for to rise Ungraded students who received IEP Diplomas were added to cohort numbers Erie 1 BOCES Data Analysis Team. All rights reserved.

Source: New Directons in Assessment, Data Use, and Accountability New York State Education Department Office of Accountability Ira Schwartz, Assistant Commissioner for DATAG Friday, July 13, 2012

School that is not a Priority or Focus School that: a) Has large gaps in student achievement among subgroups or students or b) Has failed to make AYP for three consecutive years with same subgroup on same measure or c) Is located in a non-Focus district but is among the lowest in the state for the performance of one or more subgroups and for which the school is not showing progress

Contact us for Clarification Timothy Johnson David Scalzo Erie 1 BOCES Data Analysis Team. All rights reserved.