1. Accountability: Where Are We Going?
Martha Musser, Coordinator
NYS Education Department

2. This may be the Dawning of the Age of Accountability
Contract for Excellence
NCLB Reauthorization
NYC and other local accountability initiatives

3. Contract for Excellence
Governor has proposed a Contract for Excellence that calls for an enhanced State accountability system, including:
New accountability standards based on State assessments and other indicators of progress, such as graduation rates or college attendance and completion rates.
Growth model by
Value-added model by based on new or revised state assessments.
Expanded SURR system, resulting in the identification of up to 5% of State school’s by for restructuring or reorganization.

4. Contract for Excellence: Plans for Intervention
School Review Teams conduct resource, program and planning audits of SINI and SRAP schools and assist all SINI and SRAP schools in development of improvement plans.
Joint School Intervention Teams, whose members are either appointed by Commissioner or educators from the district, review and recommend plans for reorganizing or reconfiguring schools that are to be closed.
Distinguished Educators assist low-performing schools and districts.
The services of all the above are a charge to the school district.

5. Contract for Excellence: Now it’s Personal
Commissioner shall define in regulation deficient district performance.
School districts hiring a superintendent after the effective date of regulation must include a provision in superintendent’s contract that after two years of deficient district performance the superintendent will fully cooperate with a distinguished educator.
After four years of deficient performance, school board must seek to remove a superintendent or provide a rationale to the Commissioner for why board should not take such action.
After six years of deficient district performance, the Commissioner shall commence action to remove the board or board members.

6. Distinguished Educators and Deficient District Performance
Consult with Commissioner on removal of superintendent.
Serve as ex officio, non-voting members of Board of Education.
Review all school and district improvement plans and either endorse or make recommendations to the Board of Education for change.
Boards of Education must make changes as required by Distinguished Educators unless upon petition by Board of Education Commissioner decides there is a compelling reason not to implement the modification.

7. Status, Growth and Value- Added Models Defined
Contract for Excellence requires SED to implement growth and value-added models and meet any Federal requirements for such models.

8. Status Models Defined
Status or improvement models, the current requirement for measuring Adequate Yearly Progress (AYP) under No Child Left Behind, measure progress by tracking the improvement at the same grade levels within the school over time. (This year’s grades 3-5 compared to last year’s grades 3-5.)

9. Growth Models Defined
Growth models generally refer to accountability models that assess the progress of a cohort of individual students over time with the intent of measuring the progress these students have made (Performance in fourth grade compared to performance in third grade).

10. Value-Added Models Defined
Value-added models generally refer to a specific type of growth model in which student demographic data or other statistical controls are used to attempt to analyze the specific effects of a particular school, program, or teacher on student learning.
These models ask whether the school has increased the measured achievement of students more than expected based on data from similar schools.

11. Purpose of Status, Growth, and Value Added Models
Status model: Determine whether an increasing percentage of students are gaining proficiency over time.
Growth model: Determine how much progress groups of students are making over time.
Value-added model: Determine success of schools, programs, or teachers by measuring student growth over time while controlling for non-school variables that impact on student performance.

12. Value-Added Models: The Holy Grail of Accountability?
Much controversy about value-added models.
Some claim that good value-added models exist (Sanders), statistical models not based on vertical scales.
Some claim that the state of the psychometric art does not currently support vertical scales.
Value-added models are not easy:
Less sophistication is required to make accurate determinations about status then growth.
Acquisition of knowledge is not linear.
Value-added models are heavily dependent upon the robustness of their assumptions.
Value-added models break down more quickly as groups become smaller.

13. Growth Model System Requirements
Annual assessments in successive grades
A unique student identification system
Vertically aligned or scaled assessments
Grade by grade standards
A minimum of two years of assessment data per student
Value-added systems require additional data on those demographic or resource factors for which the model seeks to control

14. NY and Growth Models
NYS Grades 3-8 Testing Program uses Vertically Moderated Standards (VMS)
Student progress is measured from grade-to-grade relative to proficiency in meeting the standards (rather than in terms of change in scale scores).
VMS is applicable to some but not all growth model designs.
NY will by the end of have only two years of test data, limiting our ability to create growth trajectories for individual students.
SED’s goal is to select a growth model architecture by April.

15. NCLB: Status Report
Revised State standards and Annual Measurable Objective’s (AMOs) established.
05-06 Accountability Decisions Released.
Development of elementary/middle level attendance standards in progress.
Regents continuing deliberations regarding raising high school graduation standards.
Regents engaging in NCLB reauthorization advocacy.

16. State Standards State Standards for 06-07 are:
155 for Grade 3-8 ELA and math
165 for HS ELA and math
State Standards for 07-8 are:
160 for Grade 3-8 ELA and math
170 for HS ELA and math
State standard is used to determine which schools must do Local Assistance Plans (LAPs) and which schools and districts may be high performing.

17. Original Annual Measurable Objectives for 2002–03 to 2013–14
School Year Elementary-Level Middle-Level Secondary-Level
ELA Math ELA Math ELA Math
2004–
2005–
2006–
2007–
2008–
2009–
2010–
2011–
2012–
2013–

18. Revised Annual Measurable Objectives for 2005–06 to 2013–14
School Year Elementary & Middle-Level Secondary-Level
ELA Math ELA Math
2005–
2006–
2007–
2008–
2009–
2010–
2011–
2012–
2013–

19. Calculating AMOs
In and combined, 20% of NY’s public school students were enrolled in school’s with PI’s below:
Grade 4 ELA: 123
Grade 8 ELA: 107
Grade 4 Math: 136
Grade 8 Math: 81
In , the percent of students below the AMO’s were:
Grade 4 and 8 ELA combined: 13.3%
Grade 4 and 8 math combined: 4.2%
Grade 3-8 ELA results:
AMO at 13.3% = 122
AMO at 20.0% = 133
Grade 3-8 math results:
AMO at 4.2% = 86
AMO at 20.0% = 132

20. Limited English Proficient (LEP) Students
All LEP students in grade K–12 must take the NYSESLAT annually.
LEP students in grades 3 through 8 enrolled in U.S. schools (not including Puerto Rico) for less than one year (enrolled on or after January 2, 2006) were not required to take the NYSTP ELA assessment in January For such students who did not take the ELA assessment, valid scores on the NYSESLAT Reading/Writing and Speaking/Listening components will meet the ELA participation requirement.
The eligible LEP students must be identified in the repository using Program Service 0242.
NYSESLAT performance levels will not be used in calculating the Performance Index. LEP students meeting the criteria to use the NYSESLAT in lieu of the ELA will not be included in the Performance Index calculation.

21. 2002 Graduation-Rate Cohort Definition
This cohort will be used to determine if the district or school meets the graduation-rate requirements for the 2006–07 school year. The 2002 graduation-rate cohort consists of all students, regardless of their current grade status, who were enrolled in the school on October 6, (BEDS day) and met one of the following conditions:
first entered grade 9 (anywhere) during the 2002–03 school year (July 1, 2002 through June 30, 2003); or
in the case of ungraded students with disabilities, reached their seventeenth birthday during the 2002–03 school year.

22. 2002 Graduation-Rate Cohort Definition (cont’d)
The State will exclude the following students when reporting data on the 2002 cohort:
students who transferred to another high school (excluded from the high school graduation-rate cohort) or district (excluded from the district graduation-rate cohort) or criminal justice facility after BEDS day 2005;
students who transferred to an approved alternative high school equivalency preparation (AHSEP) or high school equivalency preparation (HSEP) program (CR 100.7) after BEDS day 2005 and met the conditions stated on the next slide;
students who left the U.S. and its territories after BEDS day and before August 30, 2006; and
students who died after BEDS day 2005 and before August 30,

23. 2002 Graduation-Rate Cohort (Transfers to GED Removed from Cohort)
Students will be removed from the cohort for the school and district from which they transferred to an AHSEP or HSEP program if the final enrollment record shows that on June 30, 2006 the student
a) earned a high school equivalency diploma; or
b) was enrolled in an AHSEP or HSEP program.
Students will be removed from the school cohort if the enrollment records showed that the student transferred to a different high school and was working toward or earned a high school diploma.
Students will be removed from the district cohort if the enrollment records show that the student transferred to a high school in a different district and was working toward or earned a high school diploma.

24. 2003 Graduation-Rate Cohort
Beginning with the 2003 graduation-rate cohort (used for accountability in ):
students are included in the cohort based on the year they first enter grade 9 (or for ungraded students, the year they turn 17).
students who have spent at least five months in a district/school during year 1, 2, 3, or 4 of high school are part of the district/school cohort unless they transfer to another diploma-granting program.

25. Inclusion Rules for the 2003 Graduation-Rate Cohort
A student will be included in the district/school cohort if the student’s last enrollment record in the district or school shows:
that the student was enrolled for at least five continuous (not including July and August) months and the ending reason was not one of the following: transferred to another New York State district or school, died, transferred by court order, or left the U.S.
fewer than five month’s enrollment and an ending reason indicating that the student dropped out or transferred to a GED program and the student’s previous enrollment record in that district/school (assuming one exists):
indicates that the student dropped out or transferred to a GED program, and
that the student was enrolled in the district/school for at least five months.

26. School Status for All State Schools
Number
Percent
Good Standing
3,743
84.4%
3,749
84.3%
In Improvement Status under Title 1
502
11.3%
506
11.4%
Requiring Academic Progress
192
4.3%
193
Total in improvement status
694
15.6%
699
15.7%
Total Schools
4,437
4,448

27. Student Enrollment by Accountability Status
NYC
Rest of State
Total
Schools
Enrollment
Requiring Academic Progress - Year 1 9
10845 36
40043 45
50888
Requiring Academic Progress - Year 2 6
5957 38
53161 44
59118
Requiring Academic Progress - Year 3 10
21515 17
18481 27
39996
Requiring Academic Progress - Year 4
82411 15
19412 51
101823
Requiring Academic Progress - Year 5 14
19748 1
1032
20780
Requiring Academic Progress - Year 6 5
1335
Requiring Academic Progress - Year 7
1759
In Need of Improvement - Year 1 62
49523 46
28107
108
77630
In Need of Improvement - Year 2 58
49617 33
27957 91
77574
In Corrective Action
32074 23
223305 67
54379
Planning for Restructuring
61551 31
30436 77
91987
Restructuring - Year 1 22
17050 20
20459 42
37509
Restructuring - Year 2 57
52336 7
3741 64
56077
Restructuring - Year 3 47
46862
7460
54322
Total Title I SINI
336
309013
170
140465
506
449478
Total SRAP 86
143570
107
132129
193
275699
Grand Total
422
452583
277
272594
699
725177
Note: Enrollment is based on counts from October 2005 BEDS survey.

28. Statewide Status of SINI and SRAP Schools 2005-06 vs. 2006-07
Status Improved or Stayed the Same
Status Became Worse
Total Schools In Status
Percent Improving or Staying the Same
SINI 1 61 70
131
46.6
SINI 2 33 51 84
39.3
Corrective Action 19 76 95
20.0
Planning for Restructuring 12 31 43
27.9
Restructuring 1 17 58 75
22.7
Restructuring 2 11 62 73
15.1
SRAP 1 44 28 72
61.1
SRAP 2 14 42
66.7
SRAP 3 15 59
25.4
SPAP 4 6 9
40.0
SRAP 5 or 6 3 1 4
75.0
Total
249
444
693
35.9
Note: A school’s status would improve or stay the same if the school made AYP in the area(s) of identification which gave the school its status. A school’s status becomes worse if it fails to make AYP in one or more of the area of identification which gave the school its status.

29. WEIGHTED PI BY ACCOUNTABILITY STATUS - ALL SCHOOLS
3-8 ELA
3-8 Math
HS ELA
HS Math
Acct Status
Acct Enrol>30
Wtd PI
In Good Standing
920,862
163
914,553
168
109,948
183
185
In Need of Improvement - Year 1
37,283
132
37,069
137
3,694
152
159
In Need of Improvement - Year 2
41,845
133
41,132
136
4,386
146
150
In Corrective Action
24,988
127
24,747
125
3,989
144
149
Planning for Restructuring
32,143
117
31,639
110
9,971
138
145
Restructuring - Year 1
20,826
119
20,447
114
1,619
122
130
Restructuring - Year 2
37,304
116
36,921
874
139
Restructuring - Year 3
41,208
40,601
111
867
103
124
Requiring Academic Progress - Year 1
9,314
154
9,214
8,346
180
184
Requiring Academic Progress - Year 2
16,516
158
16,364
161
8,700
178
182
Requiring Academic Progress - Year 3
9,700
9,602
6317
169
6,317
170
Requiring Academic Progress - Year 4
12,526
142
12,418
141
17,151
Requiring Academic Progress - Year 5
3,763
3,722
113
3,628
Requiring Academic Progress - Year 6
443
427 99
419
118
Requiring Academic Progress - Year 7
1,236 94
1,220 91
212
Has No Status - Regulations Do Not Apply
316
308
140
Total Title I SINI
235,597
123
232,556
25,400
147
Total SRAP
53,498
148
52,967
44,773
164
Grand Total
1,210,273
1,200,384
180,121
171
175

30. NCLB Reauthorization:
Regents have developed positions regarding:
Schools and districts with multiple federal designations
Growth Models
Targeted interventions
AYP Methodologies
Additional time to meet graduation standards
ELL testing issues
Implementation of choice and SES
Safe Schools

31. NCLB Reauthorization: What we are Hearing
Serious discussions about NCLB are beginning but reauthorization more likely to occur in 2009 than 2007, very unlikely to occur in 2008.
Key issues being discussed:
Full funding
Growth models
N size and confidence intervals
Required interventions, including SES and choice
Assessments: LEP, SWD, high school
Highly Qualified Teachers vs. High Quality Teaching
National Standards
Science Assessments

32. Other Accountability Initiatives: NYC
Children First:
Valued Added Model.
More Ragu than KISS.
Each school receives letter grade (A-F) on school progress report.
Progress report consists of four categories:
School Environment (15%)
Student Performance (30%)
Student Progress (55%)
Additional Credit

33. Children First Progress Report
School’s grade is based upon how well the school performs relative to other City schools (1/3 of grade) and to a peer group of approximately 40 schools with similar demographics (2/3 of grade).
State assessments used to measure performance of students over time.
Additional credit given to improved performance among low achieving students and various disaggregated groups.
Factors such as parent, student, and teacher surveys; credit accumulation; PSAT scores used in determining school grade.

34. Challenges for SED Growing Pains Testing Times Vision 2020
A Cart Before the Horse?

35. Challenges Ahead for School Districts
Immediate:
Single Grade 3-8 Performance Index makes schools and districts responsible for more disaggregated groups.
New standard setting for grade 3-8 assessments may challenge middle schools even more.
Changes in testing practices for LEP students require that bilingual education programs emphasize rapid acquisition of English as well as fluency in the native language.
New graduation standards will raise expectations for middle schools and force high schools to reveal “hidden students.”
Longer term:
If Governor and Legislature show school districts the money, they are also going to demand that districts show results.

36. Challenges Ahead: Lists, Lists, Lists
How do we make all of these work together:
SURR
SINI, DINI
SRAP, DRAP
IDEA Districts
Title III AMAO’s
Persistently Dangerous Schools
High School Initiative
Highly Qualified Teachers

37. This may be the Dawning of the Age of Accountability
Key Questions:
How do we design accountability models that compel movement from awful to adequate without impeding the movement from good to great?
How do we move from beating the odds to changing the odds?

38. More Information Ira Schwartz, Coordinator
Office of School Improvement and Community Services (NYC)
Accountability PowerPoint for :
Manuals for NYS Student Identifier System and the Repository System: