1. Commissioner’s Performance-Based Accountability Task Force: System, Indicators, and Inputs Deb Wiswell, Keith Burke, & Scott Marion December 18, 2009

2. Overview of Meeting Commissioner's Performance Based Accountability System Task Force. 11/30/092  Brief overview of entire system  Presentation of the “input” system  Indicators and metrics  A general approach for the performance system

3. Adequacy Accountability System Commissioner's Performance Based Accountability System Task Force. 12/18/09  Two approaches for demonstrating that the school is providing the opportunity for an adequate education  Input System—Based on a subset of the school approval standards  Performance System—what this committee is required to design  A school has to demonstrate that it is providing an opportunity for an adequate education by meeting the requirements of at least one of the systems, but not both.

4. The Input System Commissioner's Performance Based Accountability System Task Force. 12/18/09 I. A school may demonstrate, through the input-based school accountability system, that it provides the opportunity for an adequate education as set forth in RSA 193-E:2-a by establishing that it met the following school standards in effect as of the effective date of this section: (a) English/language arts and reading as set forth in Ed 306.37. (b) Mathematics as set forth in Ed 306.43. (c) Science as set forth in Ed 306.45. (d) Social studies as set forth in Ed 306.46. (e) Arts education as set forth in Ed 306.31. (f) World languages as set forth in Ed 306.48. (g) Health education as set forth in Ed 306.40. (h) Physical education as set forth in Ed 306.41. (i) Technology education, and information and communication technologies as set forth in Ed 306.42 and Ed 306.47. (j) School year as set forth in Ed 306.18. (k) Minimum credits required for a high school diploma as set forth in Ed 306.27(f) and (m).

5. III. In order to demonstrate that a school provides the opportunity for an adequate education through the input-based school accountability system under paragraph I: Commissioner's Performance Based Accountability System Task Force. 12/18/09 (a) T he commissioner shall require school officials to submit a narrative explanation detailing how the school has complied with each of the standards included in the opportunity for an adequate education contained in paragraph I. The school principal and school district superintendent shall certify in writing that the responses submitted are accurate. The commissioner shall develop a form which conforms to the provisions of this paragraph. (b) The commissioner shall review the responses to each school’s self-assessment required under this section and shall verify that the responses comply with the standards included in the opportunity for an adequate education specified under paragraph I. (c) Schools that successfully demonstrate that they provide the opportunity for an adequate education through the input-based school accountability system for any year beginning with the 2009-2010 school year shall be required by the commissioner to resubmit the narrative explanations at least once every 2 years. (d) Schools that are unable to demonstrate that they provide the opportunity for an adequate education through the input-based school accountability system for the 2009-2010 school year, or for any year thereafter, shall be required by the commissioner to resubmit the narrative explanations annually until such demonstration has been made. (e) The commissioner shall integrate, to the maximum extent practicable, the input-based school accountability system to demonstrate the opportunity for an adequate education with the school approval process pursuant to RSA 21-N:6, V. (f) Beginning September 1, 2012, the department shall annually conduct site visits at 10 percent of schools statewide to assess the validity of the input-based school accountability system and to determine whether those schools demonstrate the opportunity for an adequate education by meeting the school standards identified in paragraph I.

6. Demonstration of Input Data Collection Commissioner's Performance Based Accountability System Task Force. 12/18/09  Note the use of the word “narrative”  The AYP task force was concerned that unless this was well structured, it would be almost impossible to score reliably and it would take too time much for school administrators to do well  Therefore, Keith Burke in collaboration with Deb Wiswell, content specialists, and the AYP task force, produced the input data collection software that we will now demonstrate  The purpose of this demonstration is to simply provide you with an overview of the types of indicators/measures being collected  You will all have an opportunity on the specifics of this approach, but NOT TODAY!

7. Switch to Input System demo Commissioner's Performance Based Accountability System Task Force. 12/18/09

8. Back to the “performance system” Commissioner's Performance Based Accountability System Task Force. 12/18/09  The discussion of potential indicators a few weeks ago was very interesting and made us think hard about how to respond to the task force’s input  We will propose a system later that we think can meet both the law and the interests of the task force (at least we understand it)  First, a few reminders of what is legally required….

9. SB 180 Task Force must…  (a) Define the performance-based accountability system to be used by schools that will ensure that the opportunity for an adequate education is maintained.  (b) Identify performance criteria and measurements.  (c) Establish performance goals and the relative weights assigned to those goals.  (d) Establish the basis, taking into account the totality of the performance measurements, for determining whether the opportunity for an adequate education exists, which may include the assignment of a value for performance on each measurement.  (e) Ensure the integrity, accuracy, and validity of the performance methodology as a means of establishing that a school provided the opportunity for an adequate education as defined in RSA 193-E:2-a.

10. SB 180 Requirements  The task force shall develop a performance- based scoring system using only the best available data and indicators which are already provided to the department and/or performance measures that schools are already required to provide the department under other state or federal law.  See handout listing already collected indicators

11. A little more on potential indicators Commissioner's Performance Based Accountability System Task Force. 12/18/09  There seemed to be considerable interest in either non- NECAP-based indicators and/or process indicators  A recent analysis of other states suggests that few, if any, states are using such data in their school accountability systems.  As you can see on the following slide, no states are venturing into using the types of process indicators we discussed in November.

12. From Perie, et al. (2007) Commissioner's Performance Based Accountability System Task Force. 12/18/09  California’s Academic Performance Index (API) includes both criterion-referenced and norm-referenced assessment results in ELA, math, science, and history/social sciences—different indicators are used at different grade levels.  Kentucky uses results from their science and social studies tests in their state accountability system. Also include a writing portfolio at grades 4, 7, and 12, test arts & humanities at grades 5, 8, and 11, and practical living & vocational studies at grades 5, 8, and 10.  Oklahoma and Ohio use additional indicators in their high schools that measure the percentage of students receiving advanced placement (AP) credit and taking the ACT or SAT.  Missouri uses the following indicators in their state accountability system in addition to the traditional achievement and participation measures:  ACT (percent of graduates scoring at or above the national mean)  Advanced Courses (percent of credits earned)  Career Education Courses (percent of credits earned)  College Placement (percent of graduates entering college)  Career Education Placement (percent of career education completers placed in occupation)

13. Process Indicators Commissioner's Performance Based Accountability System Task Force. 12/18/09  What process and/or non-cognitive indicators are recommended?  Andy Porter, after surveying the school effectiveness literature, recommended a set of school quality indicators as a starting point for having schools focus on high leverage processes and indicators

14. Porter, Andrew C. (1991). Creating a System of School Process Indicators. Educational Evaluation and Policy Analysis, 13, 13-29. Commissioner's Performance Based Accountability System Task Force. 12/18/09  Instructional leadership  Existence of a focused mission  High expectations for all students  Task-oriented climate  System for monitoring outcomes within the school  School and class size  Grade level organization  Teacher planning time  Availability, quantity, and credentials of teacher aides  Number of teacher preparations required  Shared decision making

15. Except… Commissioner's Performance Based Accountability System Task Force. 12/18/09  NHDOE is not in a position to collect data on most of the indicators on the previous slide  Further, NH schools are not in a position to provide systemic, accurate, and reliable data to the STATE on most of Porter’s indicators  Are these things important?  Of course, but does that mean the state needs to collect them?  We would argue that the answer is a resounding “no”!  But should schools and districts be collecting such data?  In this case, we would argue that the answer is “yes” IF the local board and stakeholders believe the indicators are important

16. Indicators and Metrics Commissioner's Performance Based Accountability System Task Force. 12/18/09  Indicators are the data sources such as the NECAP math testing  Metrics are the way that we quantify such indicators  Status = A point-in-time measurement, e.g., 72% of the students are proficient in math  Improvement = Is generally the change in status measures when students are NOT matched, e.g., 5% more students are proficient this year compared to last year  Growth = Is based on determining changes over time based on following the same students, e.g., our 5 th grade students grew at a rate of X compared to where they were in 4 th grade.

17. Potential NECAP Metrics Commissioner's Performance Based Accountability System Task Force. 12/18/09  Status  Growth  Gaps  Status—e.g., are there significant differences in index values for students receiving free and reduced price lunch compared with those who do not receive free and reduced price lunch?  Growth—e.g., are there significant differences in growth rates for students receiving free and reduced price lunch compared with those who do not receive free and reduced price lunch?

18. An example from Colorado: Indicators in each of four categories Status (Performance Rating for status assigned based on % of metrics falling in each rating category) Overall School Status Status Gaps (or could fall under a “gap” indicator) Growth (Performance Rating for growth assigned based on % metrics falling in each rating category) Overall Growth Status Growth Gaps (or could fall under a “gap” indicator) Post-Secondary (Performance Rating for PS assigned based on % metrics falling in each rating category) Colorado ACT Graduation Rate Dropout Rate Auxiliary (factored in as business rules – e.g., if school does not meet “TP”, school cannot be rated on “performance plan” Test Participation School Safety School Finance 18

19. A Multi-Level Performance Accountability System for NH Commissioner's Performance Based Accountability System Task Force. 12/18/09  Level One  A very limited set of common (across the state) indicators and metrics  Level Two  Locally determined goals, targets, and indicators

20. Level One: State System Commissioner's Performance Based Accountability System Task Force. 12/18/09  A very limited set of common indicators and metrics  Applied consistently across all schools in the state  Focused on unarguable outcomes, e.g., NECAP, graduation rate, postsecondary assessments, attendance

21. Level One: State System Indicators Commissioner's Performance Based Accountability System Task Force. 12/18/09  All schools  Test participation  NECAP index scores (status)—reading, writing, math, science  NECAP status gaps—reading, writing, math, science  Attendance/truancy  Additional indicators for K-8 Schools  NECAP growth percentiles (growth)—reading, math  NECAP growth gaps—reading, math  Additional indicators for High Schools  Graduation rate  Postsecondary assessments (AP, SAT, ACT)?

22. Level Two: Locally-determined system Commissioner's Performance Based Accountability System Task Force. 12/18/09  A very limited set (e.g., 2-5) of district/school-determined goals, targets, and indicators  For example, “increase the % of students achieving their NWEA growth targets to 90% by 2015”  Could (should?) use the Follow-the Child framework  Local Board approval of goals, targets, and indicators  Local district would have to publish the goals and indicators it is using AND the results of how it is doing on achieving these goals  NH DOE would provide guidelines on appropriate process and outcome indicators  NH DOE would have to approve the goals and indicators process (but not necessarily the specific goals)

23. Considerations Commissioner's Performance Based Accountability System Task Force. 12/18/09  Level One—can meet the requirements of SB 180 on its own  Level Two—should it be required of all districts/schools or left optional?  Theory of action supports requirement  “unfunded mandate” argues for optional  If optional, it all rests on Level One  How should the evaluations from Level One and Two be combined?  Relatively equal weight?  Level two as extra credit or tie-breaker?  Other considerations, concerns, questions, etc?