State Reform Efforts: Multiple Measures of Accountability & Improvement Institute for Governors’ Education Advisors NGA Association for Best Practices Amelia Island, Florida November 3-5, 2001 Mari Pearlman

2 The current landscape: dangers and opportunities Dangers: Single focus on annual test scores Lack of useful data to interpret these scores Little connection between testing and instruction Search for the guilty rather than search for success Opportunities Systemic improvement for all parts of the education system Useful data collection and interpretation to make improvement an empirical rather than rhetorical and emotional issue

3 Protection against the dangers: first steps Align the entire accountability system, with particularly careful connections between what is taught and what is tested Establish direct feedback loops between test score information and teacher instructional planning Use multiple different measures of student achievement, including but not limited to standardized tests Employ sensible data collection and analysis plans

4 Aligning the accountability system - 1 Criteria or standards for learning A clear definition of what students will be able to do to demonstrate that learning— the specifications for assessments, both ongoing (classroom-based) and summative (school, district, state-based) A coherent continuum for applying those criteria by developmental level of students

5 Aligning the accountability system - 2 A curriculum that supports the criteria for learning and includes demonstrations of learning that are aligned with the assessments Instruction that implements the curriculum, with continuing teacher professional development A plan for collecting data that defines what questions the data should answer and then specifies the data to be collected A plan for using those data to inform curriculum, instruction, teacher professional development, and assessment designs

6 Scope and sequence: Content, performance & assessment The sequence of alignment matters: standards for what students must know (and teachers must teach)—content standards Standards for how much of the content students must know – performance standards A definition of what learning looks like—what should students—at each level--be able to do if they meet the performance standards— assessment specifications

7 Scope and sequence— Curriculum & Instruction A careful examination and articulation of what should be taught, when it should be taught, and how the knowledge and skills articulated are connected to the final expectations for student learning—the curriculum Classroom instruction that really focuses on the curriculum—helping teachers make good choices about what, of all possible things, should be taught—teacher mentoring and evaluation and professional development

8 Scope & Sequence: Data collection and interpretation What questions do you want to be able to answer about student learning and the effectiveness of the education system? To answer those questions, what data would you need to collect? How can you use these collected data to close the loop and improve the criteria for student learning, the curriculum, the instruction, and teacher professional development?

9 Realistic Expectations Think realistically about milestones for student learning and their connections with annual testing What is the gain you expect to see from year to year? Given a particular range of performance in year 1 (say 1 st quartile on the test, or lowest 25%), what is reasonable to expect as a growth goal in the next year?

10 Check those expectations against What the curriculum says teachers should teach How much actual instructional time is devoted to these areas of learning at each level What kind of assessment data you actually have that would be evidence of learning against these expectations

11 And against Whether assessment data are ever available in time for teachers to change the way they teach particular students Whether teachers are given any structured guidance about the use of assessment data, changes in instruction to serve student learning, or aligning their practice with ultimate goals for student learning

12 Using multiple measures of educational effectiveness: outputs Output measures Multiple formal assessments: this is not impossible. All teachers in a school or district can come up with easy-to-use performance measures that could supplement norm- referenced standardized results. Informal assessments: teacher standardized assessment at the beginning and end of each year—create a standard form and keep those data. Grades and teacher evaluations of student performance Student performance itself—the science fair, the music program, the art exhibit, the auto shop, the contests and awards from disciplinary organizations

13 Multiple measures 2: inputs Teacher profiles- Number with full certification Number with content majors and advanced degrees Years of experience School leadership profiles— Teaching experience Training and experience as a school leader Demographic variables that matter % of students who come and go within a teaching year % of drop outs Absenteeism

14 Realistic Expectations Recognize that this is an ongoing, continuous process— this is about human growth from birth to age 21 Student learning—in particular, not enough of it—is not like an infection that can be cured with a massive dose of antibiotics Everyone must be accountable—in particular, it is not the fault of the students who are struggling. Improving student achievement is not just the job of teachers. Parents, principals, superintendents, legislators, the public all have a role and responsibility