Educator Effectiveness: State Frameworks and Local Practice(??) CCSSO Annual Conference, June 2012 Juan M. D’Brot Executive Director of Assessment and Accountability West Virginia Department of Education

Assessment Continuum in WV Promoting a balanced assessment system against WV State Standards Formative Diagnostics Interim Diagnostics Interim Benchmarks Summative Assessments

Formative, Interim, and Summative Assessments Summative – WESTEST 2 – Online Writing Assessment (PT) Interim – Benchmarks in Math, ELA, Science, Social Studies Formative – Acuity Item Banks in Math, ELA, Science, Social Studies – West Virginia Writes: Practice Writing Program – Other District Purchased Supports – Teacher Developed Assessments

Major Question: Purpose? Purpose of Summative – Evaluative against a set of standards – Least flexible: long-cycle – Broad sense of the educational state of the state or district Purpose of Interim – Instructional, – Evaluative, and/or – Predictive – Medium-cycle Purpose of Formative – Process to adjust teaching and learning – Instructional diagnostics – Typically classroom specific – Very short-cycle

Examining the Use of Formative and Interim High Use vs. Effective Use – Emerging case study of a district and school Context – Targeted professional development – Dedicated school leadership and staff – State-level support around the use of formative and interim assessments Early Results:

Grade 9 School A and B

Grade 10 School A and B

Grade 11 School A and B

WESTEST 2 Writing: At or Above Proficiency

Examining the Use of Formative and Interim Strong support for use of writing short- cycle assessments Currently examining use of other short- cycle (formative) and medium-cycle (interim) assessments on summative assessment results Implications on teacher evaluation systems looking for triangulated evidence of student learning

Multiple Use Concerns Tension between Compliance (accountability) and Continuous Improvement (short-cycle assessments) Adequate Yearly Progress Improvement and Growth Teacher Evaluation Systems Student Learning Conflicting?

Multiple Use Concerns Tension between Compliance (accountability) and Continuous Improvement (short-cycle assessments) – Summative Assessment Gaming Cheating “Teaching to the Test” – Interim/Formative Assessment Fabricated student learning gains? Student sorting to highlight teacher “gains”? What does the research say? Not impact/effect, but valid link to teacher input/output? Cognitive vs. Non-Cognitive measures? Scratching the surface

The West Virginia Case A transition from nothing to something – Old: 5 years experience  evaluation “only when the principal determines it to be necessary…” – New: Yearly evaluation for all educators, principals, and evaluators using research- based professional standards and evidence of student learning

Developing the System Goal: To develop a supportive system around valid data to drive improvement Access: A variety of data to support self- reflective practice: – Summative Growth Data – Interim Benchmarks – Formative Diagnostic

Developing the System Context: – Politics: National vs. State Pressure – Policy: Accountability – Pragmatics: Using actionable and valid data Development: Stakeholder driven – Board of Education – SEA Staff – LEA staff – Teacher Unions – Legislators – Community Members

CategoryInputOutput 5 Professional Teaching Standards80% 14 Critical Standard Elements – 4 pt. rubric 5.7%/CSE Student Learning Outcomes20% Summative Growth Data Math & ELA Only 5% Other Student Learning Evidence Any Content15% Professionalism (Mandatory)N/A The System System Components

The System 20% Student Learning Outcomes – Minimum 5% Student Assessment Data School-wide Growth – Student Growth Percentiles – A host of evidence – Agreed upon with principal input – Multiple measures Acuity (Math, ELA, Science, Social Studies) WV Writes (Writing) Other evidence of student learning Teacher Developed Assessments

The System Overall Ratings Category100 Point Scale Distinguished Accomplished54-88 Emerging33-54 Unsatisfactory <33

Implementation – Concluded 1 st pilot year in a small number of schools – Self-ratings – Accomplished across the board… – 2 nd pilot year with schools in every district represented – Full statewide implementation for all schools in the state

Implications for a “Young System” Embedded Program Evaluation – WVDE Office of Research small scale implementation study and program evaluation – Formative data has resulted in changes to the system and program – Exploring third-party consulting for enhancements to guidance, system, and process (post-revision)

Lessons Learned Politics vs. Pragmatism – Representative inclusion vs. Over/under- representation of certain views – Political support to push a bill to law Capacity and Foresight – In-house development and reinventing the wheel? (…more politics) – Building a system for the present vs. building a system for the future – Revising the re-invention

Thank You Juan M. D’Brot Executive Director Office of Assessment and Accountability West Virginia Department of Education