2010 OSEP Leadership Mega Conference Collaboration to Achieve Success from Cradle to Career Evaluating for Impact Washington State’s Perspective: Our Blueprint for Systems Improvement Leslie Pyper and Valerie Arnold Strand 4 : State Directors Second Session

Outcome-Based Evaluation “Not how many worms the bird feeds its young, but how well the fledgling flies” United Way of America, 2002

Logic Model

Framework Begin with the end in mind….Dr. Covey Focus on long-term data driven planning process….shifting from pure emphasis on compliance to dedicating resources to enhance outcomes. Time-released strategies to date:  Internal  External  Continuous Improvement Planning

Benefits of Logic Model Understand, verify, and increase the impact of improvement activities Improve implementation mechanisms to be more effective and efficient Verify that we are doing/achieving what we think we are…plans change over time Produce data and verify results Valid comparisons between improvement activities…what to retain or let go Fully examine and report improved results

Evaluative Inquiry Needs/Asset Assessment – Characteristics, needs, and priorities? – Potential barriers and facilitators? – Most appropriate? Process Evaluation – How is SPP/APR implemented? Fidelity? – Activities/services delivered as intended? – Participants being reached as intended? Outcome Evaluation – Desired changes occurring? – SPP making a difference? – What seems to work well? Not work? – Are there unintended outcomes?

Internal Strategies Reorganization of special education department Staff Retreat – Vision & Structure Defined work group roles & responsibilities Centralized SPP/APR planning – core team w/cross representation Development of web-based non-compliance identification, notification, verification, and validation system

“General Supervision” AND Washington State Special Education Monitoring 8 Dispute Resolution Safety Net Integrated Program Review Fiscal Mgmt Learning Improvement Data Collection & Analysis Restructured Work Groups…

Core Functions & Standards Allocate FacilitateRegulate Vision & Structure…

SPP/APR 20 Indicators LEA App Self- evaluation Self-study System Analysis Washington Special Education Integrated Program Review Process Data Collection Training & TA Review, Analysis, Selection Review, Analysis, Selection TA/ feedback District Profile Verification Visits On-site cooperative venture TA Specific Role Definitions…

Core Team w/Cross Representation Centralized Planning Team Fiscal Dispute Resolution Learning Imprvmt Data Program Review Washington State Performance Plan

iGrants IDEA Compliance Package 442 Web-based Corrective Action System… District ESD

External Strategies Alignment of Coordinated Services Agreement with SPP/APR Training ESDs to access regional data profiles for T/TA planning Alignment of State Needs Projects with SPP/APR priorities Strengthened relationship with and broadened access to Western Regional Resource Center Leadership initiatives within Learning Improvement Team

Regional Educational Service Districts Coordinated Service Agreements 2010 Regional Data Profiles for T/TA

State Needs Projects Alignment with SPP/APR

Consultation and Leadership Initiatives Learning Improvement

Continuous Improvement Planning Possible formation of policy review group within existing staff resources Integration of planning and monitoring within general supervisory system Development and implementation of state-wide impact evaluation system for discretionary funds aligned with SPP/APR priorities

Leslie Pyper, Learning Improvement Coordinator at Valerie Arnold, Program Review Supervisor at Primary Website Link: For More Information…