Presentation to the SIGnetwork SPDG Directors’ Session Westat’s Evaluation of the SIG Program: Findings from the Leadership Study Alison Langham, Westat AlisonLangham@westat.com Presentation to the SIGnetwork SPDG Directors’ Session April 16, 2010 Project Funded by: Office of Special Education Programs U.S. Department of Education AlisonLangham@westat.com 1

Presentation Outline Overview of the SIG Program Overview of Westat’s Evaluation of the SIG Program Brief explanation of the Leadership Study Results from the Leadership Study Summary and implications AlisonLangham@westat.com

Background on the State Improvement (SIG) Program IDEA ‘97 Joint effort of the feds and states to improve results for children with disabilities Money to states on a competitive basis Professional development, information dissemination, TA…SYSTEMIC CHANGE AlisonLangham@westat.com

Background on the SIG Program Evaluation Commissioned by OSEP in 2000 Five-year cooperative agreement, that continued for six years Formative Focused on the overall SIG Program, not individual states—but individual SIG projects were the unit of analysis Not intended to provide technical assistance to projects Instead, intended to describe what states were doing to inform OSEP, and the states themselves AlisonLangham@westat.com

Background on the SIG Program Evaluation Premises of the SIG Program that guided our evaluation Systems change is necessary to effect significant statewide improvements Improving systems requires comprehensive planning that involves multiple individuals, agencies, and institutions SEAs need to play a leadership role in engaging partners and bringing about the systems changes AlisonLangham@westat.com

Background on the SIG Program Evaluation Multiple evaluation activities Logic Models Cross-state comparisons Systemic Evaluation Inquiry Model Model of Theory of Systemic Change Outcomes Study Administrative Leadership Study AlisonLangham@westat.com

Administrative Leadership Study: Framework AlisonLangham@westat.com

Administrative Leadership Study: Methodology Nine states Iterative data collection and analysis Telephone interviews and document reviews Qualitative coding AlisonLangham@westat.com

Administrative Leadership Study: Results Influence Persuasion Guidance Authority Use of position or title Control of funds Accountability Setting expectations Establishing standards of performance AlisonLangham@westat.com

Administrative Leadership Study: Results Influence Strategies Strategy A: Implementing professional development (PD) as a systemic tool for change Strategy B: Communicating a vision of change Strategy C: Creating local buy-in Strategy D: Integrating PD efforts with general education reform Strategy E: Using collaborative working partnerships that promote joint learning AlisonLangham@westat.com

Administrative Leadership Study: Results   Administrative Leadership Strategies Influence TOTAL Cohort State A B C D E FY1999 a ü 4 b 2 c d e 5 f 3 FY2000 g h FY2001 i 6 7 9 AlisonLangham@westat.com

Administrative Leadership Study: Results Authority Strategies Strategy F: Asserting delegated authority Strategy G: Setting direction Strategy H: Justifying actions by evoking a higher authority Strategy I: Providing support for partners and subgrantees who buy-in AlisonLangham@westat.com

Administrative Leadership Study: Results Cohort State Administrative Leadership Strategies   Authority TOTAL F G H I FY1999 a ü 3 b 2 c 1 d 4 e f FY2000 g h FY2001 i 8 5 AlisonLangham@westat.com

Administrative Leadership Study: Results Accountability Strategies Strategy J: Standardizing the evaluation methodology Strategy K: Prioritizing the tasks associated with accountability Strategy L: Using accountability data to justify changes in project AlisonLangham@westat.com

Administrative Leadership Study: Results Cohort State Administrative Leadership Strategies   Accountability TOTAL J K L FY1999 a ü 3 b c d 2 e f 1 FY2000 g h FY2001 i 6 5 AlisonLangham@westat.com

Administrative Leadership Study: Summary and Implications Administrative Leadership Study Findings: Every SIG director used multiple leadership strategies SIG directors used influence strategies most and accountability strategies least Awareness of self, Project/Program, current events and culture influenced which strategies SIG directors used AlisonLangham@westat.com

Administrative Leadership Study: Summary and Implications Levels of Awareness AlisonLangham@westat.com

Administrative Leadership Study: Summary and Implications Most important SIG Evaluation finding: strong leadership coincided with strong SIG project outcomes Implication: leadership will matter in SPDG projects, too AlisonLangham@westat.com