1. Presentation to the SIGnetwork SPDG Directors’ Session
Westat’s Evaluation of the SIG Program: Findings from the Leadership Study
Alison Langham, Westat
Presentation to the SIGnetwork SPDG Directors’ Session
April 16, 2010
Project Funded by:
Office of Special Education Programs
U.S. Department of Education 1

2. Presentation Outline Overview of the SIG Program
Overview of Westat’s Evaluation of the SIG Program
Brief explanation of the Leadership Study
Results from the Leadership Study
Summary and implications

3. Background on the State Improvement (SIG) Program
IDEA ‘97
Joint effort of the feds and states to improve results for children with disabilities
Money to states on a competitive basis
Professional development, information dissemination, TA…SYSTEMIC CHANGE

4. Background on the SIG Program Evaluation
Commissioned by OSEP in 2000
Five-year cooperative agreement, that continued for six years
Formative
Focused on the overall SIG Program, not individual states—but individual SIG projects were the unit of analysis
Not intended to provide technical assistance to projects
Instead, intended to describe what states were doing to inform OSEP, and the states themselves

5. Background on the SIG Program Evaluation
Premises of the SIG Program that guided our evaluation
Systems change is necessary to effect significant statewide improvements
Improving systems requires comprehensive planning that involves multiple individuals, agencies, and institutions
SEAs need to play a leadership role in engaging partners and bringing about the systems changes

6. Background on the SIG Program Evaluation
Multiple evaluation activities
Logic Models
Cross-state comparisons
Systemic Evaluation Inquiry Model
Model of Theory of Systemic Change
Outcomes Study
Administrative Leadership Study

7. Administrative Leadership Study: Framework

8. Administrative Leadership Study: Methodology
Nine states
Iterative data collection and analysis
Telephone interviews and document reviews
Qualitative coding

9. Administrative Leadership Study: Results
Influence
Persuasion
Guidance
Authority
Use of position or title
Control of funds
Accountability
Setting expectations
Establishing standards of performance

10. Administrative Leadership Study: Results
Influence Strategies
Strategy A: Implementing professional development (PD) as a systemic tool for change
Strategy B: Communicating a vision of change
Strategy C: Creating local buy-in
Strategy D: Integrating PD efforts with general education reform
Strategy E: Using collaborative working partnerships that promote joint learning

11. Administrative Leadership Study: Results
Administrative Leadership Strategies
Influence
TOTAL
Cohort
State A B C D E
FY1999 a ü 4 b 2 c d e 5 f 3
FY2000 g h
FY2001 i 6 7 9

12. Administrative Leadership Study: Results
Authority Strategies
Strategy F: Asserting delegated authority
Strategy G: Setting direction
Strategy H: Justifying actions by evoking a higher authority
Strategy I: Providing support for partners and subgrantees who buy-in

13. Administrative Leadership Study: Results
Cohort
State
Administrative Leadership Strategies
Authority
TOTAL F G H I
FY1999 a ü 3 b 2 c 1 d 4 e f
FY2000 g h
FY2001 i 8 5

14. Administrative Leadership Study: Results
Accountability Strategies
Strategy J: Standardizing the evaluation methodology
Strategy K: Prioritizing the tasks associated with accountability
Strategy L: Using accountability data to justify changes in project

15. Administrative Leadership Study: Results
Cohort
State
Administrative Leadership Strategies
Accountability
TOTAL J K L
FY1999 a ü 3 b c d 2 e f 1
FY2000 g h
FY2001 i 6 5

16. Administrative Leadership Study: Summary and Implications
Administrative Leadership Study Findings:
Every SIG director used multiple leadership strategies
SIG directors used influence strategies most and accountability strategies least
Awareness of self, Project/Program, current events and culture influenced which strategies SIG directors used

17. Administrative Leadership Study: Summary and Implications
Levels of Awareness

18. Administrative Leadership Study: Summary and Implications
Most important SIG Evaluation finding: strong leadership coincided with strong SIG project outcomes
Implication: leadership will matter in SPDG projects, too