1. 2010 OSEP Leadership Mega Conference Collaboration to Achieve Success from Cradle to Career Evaluation-Zen: The Pathway to Results The Story of North Carolina Nancy Johnson, Ed.D., State Performance Plan Consultant Bobbie Grammer, Consultant, Policy, Monitoring, Audit Section Strand Presentation # 4

2. The Story of North Carolina In the beginning, there was a State known as Confusion. The State thought that it was important to report every activity and initiative it implemented. And so the State did!

3. The Story of North Carolina In the beginning… State Performance Plan/Annual Performance Reports - developed by subcommittees for different indicators Different sections of the reports pieced together Included a multitude of improvement activities

4. The Story of North Carolina And then… Refined data collection processes and data analyses Recognized need for more focused improvement activities Began conducting monthly half-day SPP/APR work sessions with entire EC Division SPP/APR Coordinator & lead Monitoring Consultant – 2007-08

5. The Story of North Carolina So one year, the State said no more! Staff rallied around and piloted an abbreviated review checklist with some of the indicators. It worked so well, the State then piloted it with its LEAs and used it with more indicators.

6. The Story of North Carolina NC’s SPP/APR Improvement Activity Review Checklist (page 1) Adapted from NCRRC’s “SPP/APR Improvement Activity Review Form” Staff review/revision of improvement activities from selected indicators Eliminated activities such as routine processes and revised activities that were too broad Through annual regional meetings, trained LEA staff to use the checklist with their Continuous Improvement Performance Plans (CIPP)

7. The Story of North Carolina However, the State began to notice that certain initiatives seemed to impact several indicators, and so the review checklist was expanded to focus on a few key State initiatives.

8. The Story of North Carolina NC’s SPP/APR Improvement Activity Review Checklist (page 2) Staff review/revision of key improvement activities that cross more than one indicator Continued review/revision process of other selected indicators, using page 1 of the review checklist Continued annual regional meetings, to assist LEA staff with their Continuous Improvement Performance Plans (CIPP) and use of the review checklist

9. The Story of North Carolina Then one day, the State was invited by the Mid-South Regional Resource Center and the Appalachia Regional Comprehensive Center to participate in a Systems Improvement Regional Forum and learned about using a logic model for evaluating improvement activities. So…

10. The Story of North Carolina … this year the State is beginning to use the logic model to evaluate one of its key initiatives, and if all goes well, plans to use it to evaluate a second key initiative.

11. The Story of North Carolina Use of A Logic Model for Evaluating Improvement Activities Selected one of our key improvement activities – Conducting focused monitoring around graduation, drop-outs, transition, and post-school outcomes for students with disabilities in targeted LEAs Data-rich Time/effort worth the improvement realized Knowing vs. believing it’s making a difference Across several indicators

12. The Story of North Carolina Use of A Logic Model for Evaluating Improvement Activities Begins with a description of improvement activity A logic model includes inputs, processes, outputs, & outcomes Builds an evaluation work plan around the problem, root causes, evaluation questions, data/information to answer the questions, and fidelity of implementation

13. The Story of North Carolina All of this challenging and valuable work takes staff time and effort, but the State is no longer feeling confused. Paramount is the implementation of key improvement activities that we know are effective in helping to improve outcomes for students with disabilities in North Carolina.

14. The Story of North Carolina The End