1. Copyright © 2011 American Institutes for Research All rights reserved. Pilot Implementation Data: What We’ve Learned So Far Gretchen Weber Director of Educator Effectiveness Initiatives January 24, 2012

2. 2 Teacher Evaluation Hopes and Worries: September 2011 72% of teachers, 83.9% of principals, and 100% of administrators were moderately to very excited about the new system. The top aspects of excitement:  Professional growth/development  Discussions with administrators  Clearly defined, evidence-based standards 23.9% of teachers, 47.6% of principals, and 50% of administrators were moderately to very worried about the new system. The top worries:  Time  Fidelity  Buy-in

3. 3 Principal Evaluation Hopes and Worries: September 2011 75% of principals and 85.7% of administrators were moderately to very excited about the new system. Top aspects of excitement:  Professional growth/development  Clearly defined, evidence-based standards 26.9% of principals and 38.5% of administrators were moderately to very worried about the new system. Top worry:  Time

4. 4 TPEP Case Studies: Methodology Self-Report Data  Professional Development Resources Surveys  Teachers  Principals  Administrators Interviews  Teachers  Principals  Pilot leads  Union/association representative

5. 5 TPEP Case Studies: Constructs The new evaluation framework and goals Performance measures Professional development Communication Time and resources for implementation District and school culture changes

6. 6 TPEP Case Studies: Cross-Case Analysis Each of the site-specific case study reports was reviewed. Major themes across sites were recorded. Survey responses from teachers, principals, and administrators across sites were produced to indicate areas of strength and weakness. The report represents the total sample.

7. 7 TPEP Case Studies: Overall Findings on Teacher Evaluation The vast majority of teacher respondents in the pilot reported understanding the new system—its goals, measures, and how to achieve a high rating. They reported that the system is useful and helpful to professional growth. They see the new teacher evaluation system as better than the previous system in helping them improve their practice. Simultaneously teacher respondents raised concerns about the new system, particularly the time involved in evaluation activities but also communication with non-pilot teachers and the extent to which the system can account for differences in teachers’ content areas or student populations.

8. 8 TPEP Case Studies: Overall Findings on Principal Evaluation The development of principal evaluation systems is at various stages in the pilots. While interviewees in a couple districts described a well-specified principal evaluation system, many interviewees in other districts suggest that the development of the system is just beginning. The vast majority of principals report understanding the goals and measures of the new principal evaluation system. A strong majority report that it is better than the previous system in terms of helping to improve their practice. Similarly to the new teacher evaluation system, principals raised concerns about the amount of time involved and communication with non-pilot principals.

9. 9 TPEP Case Studies: General Findings The new evaluation system takes more time and money to understand and implement than the previous one.  Over 760 total session hours have been spent formally training educators on the new system thus far (through September 30, 2011).  64.4% of teachers reported spending a much greater amount of time on evaluation activities than previously.  65.8% of principals spent more than 10 hours on evaluation activities per teacher in the new evaluation system thus far.  In total, 57.9% of administrators reported spending 21 or more hours on evaluation activities for principals.  Only 47.8% of administrators believe there are adequate monetary resources to support the implementation of the new evaluation systems.

10. 10 TPEP Case Studies: General Findings The investment may be worth it. The practice of educators, already of high quality in Washington, appears to be improving even more.  The measures being used in the new teacher evaluation system provide feedback for improving instructional practice. More than 88% of teachers and principals agree.  The system supports professional growth. 75.4% of teachers consult the instructional framework to inform their professional practice. 45.8% of principals who report having a leadership framework consult the framework to inform their practice.  Over 84% of teachers and principals agree that the measures in the teacher evaluation system are fair, incorporate the most important aspects, and provide a comprehensive picture of teacher performance.  84.7% of principals indicate that their system provides useful feedback to improve their practice as a leader.

11. 11 TPEP Case Studies: General Findings Over the long-term the investment should continue to pay off. A shift in the culture across schools and districts appears to be happening and the work of the pilots is just the start.  74.6% of teachers agree that the teacher evaluation system is more objective than before.  Almost 90% of teachers and principals indicate that that the new systems specifies how to improve, identifies areas of strength, and encourages better teaching.  70.8% of principals and 90.9% of administrators agree that the new principal evaluation system is better than the old one to improve leadership practice.  78.3% of principals and 100.0% of administrators report more productive conversations due to new principal evaluation system.

12. 12 TPEP Case Studies: Next Steps Spring data collection  Self-report data (time and resources)  Surveys  Document review Final case studies and cross-case analysis

13. 13 Gretchen Weber P: 630-649-6511 E-Mail: gweber@air.org 1120 East Diehl Road, Suite 200 Naperville, IL 60563-1486 General Information: 800-356-2735 Website: www.air.org