1. The Design and Implementation of Educator Evaluation Systems, Variability of Systems and the Role of a Theory of Action Rhode Island Lisa Foehr Rhode Island Department of Education Thanos Patelis Center for Assessment Presentation at the National Conference on Student Assessment New Orleans, June 25, 2014

2. Overview Overview of the educator evaluation system Overview of the educator evaluation system Some results Some results Variability that exists Variability that exists Theory of Action Theory of Action Recap Recap 2 Foehr & Patelis – NCSA - 06/25/2014

3. Overview of the System 3 In 2012-2013, full implementation of educator evaluations. In 2012-2013, full implementation of educator evaluations. Multiple measures are utilized to develop final effectiveness ratings for each educator. Multiple measures are utilized to develop final effectiveness ratings for each educator. Foehr & Patelis – NCSA - 06/25/2014

4. Overview of the System 4 The evaluation system is embedded within a year-long process representing goal setting, observation, data collection, and reflection. The evaluation system is embedded within a year-long process representing goal setting, observation, data collection, and reflection. Foehr & Patelis – NCSA - 06/25/2014

5. Overview of the System 5 Professional Practice Rating involves observation using a rubric on eight components adapted from Domains 2 and 3 of the 2013 version of Danielson’s Framework for Teaching. Professional Practice Rating involves observation using a rubric on eight components adapted from Domains 2 and 3 of the 2013 version of Danielson’s Framework for Teaching. Foehr & Patelis – NCSA - 06/25/2014

6. Practice 6 Adapted from Domains 2 and 3 of the 2013 version of Charlotte Danielson’s Framework for Teaching

7. Foundations 7 Foehr & Patelis – NCSA - 06/25/2014

8. Overview of the System 8 Foehr & Patelis – NCSA - 06/25/2014 +

9. Results from 2012-2013 - Teachers Highly Effective = 47.2% Highly Effective = 47.2% Effective = 48.4% Effective = 48.4% Developing = 3.7% Developing = 3.7% Ineffective = 0.5% Ineffective = 0.5% 9 Foehr & Patelis – NCSA - 06/25/2014

10. Results - Teachers 10 Foehr & Patelis – NCSA - 06/25/2014

11. Results – Building Administrators 11 Foehr & Patelis – NCSA - 06/25/2014

12. Variability Rhode Island has four approved systems for teachers and two approved systems for building administrators: Rhode Island has four approved systems for teachers and two approved systems for building administrators: Teacher Evaluation Systems Teacher Evaluation Systems 1. Coventry Teacher Evaluation System Used by one LEA Used by one LEA 2. Learning Community Teacher Evaluation System 1. Used by a charter chool 3. RI Innovation Consortium Teacher Evaluation and Support System Used by 7 LEA’s Used by 7 LEA’s 4. RI Model Teacher Evaluation and Support System Used by 46 LEAs Used by 46 LEAs Building Administrator Evaluation Systems Building Administrator Evaluation Systems 1. Coventry Building Administrator Evaluation System Used by one LEA Used by one LEA 2. RI Model Building Administrator Evaluation System Used by 53 LEAs Used by 53 LEAs A new evaluation system for support professionals is being developed and gradually implemented in the 2013-14 school year in preparation for full implementation during the 2014-15 school year. A new evaluation system for support professionals is being developed and gradually implemented in the 2013-14 school year in preparation for full implementation during the 2014-15 school year. 12 Foehr & Patelis – NCSA - 06/25/2014

13. Variability (cont’d) LEAs have the opportunity to establish local policies and structures that will support the effective implementation of their educator evaluations. LEAs have the opportunity to establish local policies and structures that will support the effective implementation of their educator evaluations. Local variability can be found on the professional practice and foundations components. Local variability can be found on the professional practice and foundations components. There are state-wide guidelines for the development that includes training. But there is local variability with the number (above the minimum of two), the type of assessments used, how targets are set, etc. There are state-wide guidelines for the development that includes training. But there is local variability with the number (above the minimum of two), the type of assessments used, how targets are set, etc. Variability permissible if meet RI educator standards. Variability permissible if meet RI educator standards. To assist LEAs with this essential work, RIDE is producing several guidance documents to help LEAs align local policies and long-term plans both with their own visions for great teaching and school leadership, and with Rhode Island’s Educator Evaluation System Standards. To assist LEAs with this essential work, RIDE is producing several guidance documents to help LEAs align local policies and long-term plans both with their own visions for great teaching and school leadership, and with Rhode Island’s Educator Evaluation System Standards. 13 Foehr & Patelis – NCSA - 06/25/2014

14. Variability (cont’d) In the 2013-14 school year a Differentiated Evaluation Process for Teachers was introduced. In the 2013-14 school year a Differentiated Evaluation Process for Teachers was introduced. The Differentiated Evaluation Process for Teachers provides schools and districts with greater flexibility for classroom observations, evaluation conferences and Professional Growth Plans. The Differentiated Evaluation Process for Teachers provides schools and districts with greater flexibility for classroom observations, evaluation conferences and Professional Growth Plans. The Differentiated Evaluation Process for Teachers establishes new minimum requirements for classroom observations and evaluation conferences, but schools and districts may choose to conduct more than the minimum of either. The Differentiated Evaluation Process for Teachers establishes new minimum requirements for classroom observations and evaluation conferences, but schools and districts may choose to conduct more than the minimum of either. 14 Foehr & Patelis – NCSA - 06/25/2014

15. Role of Theory of Action Summarizes the goals and expectations of the system. Summarizes the goals and expectations of the system. Assists in the evolution and development of the systems. Assists in the evolution and development of the systems.  Offers a means to evaluate local policies and structures. Permits the development of a validation agenda. Permits the development of a validation agenda. 15 Foehr & Patelis – NCSA - 06/25/2014

16. Overarching Goals of RI’s Theory of Action Obtain measurable improvements in student learning and achievement. Obtain measurable improvements in student learning and achievement. Obtain measurable improvement in overall educator effectiveness. Obtain measurable improvement in overall educator effectiveness. Establish a fair, reliable, legally defensible educator effectiveness measure to support human capital decisions. Establish a fair, reliable, legally defensible educator effectiveness measure to support human capital decisions. Incrementally increase the number of new educators who reflect the range of knowledge, skills and competencies necessary to be effective. Incrementally increase the number of new educators who reflect the range of knowledge, skills and competencies necessary to be effective. Improve perceptions of educator quality and accountability across various stakeholder groups. Improve perceptions of educator quality and accountability across various stakeholder groups. Increase support provided to educators to improve effectiveness. Increase support provided to educators to improve effectiveness. 16 Foehr & Patelis – NCSA - 06/25/2014 Source: Hall, 2013.

17. Evaluation Criteria & Claims Three criteria: Three criteria: Professional Practice – a measure of effective instruction and classroom environment Professional Practice – a measure of effective instruction and classroom environment Professional Foundations – a measure of instructional planning and the contributions teacher make as members of their learning community Professional Foundations – a measure of instructional planning and the contributions teacher make as members of their learning community Student Learning – a measure of a teachers’ impact on student learning. Student Learning – a measure of a teachers’ impact on student learning. For each of the criteria produced statements for the following: For each of the criteria produced statements for the following: Description Description Data source Data source Scoring methodology Scoring methodology Performance categories Performance categories Rational for inclusion in the system Rational for inclusion in the system Associated claims Associated claims 17 Foehr & Patelis – NCSA - 06/25/2014 Source: Hall, 2013.

18. Evaluation Criteria & Claims 18 Foehr & Patelis – NCSA - 06/25/2014 Source: Hall, 2013. Sample for one criterion… Criterion: Professional Practice (PP) Claim: Teachers who score high on the PP display the instructional skills and classroom environment expected from teachers. Teachers who score low on the PP can benefit fro actionable feedback and targeted professional development.

19. Recap Educator evaluations systems in Rhode Island (now two years old) represent an effort to capture multiple measures involving professional practice, professional foundations and student learning. Educator evaluations systems in Rhode Island (now two years old) represent an effort to capture multiple measures involving professional practice, professional foundations and student learning. Variability within Rhode Island is possible, as long as systems conform to the RI Professional Teaching Standards, RI Educational Leadership Standards and the RI Code of Professional Responsibility and as expressed in the Rhode Island Theory of Action. Variability within Rhode Island is possible, as long as systems conform to the RI Professional Teaching Standards, RI Educational Leadership Standards and the RI Code of Professional Responsibility and as expressed in the Rhode Island Theory of Action. Ongoing feedback is captured and incorporated into revisions. Ongoing feedback is captured and incorporated into revisions. Results show comparability. Results show comparability. Efforts and processes represent major undertaking for the department. Efforts and processes represent major undertaking for the department. Theory of Action assists in decision making, particularly around understanding and accommodating variability. Theory of Action assists in decision making, particularly around understanding and accommodating variability. Theory of Action permits the development of a validation agenda to substantiate claims. Theory of Action permits the development of a validation agenda to substantiate claims. 19 Foehr & Patelis – NCSA - 06/25/2014

20. References Bell, C. A., Gitomer, D. H., McCaffrey, D., Hamre, B., Pianta, R., & Qi, Y. (2012). An argument approach to observation protocol validity. Educational Assessment, 17, 1-26. Danielson, C. (2013). The Framework for Teaching Evaluation Instrument. Princeton, NJ: The Danielson Group. Hall, E. (2013). Validation Framework for the Rhode Island Model Evaluation and Support System. Unpublished report produced for the Rhode Island Department of Education. 20 Foehr & Patelis – NCSA - 06/25/2014

21. Questions and Comments? Center for Assessment www.nciea.org Thanos Patelis tpatelis@nciea.org 21 Foehr & Patelis – NCSA - 06/25/2014