1. MEASURING TEACHING PRACTICE Tony Milanowski & Allan Odden SMHC District Reform Network March 2009

2. Session Overview 1.Our Vision 2.Summary of Our Review of Seven Assessment Approaches 3.Your Assessment Needs & Issues 4.How Can We Move Forward?

3. Our Vision District Vision of Effective Instruction Embodied in Teaching Competency Model Methods to Measure Teaching Competency Anchors HRM programs to acquire, develop, motivate, & retain teacher human capital

4. Teaching Assessment Links Competencies to HC Management Systems

5. But with value-added, why bother? 60-69% of teachers do not teach subjects with testing that allows value-added estimation There are issues with the validity of value- added estimates which are often not recognized –High sampling error  hard to distinguish any but top and bottom teachers –Low stability of estimates across years (Goldhaber & Hanson, 2008; McCaffery et al, 2008; Koedel & Betts, 2007)

6. But with value-added, why bother? Behavioral science research suggests that results-only feedback is not that effective for performance improvement –What do you tell teachers to do in order to improve their value-added? –Like teaching golf or tennis using only scores How do you know your strategies are being applied?

7. Effective Measurement Requirements Capture Your Vision of Instruction –Reflect key drivers of student achievement –Reflect district strategies for improving student achievement Reliability & Validity Efficiency Acceptance by Teachers & Administrators Adaptability to Varied HCM Uses

8. Adapting to Multiple HRM Uses Doesn’t mean that the same assessment process is applied for every use But the same competency model should underlie all the uses Different methods of data collection about the competencies could be used (more later)

9. Seven Assessment Approaches Classroom Assessment Scoring System (CLASS) Formative Assessment System Continuum of Teacher Development - New Teacher Center at University of California Santa Cruz Framework for Teaching & Framework adaption by the Cincinnati Public Schools National Board for Professional Teaching Standards Certification Assessments Teacher Advancement Program (National Institute for Excellence in Teaching) Performance Assessment for California Teachers (PACT) PRAXIS III © (Educational Testing Service)

10. Preliminary Conclusion 1 We have several potentially appropriate competency models to choose from –Underlying competency models have substantial similarities Engagement HOTS Differentiation of instruction Use of formative assessment –But specific differences in emphasis are worth noting PACT & National Board assessments are most concerned with content & pedagogical content knowledge The Formative Assessment System-NTC is concerned with engagement dimensions CLASS focuses on social interactions In Cincinnati, the emphasis is on standards

11. Preliminary Conclusion 2 Any model will need to be adapted by incorporating key elements of district strategy (Cincinnati, TAP) –Student standards –Curriculum alignment –Grade or subject teaming –Use of particular literacy or math approach

12. Preliminary Conclusion 3 Convergence on what is needed for reliable assessment –Clear standards & multi-level behavior-based rubrics –Assessor training –Accountability for accuracy The technology to reliably assess teaching exists –NB, PACT, CLASS, Cincinnati all can produce reliable ratings

13. Preliminary Conclusion 4 There is promising technology to improve the efficiency of teaching assessments –Videos & written reflections as used in NB & PACT can allow assessment outside of the school year –MyTeachingPartner based on CLASS shows how efficient formative assessment can work

14. Preliminary Conclusion 5 Some promising evidence on relationship to student achievement for: –PACT predecessor (Connecticut BEST) –National Board –CLASS Pre-K –Cincinnati Framework adaptation

15. Preliminary Conclusion 6 One assessment method is not going to fit all HRM purposes –Hard to get good information on content & pedagogical content knowledge from an observation –Need to also consider walk throughs –Would need to develop observational protocols for NB or PACT models for some uses –The point is to base different data collection methods on the same competency model

16. Assessment Processes & Purposes SelectionInduction Tenure Decision Perf. Mgmt PD Compen- sation Formal Observations XXX Informal Observations XXX Walk Throughs XXX Performance Assessment XXXX

17. Your Assessment Needs & Issues 1.What does your district do to assess teaching? 2.Are you satisfied with your current practice? 3.What areas need improvement? 4.How do you know you are measuring the right stuff? 5.What are the barriers to teacher & administrator acceptance? 6.How important are state teaching standards & licensing/relicensing requirements?

18. How Can We Move Forward? Our Tentative Roadmap Does this make sense to you?

19. How Can We Move Forward? Using roadmap, work collaboratively with 3-4 districts to develop their existing teaching assessment approach or adapt another approach. Districts would then pilot over the next two school years for teacher development, evaluation and/or tenure decision making. We would help conduct a formative evaluation of the implementation and effectiveness of the approach, including an assessment of the relationship between practice measurements and student achievement.