1. Washington State Teacher and Principal Evaluation Project Preparing and Applying Formative Multiple Measures of Performance Gathering Evidence 1 June 2013

2.  As you enter, jot down on a sticky note:  One disadvantage of having an observation only evaluation system  One advantage of the current evaluation system 2 Entry Task

3. 3  Introductions  Logistics  Agenda  Connecting  Learning  Implementing  Reflecting  Wrap-Up Welcome!

4.  Introduction to Educator Evaluation in Washington  Using Instructional and Leadership Frameworks in Educator Evaluation  Preparing and Applying Formative Multiple Measures of Performance: An Introduction to Self-Assessment, Goal Setting, and Criterion Scoring  Including Student Growth in Educator Evaluation  Conducting High-Quality Observations and Maximizing Rater Agreement  Providing High-Quality Feedback for Continuous Professional Growth and Development  Combining Multiple Measures Into a Summative Rating 4 Modules

5.  Pausing  Paraphrasing  Posing Questions  Putting Ideas on the Table  Providing Data  Paying Attention to Self and Others  Presuming Positive Intentions  What Else? 5 Session Norms

6. Connecting Builds community, prepares the team for learning, and links to prior knowledge, other modules, and current work 6

7. Participants will know and be able to:  Determine specific measures to support implementation of evaluation in their districts  Understand the evaluation cycle across a school year and how multiple measures fit in  Articulate ways to integrate multiple forms of evidence into educator evaluation to move beyond an observation- only evaluation system  Determine how to create criterion scores after gathering of evidence aligned with the Washington State Criteria 7 Overview of Intended Participant Outcomes for Part D

8. 8 Guidance Icon Key A capital “G!” indicates that the guidance represents Washington state law. A lower-case “g” indicates that the guidance represents research-based best practice but is not mandated by law.

9.  Part A described how using multiple measures strengthens a teacher evaluation system.  Part B discussed self-assessment.  Part C discussed goal setting.  The goal of criterion scoring is to begin the process of putting the pieces of a comprehensive teacher evaluation system together. 9 Connecting the Pieces of the Puzzle

10. 10 The Year-Long Evaluation Cycle 8 Criteria Frameworks + Student Growth Rubrics State- determined process Distinguished Proficient Basic Unsatisfactory District- determined process Distinguished Proficient Basic Unsatisfactory Observation Student Growth Evidence Step 1 Criteria aligned to instructional /leadership and student growth rubrics Professional goals (g) Instructional/leadership goals (g) Student growth goals (G!) Step 5 Summative Score (G!) Step 2 & 3 Select and collect evidence 2 observations (G!) Student growth (G!) Other evidence (g)

11. 11 Evidence Cover Page

12. 12 Evidence Collection Form

13.  Let’s return to his self-assessment on Criterion 5  Student strengths: Last year’s data demonstrate that students are well-behaved.  Student weaknesses: Students need to learn how to regulate their behavior; students need to learn how to work constructively with their peers.  Teacher strengths: Scored high on Criterion 5 last year; Students felt comfortable asking for help from the teacher last year  Teacher weaknesses: Did not participate in service-learning projects last year; decreased engagement in classroom as a result. 13 Digging for Evidence: Evidence for John Anderson

14.  Included are six artifacts that can be used as evidence for Criterion 5.  Evidence cover page has two things missing  Alignment to instructional framework dimensions in criterion 5  Evidence statement  With your partner, take a set of artifacts, align it to the framework in Criterion 5, and develop evidence statements 14 Digging for Evidence: Artifacts of John Anderson SetArtifactsHandout AStudent survey Copies of student awards BObservation notes Classroom layout CStudent behavior plan Rules and procedures document

15.  With your district team:  Do you think you need all six pieces of evidence to measure Criterion 5? Why or why not?  How would you score each piece of evidence?  How would you put all of this evidence together to create a score for Criterion 5? 15 Digging for Evidence Debrief

16.  Key messages for multiple measures  Need a variety of evidence  Not about collecting as much as you can; this is about quality, not quantity  Evidence should be based on naturally occurring documents/materials  Prioritize evidence collected based on goals and criteria. 16 Critical Decisions About Methods Criteria and Instructional Frameworks MeasuresAlignment

17. Learning Understand best practices in gathering evidence 17

18.  The primary goal of any system of teacher evaluation is to promote teacher and student learning.  Accurate teacher evaluation requires trained observers using a research- based instructional framework. Trained observers make accurate assessments of practice based on evidence.  The value of accurate assessments of practice is to shape the conversations that lead to improved practice.  Embedded in each instructional framework is a system for growth in teaching practice.  Reliability and validity of the instructional framework relies on implementation of the full framework rather than individual components/indicators.  It is imperative to remain in the formative mindset until the final summative rating is determined. 18 OSPI’s: Guiding Principles for Criterion Scoring

19.  Step 1. Collect, align, and draw conclusions about evidence based on framework alignment to criteria  Step 2. Determine score for each indicator based on preponderance of evidence  Step 3. Enter score in Criterion Scoring Matrix  Step 4. Determine criterion score Criterion Scoring Process 19 The Criterion Scoring Process 1. Collect, align, score evidence 2. Determine score, preponderance of evidence 3. Enter indicator score 4. Determine criterion score

20. Step 1: Collect, align, and draw conclusions about evidence based on framework alignment to criteria.  Organize the evidence that you have – both scores and more formative information  Consideration:  How did you score the evidence? Is it on the same scale as the criteria? If not how do you convert it?  If evidence aligns to multiple indicators, is the score the same on each indicator?  For example, if you administer a student survey that relates to multiple indicators, will each indicator get the same score, or a different score based on the dimension in the survey? 20 Learning Activity: Step through the process

21. 21 Step 1: Sorting through the evidence ObsStudent Behavior Plan Rules/ Procedure Doc Student Survey Student Awards Physical Layout Other Notes Indicator Score 2a Respect rapport 33Her class is polite to others in the hallway 2c Class procedure 23 2d Student behavior 321Students behave great throughout school 2e Physical Space 23 Criterion 5: Learning Environment (Danielson)

22. Step 2: Determine score for each indicator based on preponderance of evidence  Based on all of the evidence that you have, determine a preponderance score for each indicator.  Consideration:  Does some evidence have more weight than other evidence?  Does the evidence provide enough information to be as accurate and close to truth as possible? 22 Learning Activity: Step through the process

23. 23 Step 2: Creating indicator score ObsStudent Behavior Plan Rules/ Procedure Doc Student Survey Student Awards Physical Layout Other Notes Indicator Score 2a Respect rapport 33Her class is polite to others in the hallway 3 2c Class procedure 232 2d Student behavior 321Students behave great throughout school 2 2e Physical space 232 Criterion 5: Learning Environment (Danielson)

24. Step 3: Enter score in Criterion Scoring Matrix  Based on the indicator scores you determined in step 2, place the scores in the Criterion Scoring Matrix Criterion 5: Learning Environment (Danielson) 24 Learning Activity: Step through the process IndicatorUnsatisfactory 1 Basic 2 Proficient 3 Distinguished 4 Score 2a Respect rapport X 2c Class procedure X 2d Student behavior X 2e Physical space X

25. Step 4. Determine Criterion Score  Use the indicator scores in the Criterion Scoring Matrix to determine criterion score.  Analyze the combination of scores by each indicator within a criteria to determine the score.  Considerations:  Based on the preponderance scores for the instructional framework indicators, is there a clear criterion score (e.g., are the majority of indicators scored at one level)?  If there is not a clear criterion score based on the preponderance scores, use the guiding questions in the scoring document (next slide). 25 Learning Activity: Step through the process

26. Step 4. Determine Criterion Score Guiding Questions  What else do I need to see or consider to make a final decision – what is available to me?  What is the distribution of evidence over time?  Has there been demonstrated and consistent improvement? If there was growth, was the growth sustained?  What would be the tipping point? If I consistently saw “X,” I would feel confident that the performance is Basic. If I consistently saw “Y,” I would feel confident that performance is Proficient.  What is the essence of the indicator? The criterion? Go back and find the key words in the framework/rubric. What does the evidence tell you about the evaluatee’s performance and growth with regard to this essential aspect of the indicator/criterion?  Is this evaluatee more Basic than s/he is Proficient, or more Proficient than s/he is Basic in this indicator? What is the evidence based in the framework/rubric to support your decision? 26 Learning Activity: Step through the process

27.  Step 4. Determine Criterion Score – clear score. Criterion 5: Learning Environment (Danielson) 27 Learning Activity: Step through the process IndicatorUnsatisfactory 1 Basic 2 Proficient 3 Distinguished 4 Score 2a Respect rapport X 2 2c Class procedure X 2d Student behavior X 2e Physical space X

28.  Step 4. Determine Criterion Score – unclear score  If, for example, the indicator scores were as follows, what would you do? This is where the formative evidence is very helpful. Criterion 5: Learning Environment (Danielson) 28 Learning Activity: Step through the process IndicatorUnsatisfactory 1 Basic 2 Proficient 3 Distinguished 4 Score 2a Respect rapport X ? 2c Class procedure X 2d Student behavior X 2e Physical space X

29.  We are going to watch a short video, Austin’s Butterfly  http://vimeo.com/38247060 http://vimeo.com/38247060  After the video, discuss:  How did you feel after watching the video?  How does this video relate to the guiding principles of OSPI?  How does this relate to educator evaluations and the goal of evaluations? 29 Learning Activity 2: Austin’s Butterfly

30.  Use the Criterion Five evidence from the beginning activity to go through the 4-step criterion scoring process  Step 1. Collect, align, and draw conclusions about evidence based on framework alignment to criteria.  Step 2. Determine score for each indicator based on preponderance of evidence  Step 3. Enter score in Criterion Scoring Matrix  Step 4. Determine criterion score 30 Learning Activity 2: Try it yourself.

31.  With your district team:  What did you like about the criterion scoring process?  What did you find difficult about this process?  Share one outstanding question you have about the criterion scoring process. 31 Learning Debrief

32. Implementing Articulate the ways collecting evidence through artifacts can operationalize the revised evaluation system to improve teaching and learning 32

33.  Districts should think strategically about how to increase clarity around the teacher evaluation process and ensure the process is feasible for teachers and evaluators.  Some examples follow on the next few slides. 33 Increasing Clarity and Feasibility

34.  Create a table with the relevant evaluation measures listed for all Washington State Criteria – so everyone knows what measures should contribute to each rating  This also shows teachers where to focus their evidence collection and decreases the amount of evidence evaluators need to review  Use Handout 13A, B, or C depending on instructional framework 34 Increasing Clarity and Feasibility – Step 1 Washington State Criteria ObservationEvidence Review … 7X 8X

35.  Describe the entire evaluation process in detail, describing roles of teachers and evaluators.  Use Handout 14, which provides multiple questions to consider. 35 Increasing Clarity and Feasibility – Step 2

36.  Create a plan for using Evidence Review in teacher evaluation.  Use the “Plan for Implementing Evidence Review” Handout 15 of your packet to structure your conversation with your school team. 36 Identifying Tools and Processes for Gathering and Organizing Evidence

37.  Each team shares one decision that was made today to increase the clarity and feasibility of the teacher evaluation process. 37 Implementing Activities Debrief

38. Reflecting 38

39.  Whip Around: One significant “ah-ha moment” today  Take a few minutes and create at least two sticky notes for the Plus/Delta Chart on your way out.  Plus: What was a real “plus” of today’s session? What went well and should be repeated?  Delta: Where is there room for improvement and change? 39 Whip Around and Plus/Delta Debrief

40.  Homework Options  District: Start a district teacher evaluation guidebook that includes all of the nuts and bolts of the teacher evaluation process. Use the information you recorded on the “Plan for Implementing Evidence Review” handout as a starting place.  School or Teams: Identify times when teachers are already asked to collect artifacts – whether that means submitting lesson plans or turning in PLC meeting minutes.  Individual: Consider authentic evidence of practice that you create. Choose a couple and identify which criteria they provide evidence of. 40 What’s Next?

41. Thank you! INSERT PRESENTER’S NAME AND E-MAIL ADDRESS 41