1. PROFESSIONAL DEVELOPMENT FOR EVALUATORS Monday, October 21 Leveraging Evaluation to Support School Priorities & Increase Effectiveness

2. Today’s Agenda Opening Contract Language Collecting Evidence: Observations Collecting Evidence: Artifacts Closing

3. Introductions  Angela Rubenstein, Office of Educator Effectiveness  Sam Varano, Assistant Principal at the Edison K-8  Sara Zrike, Teacher Leader at the Hurley K-8

4. Getting Started: Revising Goals Please consider the goals– are there any revisions you would suggest? Characteristics of high-quality goals: SMART Aligned to school priorities High leverage Includes mid-year benchmark

5. Objectives of the training Objective 1: Evaluators will know how to implement the new system and how to get it done and done well technical piece of implementation evaluation best practices

6. Overall goals of the training Objective 2: Evaluators will leave with concrete, specific plans for implementation in their school, including how to: Use the evaluation system to further their school priorities Talk with teachers about the evaluation system

7. Overall goals of the training Objective 3: Evaluators will leave knowing the specific responsibilities of evaluators and teachers under the new evaluation system.

8. http://boston.schoolwires.net/oee

10. Resources

11. Implementation Stage of the Five-Step Cycle Self- Assessment Analysis, goal-setting, & plan development Implementation of the plan Formative Assessment/ Evaluation Summative Evaluation

12. Today’s Agenda Opening Contract Language Collecting Evidence: Observations Collecting Evidence: Artifacts Closing

13. Components of the Process: Evaluation of Goals & Standards Progress on Ratings on OVERALL 2 Goals 4 Standards RATING Student Learning Professional Practice I. Curriculum, Planning and Assessment* II. Teaching All Students* III. Family & Community Engagement IV. Professional Culture Exemplary Proficient Needs Improvement Unsatisfactory

14. Educator plan is determined by the performance rating and career stage

15. Timelines and Requirements Type of Educator Plan Self-Directed Growth Plan 1 school year* Directed Growth Plan Less than 1 school year Improvement Plan 30 calendar days to 1 school year Developing Educator Plan 1 school year Announced observations None required 1 1 Unannounced observations 1 2 2 if plan is less than 6 months 4 if plan is between 6 months and 1 year 4 Required Dates Oct. 1: Educator submits self-assessment & proposes 2 goals Nov. 1: Evaluator completes educator plans by approving goals & action steps May 15: Evaluator completes Summative Evaluation Report June 1: Evaluator meets with educators whose overall Summative Evaluation ratings are moved from Proficient or Exemplary to Needs Improvement or Unsatisfactory Dates established in educator plan. Oct. 1: Evaluator meets with 1 st year educators to assist with self-assessment and goal-setting. Other dates are same as above for 1-year Self- Directed Growth Plan.

16. Observation Requirements Announced Observations Unannounced Observations Advance notice to teachers Feedback in EDFS in 5 days Post conference Feedback in EDFS in 5 days

17. Contract Highlights Timeline & Observation Requirements  Dates for educators plans of 1 or 2-years  Set minimums for # of observations  Some plans require announced observations

18. Timelines  Self-Directed Plans of 1 year  By Oct1: Educator submits self-assessments & goals  By Nov 1: Evaluator approves goals and action steps  By Nov 15: Every educator observed  By May 15: Evaluator completes summative evaluation  By June 1: Evaluator meets with anyone moving down  Directed & Improvement Plans  Dates established by evaluator in educator plan  Developing Educator Plans  By Oct 1: Evaluator meets to assist with goal setting

19. Directed and Improvement Plans  Meet with educator within 10 days of assigning the plan to provide goals and action steps  Educators should self-assess prior to meeting  Suggested lengths:  Less than 1 year, but not too short to complete all the steps  Plan officially begins as soon as goals are approved  Evaluators should communicate timeline from the outset

20. Next year’s educator plan is determined by the performance rating and career stage

21. Today’s Agenda Opening Contract Language Collecting Evidence: Observations Collecting Evidence: Artifacts Closing

22. Evidence is collected throughout the cycle  Every educator is an active participant in an evaluation  Process promotes collaboration and continuous learning Self-Assessment Analysis, goal-setting & plan development Implementation of the plan Formative Assessment/Evaluation Summative Evaluation Continuous Learning

23. All Evidence is entered in EDFS Both educators and evaluators can upload artifacts Evaluators upload both observation evidence & feedback

24. Evidence should be collected on … Progress on Ratings onOVERALL (2) Goals (4) Standards RATING * Student Learning * Professional Practice - Curriculum, Planning and Assessment - Teaching All Students - Family & Community Engagement - Professional Culture - Exemplary - Proficient - Needs Improvement - Unsatisfactory

25. Evidence Collection Evidence should be explicitly related to specific standards & goals. When appropriate, evaluator provides feedback to educator using language from the rubric. Evaluator does not rate educator practice after each piece of evidence, but during the formative or summative evaluation. 25

26. Teacher Rubric At-A-Glance Standard I: Curriculum, Planning, and Assessment Standard II: Teaching All Students Standard III: Family and Community Engagement Standard IV: Professional Culture A. Curriculum and Planning Indicator 1.Subject Matter Knowledge 2.Child and Adolescent Development 3.Rigorous Standards-Based Design 4.Well-Structured Lessons A. Instruction Indicator 1.Quality of Effort and Work 2.Student Engagement 3.Meeting Diverse Needs A. Engagement Indicator 1.Parent/Family Engagement A. Reflection Indicator 1.Reflective Practice 2.Goal Setting B. Professional Growth Indicator 1.Professional Learning and Growth B. Assessment Indicator 1.Variety of Assessment Methods 2.Adjustments to Practice B. Assessment Indicator 1.Safe Learning Environment 2.Collaborative Learning Environment 3.Student Motivation B. Assessment Indicator 1.Learning Expectations 2.Curriculum Support C. Collaboration Indicator 1.Professional Collaboration C. Analysis Indicator 1.Analysis and Conclusions 2.Sharing Conclusions With Colleagues 3.Sharing Conclusions With Students C. Analysis Indicator 1.Respects Differences 2.Maintains Respectful Environment C. Analysis Indicator 1.Two-Way Communication 2.Culturally Proficient Communication D. Decision-making Indicator 1.Decision-making E. Shared Responsibility Indicator 1.Shared Responsibility D. Expectations Indicator 1.Clear Expectations 2.High Expectations 3.Access to Knowledge F. Professional Responsibility Indicator 1.Judgment 2.Reliability and Responsibility

27. Rubric Look-Fors

28. Example of school-wide “unpacking”

29. Evidence from Observations

30. Observations Turn & Talk: What have your experiences with observation been like?

31. Evidence vs. Feedback EvidenceFeedback Facts Aligned to Rubric Ex 1: Lesson plan submitted as artifact Ex 2: Ms. Y said, "Yes, I know. You are working hard. You can do this.” Claim & Interpretation Aligned to Rubric Ex 1: The lesson plan does not include a closing activity. How is summarizing incorporated into the lesson? Ex 2: Ms. Y's response show that she continues to foster a safe intellectual environment where students take academic risks.

32. What the Research Says... Evaluators need multiple opportunities and settings to observe and assess educator practice Multiple observations paired with timely feedback are a key part of a strong evaluation system 32

33. Observing Lessons … CAUTION! … We observe all the time, which means that it is easy to do, but hard to do objectively. Remember We tend to see what we want to see We have to be aware of and avoid bias, particularly personal idiosyncrasies We have to avoid preconceptions Observation should be based on agreed upon criteria

34. Observation Bias What is bias? What are some experiences you have had with bias? What are some examples of bias you have about teaching? What can we do to diminish the influence of bias in our work?

35. Principles of Observations Frequent Focused Varied Useful and Timely Feedback  face-to-face when possible

36. Strategies for Collecting Evidence Identify a focus ahead of time  Goals, specific Elements Record evidence, not judgment  Quotations, observed actions or movements by teacher and students, times, numbers, literal descriptors, etc… Be an efficient note-taker  Establish abbreviations, paraphrase When possible, meet with the teacher  Check in briefly to ask questions/give feedback 36

37. Examples Observations & Tools from Sara Observations & Tools from Sam

38. Sara’s tracking tool

39. Sam’s Tools

41. CHS approach to scheduling

42. Other forms of Observations: Team Feedback

43. Observations in the EDFS BPS developed an observation tool in EDFS Evidence and feedback should be entered in EDFS, and tagged to an appropriate area of the rubric Observations may only be used in formative or summative evaluations if the educator received feedback within 5 days of the observation

44. EDFS: Manager View

45. Observations: EVALUATOR View

49. Next version of the observation tool

51. Observations: EVALUATOR View

52. Observations: Educator &Evaluator View

53. Observing Practice Chris McCloud 7 th Grade Math Teacher

54. StrengthsNeeds Improving student performance in math for students who enter my class performing below grade level Indicators/Elements: Teaching classrooms with diverse needs (especially meeting the needs of student with an IEP) Indicators/Elements: Family outreach and communication Indicators/Elements: Additional support for implementing the revised MA Curriculum framework Indicators/Elements: Improving communication with families for whom English is a second language Indicators/Elements: Strengthen leadership skills Indicators/Elements: C. McCloud’s self-assessment against the rubric 54

55. Chris McCloud’s Goals: SL: Based on the fact that my 7 th grade ELL students averaged 30% on my pre-assessment, my goal is that they will average at least 80% on each of the six unit assessments that they will take before May 15. PP: In order to support my ELL students in averaging 80% on unit assessments, I will consistently identify and teach symbols, key terms and other math vocabulary, and use daily exit tickets that measure both vocabulary and conceptual understanding. I will measure my progress through student vocabulary notebooks and tracking exit ticket performance data.

56. Observation of Chris McCloud: Selective Scripting 1. Read over the two excerpted elements from the rubric (I-A-4. Well-Structured Lessons, II-A-2. Student Engagement) 2. Watch the video. 3. While you are watching the video, collect evidence that will allow you to rate Mr. McCloud on the two elements. Use the worksheet to selectively script evidence related to each of the elements.

57. How did this process of observation go for you? (We’ll talk about feedback for Chris in just a minute…)

58. Value of Feedback Targeted, Ongoing Feedback Improved Educator Practice Improved Student Achievement

59. Characteristics of Effective Feedback FOCUSED: feedback should focus on what was observed EVIDENCE-BASED: feedback should be grounded in evidence of practice ACTIONABLE: feedback should reinforce effective practice and identify areas for continued growth TIMELY: feedback should be provided shortly after the observation

60. Constructing Feedback Using your observation notes and the rubric: With a partner, discuss a critical point of feedback for Chris, and the next steps – what would you want to see next time you come to observe? Focus on feedback that is  Focused  Evidence-based, and  Actionable

61. Feedback conversations What has been your experience with feedback conversations? Volunteers for a roleplay?

62. Feedback conversations: Resources Beyond the Scoreboard, from Educational Leadership Talk About Teaching, by Charlotte Danielson Leverage Leadership, by Paul Bambrick-Santoyo Difficult Conversations, by Stone, Patton & Heen

63. Reflect What are the implications of what we just discussed for your practice? For your school community?

64. Today’s Agenda Opening Contract Language Collecting Evidence: Observations Collecting Evidence: Artifacts Closing

65. Collecting Evidence: What can you see? In the classroomOutside of the classroom In artifacts/ conversations Well-structured lessons Student engagement Classroom management Differentiated instruction A safe learning environment Professional collaboration Family and community engagement Reflection Results of data analysis Adjustments to practice

66. Collecting and Assessing Evidence - Artifacts Artifacts may include:  Tracking & analysis of student assessment data  Student work  Lesson plans Identify & chart possible sources of evidence in the standard you are assigned. What kind of documents would be useful for teachers upload?

67. Artifacts: Clarifying Expectations How many would you like people to submit? What kind of documents? How detailed of a rational would be useful?

68. Artifacts Guidance A suggested format for rationales: Step 1: Identify the standard, indicator, element, or goal that the artifact addresses. Step 2: Describe the artifact and identify the section that directly connects to the standard, indicator, element, or goal. Step 3: Highlight the artifact’s impact on student learning. Step 4: Specify the evidence of professional growth or proficiency the artifact provides.

69. Artifacts Guidance Look over the examples on p. 4-6 What is strong? Anything more you would add?

70. Mid Year Check Point

71. Artifacts: Examples Looking at these artifacts, what evidence do you have of the teacher’s practice?

72. Trio Activity Thinking about a specific team or educator, think about which artifacts you would suggest they upload. Consider what kinds of artifacts would show… - elements that cannot be observed? - evidence of growth in areas noted for improvement? - progress on goals? - performance on each standard?

73. Artifacts on EDFS: Educator & Evaluator View

74. Artifacts: Educator & Evaluator View

79. Artifact Tips Start an electronic folder for artifacts on your desktop Keep file names short (fewer than 32 characters) Upload videos somewhere else, and include a link Genius Scan – app for turning pictures into PDFs

80. Artifact Tips Cont’d All artifacts have to be uploaded to EDFS – even if the hard copy is not scanned, it has to be entered as evidence and a hard copy kept in a binder. You can comment on artifacts through EDFS, but the online system is not a substitute for face-to-face conversation.

81. Reflect What are the implications of what we just discussed for your practice? For your school community?

82. Today’s Agenda Opening Contract Language Collecting Evidence: Observations Collecting Evidence: Artifacts Closing

83. Next Week: Writing evaluations Managing Directed Growth & Improvement plans Breakouts:  Co-evaluation  Teacher evaluators  Your own evaluation

84. What progress have we made toward the objectives of the training? Objective 1: Evaluators will know how to implement the new system and how to get it done and done well technical piece of implementation evaluation best practices

85. What progress have we made toward the objectives of the training? Objective 2: Evaluators will leave with concrete, specific plans for implementation in their school, including how to: Use the evaluation system to further their school priorities Talk with teachers about the evaluation system

86. What progress have we made toward the objectives of the training? Objective 3: Evaluators will leave knowing the specific responsibilities of evaluators and teachers under the new evaluation system.

87. Resources, Support, Questions, and Feedback For more information, visit:  EDFS: http://eval.mybps.org/http://eval.mybps.org/  OEE website: http://boston.schoolwires.net/OEEhttp://boston.schoolwires.net/OEE Email questions, comments, and feedback to:  Bpsevaluation@boston.k12.ma.us Bpsevaluation@boston.k12.ma.us MA Department of Elementary and Secondary Evaluation (DESE) Evaluation Site:  http://www.doe.mass.edu/edeval/ http://www.doe.mass.edu/edeval/

88. http://boston.schoolwires.net/oee

89. Contact us if you have questions Sam Varano, Edison K-8, Sara Zrike, Hurley K-8 NetworkEvaluation Support TeamBPS email AEmily Qazilbasheqazilbash BKris Taylorktaylor CNicole Irelandnireland DAngela Rubensteinarubenstein EJared Joinerjjoiner FLeah Levinellevine2 High SchoolsChason Ishino & Theresa O’Neillcishino, toneill EDFS tech support Jenna Costinjcostin Data analysisJen Kozinjkozin Ross Wilson, Asst. Superintendent for Educator Effectiveness

90. Please complete the exit ticket, and we’ll see you next Monday!