1. PROFESSIONAL DEVELOPMENT FOR EVALUATORS Wednesday, July 31 Leveraging Performance Management to Support School Priorities

2. Getting Started: Revising Goals Please consider the goals – are there any revisions you would suggest? Characteristics of high-quality goals: SMART Aligned to school priorities High leverage Includes mid-year benchmark

3. Today’s Agenda Collecting Evidence  A Closer Look at Observations  A Closer Look at Artifacts Lunch Evaluations Breakouts  Planning for the work at your school  Managing Directed Growth & Improvement plans Closing

4. What progress have we made toward the goals of the training? Goal 1: Evaluators will know how to implement the new system (technical 5-step implementation) and how to get it done and done well (evaluation best practices).

5. What progress have we made toward the goals of the training? Goal 2: Evaluators will leave with concrete, specific plans for implementation in their school, including how to: Use the evaluation system to further their school priorities Talk with teachers about the evaluation system

6. What progress have we made toward the goals of the training? Goal 3: Evaluators will leave knowing the specific responsibilities of evaluators and teachers under the new evaluation system.

7. Evidence: Observation & Artifacts

8. Evidence is collected throughout the cycle  Every educator is an active participant in an evaluation  Process promotes collaboration and continuous learning Self-Assessment Analysis, goal-setting & plan development Implementation of the plan Formative Assessment/Evaluation Summative Evaluation Continuous Learning

9. All Evidence is entered in EDFS Both educators and evaluators can upload artifacts Evaluators upload both observation evidence & feedback

10. Evidence should be collected on … Progress on Ratings onOVERALL (2) Goals (4) Standards RATING * Student Learning * Professional Practice - Curriculum, Planning and Assessment - Teaching All Students - Family & Community Engagement - Professional Culture - Exemplary - Proficient - Needs Improvement - Unsatisfactory

11. Evidence Collection Evidence should be explicitly related to specific standards & goals. When appropriate, evaluator provides feedback to educator using language from the rubric. Evaluator does not rate educator practice after each piece of evidence, but during the formative or summative evaluation. 11

12. Evidence vs. Feedback EvidenceFeedback Facts Aligned to Rubric Ex 1: Lesson plan submitted as artifact Ex 2: Ms. Y said, "Yes, I know. You are working hard. You can do this.” Claim & Interpretation Aligned to Rubric Ex 1: The lesson plan does not include a closing activity. How is summarizing incorporated into the lesson? Ex 2: Ms. Y's response show that she continues to foster a safe intellectual environment where students take academic risks.

13. Evidence from Observations

14. Observations Turn & Talk: What have your experiences with observation been like?

15. What the Research Says... Evaluators need multiple opportunities and settings to observe and assess educator practice Multiple observations paired with timely feedback are a key part of a strong evaluation system 15

16. Observing Lessons … CAUTION! … We observe all the time, which means that it is easy to do, but hard to do objectively. Remember We tend to see what we want to see We have to be aware of and avoid bias, particularly personal idiosyncrasies We have to avoid preconceptions Observation should be based on agreed upon criteria

17. Observation Bias What is bias? What are some experiences you have had with bias? What are some examples of bias you have about teaching? What can we do to diminish the influence of bias in our work?

18. Principles of Brief Observations Frequent Focused Varied Useful and Timely Feedback

19. Strategies for Collecting Evidence Identify a focus ahead of time  Goals, specific Elements Record evidence, not judgment  Quotations, observed actions or movements by teacher and students, times, numbers, literal descriptors, etc… Be an efficient note-taker  Establish abbreviations, paraphrase When possible, meet with the teacher  Check in briefly to ask questions/give feedback 19

20. Examples Observations & Tools from Sara Observations & Tools from Mary

21. Other forms of Observations: Team Feedback

22. Sara’s tracking tool

23. CHS approach to scheduling

24. Systems for Tracking Observations

25. Observations in the EDFS BPS developed an observation tool in EDFS Evidence and feedback should be entered in EDFS, and tagged to an appropriate area of the rubric Observations may only be used in formative or summative evaluations if the educator received feedback within 5 days of the observation

26. Observations: EVALUATOR View

29. Next version of the observation tool

31. Observations: EVALUATOR View

32. Observations: Educator &Evaluator View

33. Observing Practice Chris McCloud 7 th Grade Math Teacher

34. StrengthsNeeds Improving student performance in math for students who enter my class performing below grade level Indicators/Elements: Teaching classrooms with diverse needs (especially meeting the needs of student with an IEP) Indicators/Elements: Family outreach and communication Indicators/Elements: Additional support for implementing the revised MA Curriculum framework Indicators/Elements: Improving communication with families for whom English is a second language Indicators/Elements: Strengthen leadership skills Indicators/Elements: C. McCloud’s self-assessment against the rubric 34

35. Chris McCloud’s Goals: SL: Based on the fact that my 7 th grade ELL students averaged 30% on my pre-assessment, my goal is that they will average at least 80% on each of the six unit assessments that they will take before May 15. PP: In order to support my ELL students in averaging 80% on unit assessments, I will consistently identify and teach symbols, key terms and other math vocabulary, and use daily exit tickets that measure both vocabulary and conceptual understanding. I will measure my progress through student vocabulary notebooks and tracking exit ticket performance data.

36. Observation of Chris McCloud: Selective Scripting 1. Read over the two excerpted elements from the rubric (I-A-4. Well-Structured Lessons, II-A-2. Student Engagement) 2. Watch the video. 3. While you are watching the video, collect evidence that will allow you to rate Mr. McCloud on the two elements. Use the worksheet to selectively script evidence related to each of the elements. 4. When you are finished, complete the worksheet and rate Mr. McCloud.

37. How did it go?

38. Value of Feedback Targeted, Ongoing Feedback Improved Educator Practice Improved Student Achievement

39. Characteristics of Effective Feedback FOCUSED: feedback should focus on what was observed EVIDENCE-BASED: feedback should be grounded in evidence of practice ACTIONABLE: feedback should reinforce effective practice and identify areas for continued growth TIMELY: feedback should be provided shortly after the observation

40. Constructing Feedback Chris McCloud: Using your observation notes and the rubric: Draft two points of feedback for Chris Focus on feedback that is  Focused  Evidence-based, and  Actionable

41. Feedback conversations What has been your experience with feedback conversations? Volunteers for a roleplay?

42. Feedback conversations: Resources Beyond the Scoreboard, from Educational Leadership Talk About Teaching, by Charlotte Danielson Leverage Leadership, by Paul Bambrick-Santoyo Difficult Conversations, by Stone, Patton & Heen

43. Reflect What are the implications of what we just discussed for your practice? For your school community?

44. Evidence from Artifacts

45. Collecting Evidence: What can you see? In the classroomOutside of the classroom In artifacts/ conversations Well-structured lessons Student engagement Classroom management Differentiated instruction A safe learning environment Professional collaboration Family and community engagement Reflection Results of data analysis Adjustments to practice

46. Collecting and Assessing Evidence - Artifacts Artifacts may include:  Tracking & analysis of student assessment data  Student work  Lesson plans Identify & chart possible sources of evidence in the standard you are assigned. What kind of documents would be useful for teachers upload?

47. Artifacts: Clarifying Expectations How many would you like people to submit? What kind of documents? How detailed of a rational would be useful?

48. Artifacts Guidance A suggested format for rationales: Step 1: Identify the standard, indicator, element, or goal that the artifact addresses. Step 2: Describe the artifact and identify the section that directly connects to the standard, indicator, element, or goal. Step 3: Highlight the artifact’s impact on student learning. Step 4: Specify the evidence of professional growth or proficiency the artifact provides.

49. Artifacts Guidance Look over the examples on p. 4-6 What is strong? Anything more you would add?

50. Zil’s SLG and Action Steps

51. Zils SLG Artifact

52. Mid Year Check Point

53. Artifacts: Examples Looking at these artifacts, what evidence do you have of the teacher’s practice?

54. Trio Activity Thinking about a specific team or educator, think about which artifacts you would suggest they upload. Consider what kinds of artifacts would show… - elements that cannot be observed? - evidence of growth in areas noted for improvement? - progress on goals? - performance on each standard?

55. Artifacts on EDFS: Educator & Evaluator View

56. Artifacts: Educator & Evaluator View

61. Artifact Tips Start an electronic folder for artifacts on your desktop Keep file names short (fewer than 32 characters) Upload videos somewhere else, and include a link Genius Scan – app for turning pictures into PDFs

62. Artifact Tips Cont’d All artifacts have to be uploaded to EDFS – even if the hard copy is not scanned, it has to be entered as evidence and a hard copy kept in a binder. You can comment on artifacts through EDFS, but the online system is not a substitute for face-to-face conversation.

63. Reflect What are the implications of what we just discussed for your practice? For your school community?

64. Evaluations

65. The basics Evaluations on EDFS Examples

66. What is a formative assessment? A mid-plan check-in  rating on progress towards each goal  rating on each standard A document that considers all evidence  observations to date  artifacts to date It may be used to change a plan  If there is a significant change in practice, however, this is not mandatory. The plan may continue until the summative.

67. What is a summative evaluation? A summary at the end of the plan  rating on progress towards each goal  rating on each standard A document that considers all evidence  the formative assessment  observations & artifacts since the formative  consistency or change in practice since the formative The overall rating determines the next plan

68. Not the “evaluation” of the past Evaluations are NOT write-ups of individual, formal observations.

69. Formative Assessment & Summative Evaluation Meetings  Upon request of educator or evaluator  Required for Summative if the overall rating is NI or Unsatisfactory  Recommended for all educators

70. Evaluations in EDFS

71. Formative Assessment Step 1 Step 2

72. Evaluations in EDFS

73. Evaluations in EDFS: Rating Goals

74. Evaluations in EDFS: Rating Standards

75. Evaluations in EDFS: Overall Rating

76. Evaluations in EDFS: Prescriptions Prescriptions for any standard in which an educator is rated NI or UNSAT.

77. Evaluations in EDFS: Comments Comments are optional for both the evaluator & the educator

78. Evaluations in EDFS: Release

79. Evaluations in EDFS: Sign-Off

80. Evaluation Examples Read one of the examples you have. What do you notice about how evidence is used in the write-up?

81. Rationales for Ratings Standard I. Curriculum, Planning, & Assessment Rating Exemplary, Proficient, Needs Improvement, Unsatisfactory Rationale Claim/Connection to rubric: Evidence 1: Evidence 2: Evidence 3: Rationale: What claim can you make? What is the evidence to support it (artifacts, observations)?

82. Reflect What are the implications of what we just discussed for your practice? For your school community?

83. Breakouts  Group 1: Planning for evaluation work at your school  Group 2: Working with teachers on improvement or directed growth plans

84. Planning the work at your school Mary’s document

85. Planning the work at your school Guiding Questions: What has your school done to support staff with the evaluation process? What additional supports do you think would be helpful? What are ways of structuring this support at the school? What timelines should guide the work? How can you and your team find the time to do this well?

86. Working with teachers rated NI or U You can change the plan at the time of the formative assessment (though a formative assessment does not HAVE to change the plan).

87. Timelines and Requirements Type of Educator Plan Self-Directed Growth Plan 1 school year* Directed Growth Plan Less than 1 school year Improvement Plan 30 calendar days to 1 school year Developing Educator Plan 1 school year Announced observations None required 1 1 Unannounced observations 1 2 2 if plan is less than 6 months 4 if plan is between 6 months and 1 year 4 Required Dates Oct. 1: Educator submits self-assessment & proposes 2 goals Nov. 1: Evaluator completes educator plans by approving goals & action steps May 15: Evaluator completes Summative Evaluation Report June 1: Evaluator meets with educators whose overall Summative Evaluation ratings are moved from Proficient or Exemplary to Needs Improvement or Unsatisfactory Dates established in educator plan. Oct. 1: Evaluator meets with 1 st year educators to assist with self-assessment and goal-setting. Other dates are same as above for 1-year Self- Directed Growth Plan.

88. Directed Growth and Improvement plans Considerations: Choosing length Using a calendar to calculate dates of formative assessment and summative evaluation Writing goals from prescriptions Developing the Action Plan, including supports and timelines Collection of evidence Supports: “Did I” sheets and timelines

89. http://educatoreffectiveness.weebly.com For more information, visit:  EDFS: http://eval.mybps.org/http://eval.mybps.org/  http://educatoreffectiveness.weebly.com http://educatoreffectiveness.weebly.com Email questions, comments and feedback to:  bpsevaluation@boston.k12.ma.us bpsevaluation@boston.k12.ma.us MA Department of Elementary and Secondary Education (DESE) Evaluation Site:  http://www.doe.mass.edu/edeval/ http://www.doe.mass.edu/edeval/

90. Office of Educator Effectiveness Ross Wilson, Assistant Superintendent for Educator Effectiveness Chason Ishino, Implementation Specialist Jared Joiner, Implementation Specialist Emily Kalejs Qazilbash, Implementation Specialist Angela Rubenstein, Implementation Specialist Kris Taylor, Implementation Specialist Jenna Costin, EDFS On-line System Coordinator Jen Kozin, Data Analyst Evaluator Training Facilitators: Mary Driscoll, Edison K-8 Sara Zrike, Hurley K-8

91. Contact us if you have questions Ross Wilson, Asst. Superintendent for Educator Effectiveness NetworkImplementation SpecialistsBPS email AEmily Qazilbasheqazilbash BNicole Irelandnireland CTBDeval@mybps.org DAngela Rubensteinarubenstein EJared Joinerjjoiner FKris Taylorktaylor High SchoolsChason Ishinocishino EDFS tech support Jenna Costinjcostin Data analysisJen Kozinjkozin

92. What progress have we made toward the goals of the training? Goal 1: Evaluators will know how to implement the new system (technical 5-step implementation) and how to get it done and done well (evaluation best practices).

93. What progress have we made toward the goals of the training? Goal 2: Evaluators will leave with concrete, specific plans for implementation in their school, including how to: Use the evaluation system to further their school priorities Talk with teachers about the evaluation system

94. What progress have we made toward the goals of the training? Goal 3: Evaluators will leave knowing the specific responsibilities of evaluators and teachers under the new evaluation system.

95. Closing Please complete the exit ticket. Thank you so much for participating, and please be in touch!