The Massachusetts Model System for Educator Evaluation Teachers 21 Dr. Patricia B Grenier I. Welcome (5 minutes) Slide 1 is the title slide. During this slide, welcome participants, introduce yourself, and ask participants to briefly do the same with their first name and role. If the group is large, ask for a quick identification of school teams by school. 1 Massachusetts Department of Elementary and Secondary Education 1

Objectives Assess Current Knowledge Reflect on Progress to Date Share Best Practices Strategize Gathering Evidence Formulate Observations and Feedback Formulating Next Steps This slide lists the modules and includes the outcomes for Module 5: Gathering Evidence. Explain: “Module 5 focuses on the collection and organization of evidence by Standard and Indicator and engages participants in thinking strategically about gathering high-quality artifacts to demonstrate performance.” Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

II. Connecting (25 minutes) Let’s Review II. Connecting (25 minutes) Connecting Content (5 minutes) Slide 4 is the title slide for the connecting section. The Connecting section will take approximately 25 minutes. This is intended to acquaint participants with the learning objectives as well as making a broader connection to the 5-Step Cycle of evaluation. Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Every educator is an active participant in the evaluation process. Every educator and evaluator collects evidence and assesses progress Explain: “Module 4 included information on creating S.M.A.R.T. goals and writing an Educator Plan. Once the Educator’s Plan is written and approved, we move to Step 3 of the 5-Step Cycle: Implementation of the Plan. Both the educator and evaluator begin collecting evidence and tracking progress towards goal attainment. “This step of the 5-Step Cycle is where educators will spend most of their time. Both Module 5 and 6 will focus on this step, with Module 5 focusing on gathering evidence and Module 6 addressing observations and feedback. “Together, Modules 5 and 6 will provide educators and evaluators with a complete understanding of the evidence collection process, including collecting evidence through artifacts and observations.” Collaboration and continuous learning are the focus. Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education 4

By documenting and analyzing practice from multiple angles and over an extended period of time, educators and evaluators develop a more complete picture of performance, which leads to a more accurate and informed evaluation.” Module 5 Facilitator’s Guide Massachusetts Department of Elementary and Secondary Education

Gathering Evidence – Key Points There are three types of evidence required by regulations. Evidence-based statements are critical and should connect to relevant Standards and Indicators. One must develop tools and processes for gathering and organizing evidence. Explain: “By the end of the session, you will be able to explain the three types of evidence that need to be collected and identify examples of each type. “You will create evidence-based statements from example artifacts and connect them to relevant Standards and Indicators from the Model System Teacher Rubric. “You will also be able to identify tools and processes for gathering and organizing evidence.” Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Artifacts in the Educator Plan Review the Educator Goal Setting and Educator Plan form for Tom Wilson. For your assigned action step, on a sticky note, write down two artifacts that could be collected to show progress toward the goal. Post your sticky note on the section of the chart paper with the same number as your action step. Connecting Activity: From Plan to Action (10 minutes) Purpose and intended outcomes: The purpose of this activity is for participants to realize that the first step in collecting quality evidence is to write a high-quality Educator Plan. Participants will look at the Educator Goal Setting and Educator Plan form of a sixth-grade science teacher named Tom Wilson. Tom Wilson is an example that will be used throughout the day. Participants then will identify possible artifacts that could be collected. Because most participants have not collected artifacts as part of their previous teacher evaluation system, this activity will give them an opportunity to think about what artifacts would authentically show evidence of progress toward goals. Facilitation notes: Participants are asked to write down on their handout two artifacts that would be collected to show progress toward the goal. At the end of the activity, participants will share their ideas with their table group. Activity detail: Explain: “A well-written Educator Plan can serve as a road map for educators as they collect evidence on progress toward their goals. We are going to take a look at the Educator Goal Setting and Educator Plan form of Tom Wilson, a sixth-grade science teacher. You can find this form labeled as Handout 1 in your handout packet. “Let me give you a few seconds to turn to that page. As you can see, Tom has identified two goals—an individual student learning goal that focuses on the achievement of his English language learners (ELLs), and a team professional practice goal that focuses on improving instruction of ELLs. “If you turn the page, you’ll see Tom’s Educator Plan form. His student learning goal action steps are on the first two pages, followed by action steps associated with his professional goal. Tom has identified five action steps, with relevant resources and benchmarks for each one. What he hasn’t done, however, is identify sources of evidence that can demonstrate each action. You are going to examine one action step individually and identify one or two sources of evidence that would demonstrate progress. To determine which action step, count off around your table by fives. Now, on your own, read your action step, related resources, and projected benchmarks, and fill in the blank titled ‘Evidence’ in the third column. What are one or two sources of evidence that might demonstrate this action step?” Common facilitation challenges and solutions: Participants may say that the task is simple because the process and outcome benchmarks are so explicit in the example. Tell participants that a well-written Educator Plan should make it simple to identify which artifacts need to be collected. Ask participants if the action plans in their school are as clear. Participants may note that the Educator Plan forms from their schools are not as detailed as this example. State that the first step in gathering evidence is to write a strong Educator Plan. Prompt participants to share this example with educators at their schools if they need to revise their plans to make them more detailed. Guiding questions: What artifacts could Tom Wilson use to show progress toward his goals? Connecting Wrap-Up/Debrief (10 minutes) Explain: “With your table group, go around and say what artifacts you identified that Tom could use to show progress toward his goals.” Give participants 3–5 minutes to share this information. Get the whole group’s attention and ask: “What did you learn about Educator Plans from this activity?” Allow participants to respond—it is likely that in many cases the Educator Plans are not as detailed as this example. Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Sources of Evidence for Summative Ratings Three categories of evidence must be collected for each educator: Multiple measures of student learning, growth, and achievement Judgments based on observations and artifacts of professional practice Additional evidence relevant to standards This includes evidence collected by the educator and shared with the evaluator relating to fulfilling Standard III: Family and Community Engagement and Standard IV: Professional Culture from the Model System Teacher Rubric Explain: “The new educator evaluation framework in Massachusetts requires three categories of evidence that should inform each educator’s evaluation. “These include: Multiple measures of student growth Judgments based on observations and artifacts of practice Additional sources that provide relevant information on an educator’s practice related to one or more performance standards “When we are talking about artifacts, that means ‘products of an educator’s work that demonstrate knowledge and skills of the educator.’ In other words, artifacts should never be documents manufactured for the evaluation.” Facilitator’s note: The definition of artifact included above comes from state regulations, 603 CMR 35.02. Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

What does this look like? Products of Practice Related to Standards Multiple Measures of Student Learning Other Evidence Related to Standards Artifacts Teacher-developed unit assessments Grade-level meeting notes Parent/teacher communication log PLC meeting notes Observations Notes/feedback from short, frequent observations (inside/outside classrooms) Notes and feedback from announced observations Student work (quizzes, homework, presentations, etc.) Portfolios Performance assessments (including arts, vocational, health and wellness) Interim assessments State or district assessments Student and staff feedback (2013–14 school year) Explain: “So what does this look like? Here you can see examples of evidence sources for each category. Under Products of Practice, artifacts could include anything from grade-level meeting notes to parent/teacher communication logs, depending on the kind of evidence an educator needs. Notes from short, unannounced observations as well as formal, announced observations also fall under this category. “The second category of evidence—Measures of Student Learning—could include a wide variety of measures, from student work to portfolios to performance assessments. Again, when thinking about how to track progress related to a student learning goal, the measures of student performance could take many forms. “And finally, the third category, Other Evidence, includes evidence from student feedback and staff feedback surveys, which aren’t required until the 2013–14 school year. “This may feel overwhelming or look like a lot to do. It’s important to keep one thing in mind. First, no single artifact or measure should be produced solely for the purpose of your evaluation—it should represent practices or activities that you already do in your daily work.” Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Implementation Responsibility Educator responsibilities: Identifying, collecting, and organizing artifacts/evidence related to goal progress Documenting action steps completed Collecting and submitting common artifacts Collecting and submitting evidence related to Standards III and IV Evaluator responsibilities: Making resources and supports available Identifying common artifacts/evidence Observing practice and providing regular and specific feedback on performance Monitoring progress—including midpoint check-ins Organizing and analyzing evidence over time Explain: “Depending on the type of evidence—whether it’s an artifact from a classroom lesson, student data, or notes from an observation—the educator or the evaluator will be responsible for its collection and organization. For the most part, the educator will collect most of the evidence over the course of the year, and the evaluator will be primarily responsible for organizing and analyzing evidence. “Here you can see the division of responsibilities for identifying, collecting, and organizing evidence. Educators are responsible for identifying, collecting, and organizing artifacts and other sources of evidence that demonstrate progress related to their goals and for providing evidence that action steps from their Educator Plan have been completed. Educators also need to collect examples of any common artifacts that all educators might be asked to collect. You will have some time at the end of this session to think about what common artifacts you might ask all educators to submit at your school. Let’s take an example. If you were part of a high school participating in the Wraparound Zone Initiative, you might have every teacher submit a lesson or unit plan that demonstrates their use of local civic, educational, or business partnerships to link the curriculum to the world beyond high school. The identification of common artifacts can help cultivate a shared sense of responsibility throughout a school while simultaneously moving the school forward in the same direction. Finally, educators are responsible for collecting evidence showing how they are progressing toward Standards III and IV, which tend to be less ‘observable’ than Standards I and II. Evaluators are responsible for letting educators know what resources and supports are available. They also need to identify common artifacts that all educators need to submit. Evaluators also need to observe practice on a regular basis, provide timely and specific feedback on performance, and monitor educator progress. At the conclusion of an evaluation cycle, evaluators review and analyze the body of evidence as it relates to goal progress and performance within the four standards of practice.” Transition: “Identifying, gathering, and submitting evidence from multiple sources is a critical component of the new evaluation process and a marked departure from previous evaluation systems that typically relied on single observations for an entire rating. By documenting and analyzing practice from multiple angles and over an extended period of time, educators and evaluators develop a more complete picture of performance, which leads to a more accurate and informed evaluation.” Facilitation note: Learning Activity 1: Evidence or Judgment? (10 minutes) Purpose and intended outcomes: Both educators and evaluators will be responsible for summarizing artifacts into evidence statements. The purpose of this activity is for participants to have practice identifying facts and opinions about evidence and revising opinion statements into more factual ones. Opinion statements often appear in evaluation feedback, and this activity will help participants avoid those kinds of statements. Activity detail: The facilitator will need three pieces of chart paper as participants share their information—one should be labeled Facts, one labeled Opinions, and one labeled Revised Statements on which to revise the opinion statements. Writing down responses will help ensure that participants are able to check their responses and understand how to change an opinion statement into one that is more factual. Participants will need Handout 2. Make sure there are enough highlighters on the tables so that each participant can have one. Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Evidence or Judgment? Examine Handout 2. Highlight factual statements. Underline judgment statements or statements not based on evidence. Activity detail: Explain: “As we saw in the list of implementation responsibilities, both educators and evaluators are responsible for collecting evidence of practice, but it’s what you do with that evidence and how you use it as a source of feedback that gives it meaning and value. Both educators and evaluators will collect artifacts and summarize those artifacts by creating factual evidence statements. Often, it is easy to let opinion or non-evidence-based statements sneak into feedback. This activity will serve as a good practice and reminder for how to create the most effective fact-based feedback. The document we are going to examine is a set of notes from Tom Wilson’s principal’s observation of a lesson Tom delivered. Handout 2 in your packet is the set of observation notes. Start with Handout 2. Read the statements recorded. Highlight factual statements that the teacher made, and underline statements that are opinions or do not provide evidence. You will have about 5 minutes to identify if each statement is fact or judgment.” After 5 minutes, get the group’s attention. Common facilitation challenges and solutions: Participants may mistakenly identify an opinion as a fact. Ask participants what evidence is provided in the artifact to show that the statement is true. If no evidence is provided, then it is a statement without evidence and should be underlined. Participants may question whether all statements in observation notes need to be factual or evidence based. The Research for Better Teaching (Jon Saphier) model is a common approach to observation in MA. This approach is highly judgment based and encourages observers to record “balance of claims, evidence, impacts and judgments” (p. 5). However, because ratings are not given to participants after each observation, it is important for the evaluator to collect only evidence-based statements to be able to accurately aggregate the balance of evidence. Guiding questions: What are some factual statements in these handouts? What are some opinion statements? What is one way each opinion statement could be revised into a factual statement? Learning Wrap-Up/Debrief 1 (10 minutes) Ask participants: “So what are some factual statements that you identified?” As participants share, record their responses on a sheet of chart paper labeled Facts. After some responses, ask: “What are some opinion statements that you found, or statements that are not based on recorded evidence?” As participants share, record their responses on a sheet of chart paper labeled Opinions. After recording the factual statements and opinion statements, ask participants how the opinion statement could be modified into a factual statement. Record their improvements on a sheet of chart paper labeled Revised Statements. An example is included below. Opinions: Application, analysis, synthesis, evaluation = yes! —yay! Revised Statements: Students were asked to analyze the differences between heterotrophs and autotrophs. Explain: “A quick show of hands—How many of you have seen observation feedback or an observation form that included opinion statements during your career in education? If the evaluator had collected only opinion statements, it would be very difficult for him or her to come to a defensible rating of that teacher at the formative or summative evaluation. So it is extremely important for evaluators to make sure that they are collecting factual statements based on evidence.” Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Artifact Cover Page Learning Content 2 (15 minutes) Explain: “In the last activity, you used one of the tools ESE developed to help educators and evaluators keep track of observational evidence: the Observation Evidence Collection form. The Observation Evidence Collection tool provides evaluators with a place to record factual notes during an observation and identify evidence of practice based on these notes that relate to specific goals or Standards. There is also a place for evaluators to construct targeted feedback based on what they observed. “The Artifact Cover Page is another tool you can use for collecting and organizing evidence—in this case, from artifact. You can see a small version of the form on this slide. For a closer view, turn to Handout 3. The cover page is intended to help educators communicate how a particular artifact demonstrates evidence of practice related to a goal or a particular Standard. Attaching this cover page to an artifact helps evaluators quickly identify the artifact’s purpose. Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Running Record of Evidence Form Explain: “An important component of both tools is the place where the educator or evaluator connects the evidence to a specific goal or Standard. This makes these two forms critical organizational tools because they help evaluators know exactly what type of evidence they have in front of them and how organize the evidence. “A tool for evaluators to keep track of all of this evidence might look something like this: a Running Record of Evidence form. The Running Record of Evidence form is in an Excel spreadsheet that is available on the ESE Educator Evaluation website. This tool includes drop-down boxes for the Source of Evidence, Standard or Indicator, and goals, so the evaluator can quickly identify and sort evidence based on a variety of criteria. You can also see a space to record the evidence statements, so if an evaluator were to sort by Standard I, Indicator A, for example, he or she could quickly read the evidence statements across multiple artifacts related to that Standard and Indicator and start to get a detailed picture of practice.” “This excel spreadsheet is just one of many organizational tools you might consider. How many of you already a system in place for collecting and organizing evidence? Some of you may have on online, paperless system that will support the new evaluation process, while others may be using a paper-based process. Either will work as long as they’re designed to help you organize the evidence using the performance Standards, Indicators and goals. This organizational process will make the analysis of evidence that much easier at the end of the 5-step cycle.” Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Importance of Strategically Collecting Artifacts Explain: “We have discussed the tools and processes that an evaluator and educator might use as evidence is collected about practice. Now we are going to discuss what you can do back at your school to ensure that educators and evaluators don’t just collect everything under the sun but, rather, a strategic sample of high-quality artifacts that contain valuable evidence. One pitfall we often see is educators building binders full of artifacts that could fill a wheelbarrow, or the flip side—educators who collect next to nothing, preferring to jot something down on the back of a napkin just prior to their evaluation. Either scenario is inevitably the product of lack of clear expectations. We want to help you and your educators avoid these pitfalls and spend time wisely collecting artifacts in a strategic way.” Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Importance of Strategically Collecting Artifacts Artifacts should be a sample that demonstrates educator performance and impact: Aligned with educator goals, the Model System Teacher Rubric, or school goals Number of artifacts to collect varies by educator Artifacts can provide evidence of more than one Standard or Indicator Explain: “‘The collection of artifacts must be seen as an opportunity to select a sample of artifacts and other data that fairly represents performance and impact. It is not intended to be a record of all that the educator has done in a year. It needs to be focused on the educator’s goals, high priority Standards and Indicators, and any critical school priorities not addressed by the professional practice and student learning goals. “There is no set number of artifacts required to be submitted, and, in fact, the number of artifacts to collect will vary by educator depending on their goals and the action steps in their plan. Whether an educator identifies 8, 10, or even 12 artifacts, the key is to ensure a balanced representation of performance.” Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Strategies for Collecting Artifacts Identify common artifacts that all or most educators will be expected to collect (e.g., lesson plans) Share examples of high-quality, valuable evidence during faculty or team meetings: Might include showing sample artifacts that provide evidence of more than one Standard or Indicator Explain: “As a school leadership team, you can utilize some strategies to promote effective, focused evidence collection in your school. “Evaluators and school leadership teams can identify common artifacts that all or most educators will be expected to collect, and communicate that expectation to your entire school staff. Common artifacts can be used to promote schoolwide goals that are aligned to school improvement plans, as well as team goals, thereby building coherence and a shared sense of accountability throughout the building. “In addition, faculty and team time should be devoted periodically to showcasing examples of well-chosen samples and their thoughtful analysis of impact. These examples might include artifacts that show evidence of multiple Standards or Indicators.” Transition: “At this point, we’ve talked about the importance of being strategic in collecting specific artifacts that align to your goals and/or Standards, and identifying a sample of artifacts that are representative of practice, as opposed to collecting everything you do as an educator. Now we are going to practice this process to see what it means to be strategic and selective—how do you translate multiple artifacts into valuable sources of evidence?”   Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Artifacts From Tom Wilson Five artifacts with partially completed Artifact Cover Pages Set Artifacts Location A A two-day lesson plan Handout 3 B Unit assessment data Team meeting minutes Handouts 4 and 5 C Parent communication log E-mail exchange Handouts 6 and 7 Learning Activity 2: Digging for Evidence (25 minutes) Purpose and intended outcomes: This activity gives participants the opportunity to look at artifacts and complete an accompanying Artifact Cover Sheet, which requires participants to synthesize the artifact into evidence statements and align those with Standards or Indicators from the Model System Teacher Rubric. Facilitation notes: The artifacts used in this activity are based on authentic artifacts, all aligned with sixth-grade science teacher Tom Wilson, whose Educator Plan participants reviewed in the Connecting activity. They are divided into three sets of documents. Set A includes one artifact—a two-day lesson plan (Handout 3). Set B includes two artifacts—a unit assessment tracking sheet and minutes from a team meeting (Handouts 4 and 5). Set C includes two artifacts—a parent communication log and an e-mail exchange with a parent (Handouts 6 and 7). Each artifact has a partially completed Artifact Cover Page for it. Participants will work in pairs to identify evidence statements for each artifact. They will write questions they have on the artifacts on sticky notes. Explain: “Now we are going to take a look at some actual artifacts from Tom Wilson. Tom submitted these artifacts to his principal leading up to his formative assessment. “There are five artifacts from Tom. Let’s take a look at them together. “In Handout 3, you will see the Artifact Cover Page for a two-day lesson plan from a unit on the Structure and Function of Cells. On the pages behind that are the actual lesson plans. “Handout 4 is an Artifact Cover Page for quarter 1 unit assessment data specific to scores of his English language learner students. Tom decided to submit this artifact because both of his goals focus on English language learners. “Handout 5 includes an Artifact Cover Page and for team meeting minutes from the science team meeting in November. “Handout 6 includes the artifact cover page and attached parent communication log. A parent communication log was one of the common artifacts that Tom’s school identified all teachers will need to submit. The log is behind the cover page. “Finally, Handout 7 is an e-mail exchange between Tom and a parent. The student they are discussing is one of the English language learners in Tom’s class.” Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Identifying Evidence in Artifacts Within your school team, divide into pairs. Each pair will do the following: Review one set of artifacts. Complete the Artifact Cover Page for those two artifacts. Consider these questions: After reviewing these artifacts, what else might you want to know about Tom’s practice? What would you want to ask this teacher? Explain: “You may have noticed something missing on the Artifact Cover Pages—the evidence statements. In this activity, you will review the artifact and identify the evidence relevant to that Indicator. Your task will be to identify evidence statements from the artifacts that align with the Standards and Indicators already mentioned on the Artifact Cover Page. For example, if you see Standard I, Indicator C: Analysis identified in the right-hand column, you might find evidence of data analysis in the document. Then you would write an evidence statement about it in the evidence space—for example, ‘Change in student scores between unit assessments is noted at the bottom of each column.’ You might also notice that just below the space where you can write your evidence is a note about how to indicate evidence related to the teacher’s goals—you can simply place a star or asterisk next to that evidence statement. So if/where you see evidence in these artifacts related to Tom’s goals on ELL student achievement and instruction, go ahead and star that evidence. Okay, let’s get going. The Teacher Rubric At-a-Glance is included as Handout 8 in your packet. You can use that as a resource, as well as the full Model System Teacher Rubric on your table. Do you have any questions?” Answer any participant questions. “Within your school team, please divide into pairs. Divide the three sets of artifacts among the pairs at your table. You will have about 15 minutes to examine the artifacts and synthesize the most relevant factual evidence statements from each one into evidence statements and record them on the cover pages. As you are doing this, also consider the two questions on Slide 19—what else do you want to know after looking at the artifacts? Based on what these artifacts demonstrate, what feedback or questions would you want to ask Tom Wilson that might serve as a starting point for further conversation between educators and evaluators? Using Post-it Notes placed on the pages, you can jot down any questions you have on the artifacts themselves or on the Artifact Cover Page.” After 15 minutes, get participants’ attention to proceed to the debrief. Common facilitation challenges and solutions: A school team might not have enough people (six) to review all of the documents as pairs or may have too many people (more than six) to pair up and review the documents. If there are fewer than six participants in a school team, have them pair up and choose a set to review. It is not essential for all of the artifacts to be reviewed by each team. If there are more than six participants in a school team, more than one pair can review a set. Participants may have difficulty identifying evidence statements for each artifact. Prompt participants to read through the Model System Teacher Rubric for the Standard and Indicator listed on the Artifact Cover Page. Ask them to identify what factual statements about the educator’s progress toward that standard they can make, based on the artifact. Guiding questions: After reviewing these artifacts, what else would you want to know from Tom? What kind of questions or feedback would you want to respond with? Learning Wrap-Up/Debrief 2 (15 minutes) Explain: “Let’s briefly review each set of artifacts and share the useful evidence that could be gleaned from each artifact review.” Ask: “Would a group that looked at Set A be willing to briefly describe the artifacts to us and share their evidence statements?” Allow one group to answer. Ask: “Would a different pair that looked at Set A have anything to add? Allow another group to answer. Ask: “Did any other group that looked at Set A identify any questions for this teacher on their Post-it Notes?” Allow any other groups who are interested to answer. “Now we are going to move on to Set B, a different pair of documents for this same teacher.” Proceed through this set of three questions until all three sets of artifacts have been discussed by the group. Optional activity for the facilitator: You may wish to create a list of artifact types (by set) and/or record participants’ answers during the discussion. One chart paper for each artifact set is recommended. Transition: “These example artifacts demonstrate how a single artifact can show evidence of practice in more than one Standard and/or Indicator. You have had some practice in how to create factual evidence statements based on artifacts and use the Artifact Cover Page. With our discussion, we also saw how artifacts could raise questions about educator practice that serve as a starting point for further conversation between educators and evaluators. It’s time to start thinking about how you will share this set of tools and the evidence collection process with your school staff.” Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

To Recap… Three categories of evidence: Multiple measures of student learning, growth, and achievement Judgments based on observations and artifacts of professional practice Additional evidence relevant to standards Explain: “Let’s take a moment to recap what we’ve learned about evidence. First, remember that the regulations call for multiple sources of evidence. Second, both educators and evaluators play a key role in collecting and organizing evidence throughout the implementation of the Educator Plan.” Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Support Organize Communicate Explain: “Given the comprehensive, strategic nature of evidence collection, it is helpful for you as a leadership team to consider what support will be given to teachers, how evidence will be organized, and how information will be communicated.” Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

1. Support Set Your School Up for Success The more concrete the Educator Plan, the easier it is to identify and collect artifacts. Share examples of high-quality, valuable evidence during faculty or team meetings: Demonstrate example artifacts that provide evidence of more than one Standard or Indicator. Identify common artifacts that all or most educators will be expected to collect (unit assessments, parent-teacher logs, etc.). Explain: “What do we mean by ‘support?’ Some key ways to provide support to educators throughout this process are listed here. “A concrete Educator Plan is the first step in the evidence collection process. As we saw in our connecting activity, Tom Wilson’s detailed Educator Plan helped you to identify relevant artifacts from the very beginning. The more you can support this step in the development of educator plans out the outset, the better positioned educators will be to collect evidence related to their practice. “Evaluators and members of the School Leadership Team are also strongly encouraged to share examples of high-quality evidence during staff meetings. This will help make sure that everyone in the school understands what high-quality evidence looks like, and how artifacts can and should include evidence of multiple Indicators. “Finally, the School Leadership Team can also work to identify a couple of common artifacts that all or most educators will be expected to collect. This helps to provide educators with some direction and helps to standardize the evidence collection process across the school.” Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

2. Organize Adopt a process for organizing artifacts and observation notes by Standard or Indicator and/or goals: Paper-based, e-mail-driven, or online “cloud-based” system Calendar: Review actions in Educator Plans and make agreed-upon supports and resources available to educator teams and individuals throughout the year. Identify key points of contact throughout the year (observations and feedback, formative assessment conferences, and summative evaluations). Explain: “Get ORGANIZED. Both educators and evaluators will have a lot of evidence to identify and collect. Adopting a standardized process for organizing artifacts and notes is essential, whether it’s paper-based, e-mail-driven, or supported by an online cloud-based system. “In addition, the creation of a schoolwide calendar will help to ensure that key points of contact occur, such as formative assessments, and will help evaluators organize their time.” Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

3. Communicate Expectations Avoiding the… OR Explain: “Finally, COMMUNICATE YOUR EXPECTATIONS. Expectations need to be clearly communicated to all educators about what and how to submit. This is critical to avoiding educators submitting evidence on the back of a napkin or needing to use a wheelbarrow to transport all of the documents they have collected.” Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

3. Communicate Expectations Artifacts should be a sample that demonstrates educator performance and impact. Evidence should be clearly tied to educator goals, Standards, or Indicators. Provide everyone with a clear idea of what, how, and when to share products of practice. Explain: “Important messages to communicate might include: Artifacts are only a sample of all evidence of educator performance and impact. Evidence should be tied to educator goals and/or Standards and Indicators. What, how, and when to share products of practice.” Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Back at your school… Work with your team to identify how you will: Support, Organize, and Communicate expectations related to the collection of evidence at your school. Implementing Activity: Setting Your School Up for Success (30 minutes) Purpose and intended outcomes: The purpose of this activity is to give participants a chance to plan how to implement the collection of evidence through artifacts, since it is a part of the new educator evaluation system that most schools have not previously implemented. Participants will work with their school team to fill out Handout 9: Plan for the Collection and Organization of Evidence. Activity Detail: Explain: “You will have the next 30 minutes to work with your school team to create an initial action plan using Handout 9: Plan for the Collection and Organization of Evidence. “Discuss how your school will: Provide necessary assistance and support to educators throughout the year, Organize evidence collection, and Communicate information and expectations related to this process clearly and effectively.” Give participants 30 minutes to work on their implementation plan. Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Feasibility Work with your team to identify how you will feasibly implement this aspect of the educator evaluation system. How will your school: Provide necessary assistance and support to educators throughout the year? Organize evidence collection? Communicate information and expectations related to this process clearly and effectively? Circulate among teams, prompting them if they get stuck on a question. Provide a time check for participants when 10 minutes are left. When 5 minutes are left, prompt teams to pause and do the following: “We have minutes left. I’d like you to stop at this point and identify two strong strategies you discussed that other schools might find helpful or interesting. Choose one person to share out those strategies with the whole group.” Give participants 5 minutes to complete this final step. Common facilitation challenges and solutions: Participants may say that a decision-making member of their school team is not present at the meeting. Prompt participants to consider this a suggested plan to bring back to the primary decision maker. Remind participants that their thinking in this area is valuable, since they have attended the training and understand the school context. Participants may finish early if they do not dig into the details of the implementation plan. Prompt participants to consider how they will share their decisions concerning implementation with all of the educators at their school. Ask them to create an agenda for that meeting. Guiding questions for discussion: How will your school: Provide necessary assistance and support to educators throughout the year? Organize evidence collection? Communicate information and expectations related to this process clearly and effectively? What are two strong strategies you discussed that other schools might find helpful or interesting?   Implementing Wrap-Up/Debrief (10 minutes) Ask the reporter from each group to share the two strategies the team identified as being particularly helpful or interesting. Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Reflecting On the back of your implementation plan, list: Three next steps for you personally Two challenges for implementing artifact collection at your school One question you still have regarding evidence collection Explain: “We have done a lot of strategic thinking today about how to collect and analyze evidence as well as how to make the most out of artifacts of practice. “Here at the end of the meeting, we are going to take a few minutes to reflect on what we have accomplished, and identify a few next steps that you need to complete individually. “On the back of your implementation plan, list: Three next steps for you personally Two challenges for implementing artifact collection at your school One question you still have regarding evidence collection.” Allow participants to have about 5 minutes to reflect on these questions. Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Homework Slide 33 is the title slide for the homework section. The homework assignments for this module will help prepare participants for Module 6. There are two homework assignments: one for school leadership teams and one for teachers. The homework assignments for this session are described on Slides 34 and 35. Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Observations and Feedback ___________________ Equipment and Materials Facilitation note: This module relies on technology. The facilitators should plan to test out the equipment and connectivity prior to the start of the session as well as generate a “back up plan” to ensure this aspect will go smoothly. Without the correct equipment, a strong internet connection, or an ESE-issued cd with the Module 6 video clips, most of the module activities will not work. Equipment: Laptop computer, projector, reliable internet connectivity OR downloaded video clips, and external speakers for video clip If choosing to stream the videos, consider a technology backup plan: if you are going to try to use wireless internet, make sure to know where and how to access a wired connection to the internet; bring two sets of speakers, in case one does not work; stream the videos at the site before the meeting starts to make sure they work; save the video links as bookmarks so it is easy to get to them during the meeting, etc. For ESE-issued cd’s with downloadable video clips, please email EducatorEvaluation@doe.mass.edu with the subject line “Module 6 video request.“ Materials: Make a copy of the Participant Handout packet for each participant. The first section of the Participant Handout packet includes the following two articles. Stephen Sawchuk: Studies link classroom observations to student achievement. Education Week Teacher Beat blog (http://blogs.edweek.org/edweek/teacherbeat/2011/04/studies_link_classroom_observa.html) Kim Marshall: Mini-Observations – 7 Decision Points for Principals (http://www.marshallmemo.com/articles/Ed%20Week%20Mini.pdf) Access to video clips for observation (also available in cd from ESE upon request) Video of Kelli (Learning Activity 2): http://www.youtube.com/watch?v=dRczDWKhwlg&feature=youtu.be Video of Chuck (Learning Activity 4): http://www.youtube.com/watch?v=AxBavxlDC9s Put the following materials on each table: Markers and highlighters (several per table) Standard size Post-It Notes (several pads per table) Bring the following materials for use by you: Chart paper Markers (at least one of each color: red, blue, green, black) Chart paper Two pieces will be needed during the Connecting Wrap-up/Debrief Four pieces will need to be hung around the room for Learning Activity 1, labeled Frequent, Focused, Varied (observations inside and outside the classroom), and Useful and Timely Feedback Massachusetts Department of Elementary and Secondary Education 29

Explain: “At the end of this module, you will be able to describe and delineate best practices in using observation as one of multiple sources of evidence in educator evaluation and craft timely, targeted feedback that is evidence-based and grounded in the Standards of effective practice.” Intended Outcomes At the end of this session, participants will be able to: Describe the role of observation as a methodology for gathering evidence of educator performance; Delineate best practices for conducting high-quality short, frequent, unannounced observations; and Craft timely, targeted feedback that is evidence-based and grounded in the Standards of effective practice. Explain: “At the end of this module, you will be able to describe and delineate best practices in using observation as one of multiple sources of evidence in educator evaluation and craft timely, targeted feedback that is evidence-based and grounded in the Standards of effective practice.” Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Every educator is an active participant in the evaluation process. Every educator and evaluator collects evidence and assesses progress Explain: “Observations are part of Step 3 of the 5-Step Cycle of evaluation, Implementation of the Plan. The Implementation of the Plan begins as soon as Educator Plans are finalized and continues until the end of the cycle and the summative evaluation occurs. In Module 5 you learned about gathering and organizing multiple types of evidence related to educator performance. Today our focus is on observations as one more source of evidence, and the role of feedback in promoting growth and development.” Collaboration and continuous learning are the focus. Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education 31

What the Research Says . . . Evaluators need multiple opportunities and settings to observe and assess educator practice Multiple observations paired with timely feedback are a key part of a strong evaluation system Connecting Activity: Expert Voices on Observation (15 minutes) Purpose and intended outcomes: The purpose of this activity is to collaboratively identify the purpose and use of observation and its role in the Model System and to connect the Model System rationale to research and experts’ view on observation. Facilitation notes: It is highly recommended that the facilitator do this activity him or herself before presenting it to participants; it will make the facilitation stronger. Each table will be assigned to read one of two articles. It should be easy to count off the tables quickly during the activity. Tables numbered 1 will read the article on Handout 1. Tables numbered 2 will read the article on Handout 2. Both articles are included in the participant handout packet. Participants will read the article identifying key words and phrases related to the use of observation and feedback. Then table groups will share their key words and phrases. Each table group will share out their words and the facilitator will record main ideas on two pieces of chart paper, one for each article.  Activity detail: “The design of the Massachusetts Model System for Educator Evaluation is intentionally grounded in relevant research that documents and represents best practice. When it comes to using observations in an effective, meaningful manner, research finds that: Evaluators need multiple opportunities and settings to observe and assess educator practice, and Multiple observations paired with timely feedback are a key part of a strong evaluation system.” “The Massachusetts Model System for Educator Evaluation recommends frequent, unannounced observations of individual educators throughout the year as a way to observe practice over time and promote ongoing communication around teaching and learning. For those educators who are in the early stages of their career or educators who are struggling, the Model recommends one or two formal, announced observations as well as a means of providing additional, more targeted support. But the emphasis—and the real shift in practice—is the incorporation of multiple, short observations over time.” Explain: “In the past, all observations tended to be limited in number (infrequent), yet often served as the sole source of evidence for an educator’s evaluation. This approach to observation and evaluation produces extremely limited information about practice is of little value to the educator when it comes to individualized, continuous improvement. A growing body of research is now dispelling the value of traditional, “bell-to-bell” observations and promoting more frequent, targeted “mini” observations as integral to improving educator practice and student learning. “In our first activity, you will have an opportunity to dip your toe into some of this research on the value of frequent classroom observations followed by targeted feedback.” Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Voices on Observation Handout 1: Studies link classroom observations to student achievement , by Stephen Sawchuck Handout 2: Mini-Observations – Seven Decision Points for Principals, by Kim Marshall Explain: “Please turn to the first few pages of your Participant Handout packet. You will see two articles: the first is a summary of two recent reports on the effect of frequent observations in Cincinnati Public Schools’ teacher evaluation model. There are some differences between the program implemented in Cincinnati and what we are doing in Massachusetts, but the research on the effectiveness of observations as part of teacher evaluation is very relevant. “The second article is a short piece by Kim Marshall on how to conduct mini observations. When Kim Marshall was a principal in Boston, he used multiple observations to build stronger norms of feedback into the regular life of teachers in his building. “Both articles are included so everyone will have a copy of each article. You will only read one article, so let’s have each table count off as either a 1 or 2.” Allow tables to count off, pointing at each table and counting off as well. Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Active Reading Underline statements you find to be important takeaways or themes from the article Circle one word or phrase that represents the key takeaway or theme Explain: “Tables that were 1s will read the article on Handout 1. Tables that were 2s will read the article on Handout 2. “As you read, underline statements you find to be important takeaways or themes from the article and then circle one word or phrase that represents the key takeaway or theme. “Any questions?” (Pause for questions) “You will have about 5-10 minutes to read the article and underline and circle the text. Go ahead and get started.” Give participants 5-10 minutes to read the text they were assigned. Monitor the time and call for the group’s attention after four minutes, to see if additional time is needed. Provide a one-minute warning and then bring the group back together after 5-10 minutes. Explain: “Now, select a recorder at your table to keep track of your group’s collective list of important words and phrases, and then share your individual responses in a quick conversation within your group. Keep in mind that this activity is not about reaching group consensus, but how the author’s work resonated with those at your table. Please listen to your colleagues’ explanation of why they selected particular key words or phrases.” Common Facilitation Challenges and Solutions: Participants may ask: Are we not supposed to conduct formal announced observation? Respond by saying that both announced and unannounced observations have a role in educator evaluation. The Massachusetts Model Evaluation System supports a range of methodologies for evaluators to use that can be applied situationally to reflect the needs of the educator. For example, the Model System recommends one to two formal observations for new or struggling educators, depending on their status, in addition to the brief, unannounced observations in order to provide additional support where needed. Participants may want to discuss how this would look in practice. Respond by encouraging each school team to complete the homework assignment at the conclusion of this module, which will allow them to discuss and plan what an annual schedule of observations might look like in their school context. Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Making Connections What are the key messages and ideas in the two articles with regard to observations and feedback? What are some best practices for observations and feedback? Call for the group’s attention after 6-8 minutes. Explain: “Let’s share what we just discussed. We’ll take it one table at a time. As we proceed, if another group has already captured ideas from your group just let us know so we can track our coherence and then report any words/phrases that have not been shared.” As groups report, the facilitator should jot down key words and phrases from each article on a piece of chart paper, using one piece of paper for each article. Guiding Questions: What are the key messages and ideas in the two articles with regard to observations and feedback? Based on what we just read, what are some best practices for observations and feedback? Some key messages that participants may identify from the two articles are listed below. Some Key Messages Sawchuk Article (Handout 1) NOTE: this article highlights the relationship between a systematic evaluation process that includes multiple observations and improved teaching and student learning outcomes High overall teaching practice can improve student achievement. Compared to student achievement in years prior to being evaluated, student performance in math increased during the year the teacher was evaluated, during which time the teacher was observed four times and rated on a 4-point scale. Statistically significant student achievement gains in math continued in years after the teacher observation. Marshal Article (Handout 2) NOTE: this article highlights the process of short, frequent observations and the value of feedback to teacher practice 5 minutes is enough to capture rich information about practice Face-to-face feedback that is thoughtful and prompt is key. Marshal had a goal of conducting five short informal observations each day. The focus for a short observation can be targeted—“short mental checklist.” Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Observing Practice: What can you see? Inside the Classroom Outside the Classroom Teacher Student engagement Classroom management Differentiated instruction Cultivating a safe learning environment Professional collaboration (team meetings) Family and community engagement Collaborative data analysis Principal Teacher Observations 2-way communication Communication of school vision Safe learning environment Collaborative planning Learning Content 1 (5 minutes) Explain: “The Standards and Indicators of Effective Teaching and Administrative Leadership Practice, as represented in the Model rubrics, represent the full scope of a teacher’s or leader’s performance. This includes not only the teaching and leading aspects of practice but engagement with families and the community and professionalism. “Not all of these aspects of practice are observable, however. Individual lesson planning or aspects of family communication may take place behind the scenes, for example. Those are areas for which educators and evaluators will want to seek out and provide artifacts to demonstrate practice, like what we reviewed in Module 5. Given this comprehensive nature of the rubrics, it’s important to realize that they are not designed to be, nor can they serve as, observation tools. Just imagine walking into a classroom with a 15 page rubric and trying to look for evidence of all 33 elements in five to ten minutes!” “That said, observation continues to be an important strategy for gathering evidence of educator practice. This includes observations of teacher and principal practice both inside and outside of the classroom. From classroom lessons to grade level meetings to school assemblies, opportunities to observe practice are abundant throughout the school day, and taking advantage of them provides a richer, more complete picture of educator practice. This doesn’t mean the educators are always under a microscope. This new approach to observation simply broadens the opportunities to observe practice beyond the four walls of a classroom. “In addition to the examples on the slide, what are some other examples of observable practice that you might look for?” Solicit some responses from the whole group. Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Principles of Brief Observations Frequent Focused Varied Useful and Timely Feedback Explain: “There are certain principles that evaluators should be aware of as they approach conducting short, frequent, unannounced observations, especially if this represents a change from their past observations work. In the first learning activity we will examine four principles of high quality brief observations: frequent, focused, varied, and resulting in useful and timely feedback.” Learning Activity 1: Approaching Short Frequent Observations (15 minutes) Purpose and intended outcomes: Connect participants’ ideas about observation to four broader guiding principles: frequent, focused, varied (inside and outside the classroom), and resulting in useful and timely feedback. Facilitation Details: The facilitator will need four charts posted in the room with the following headers: Frequent (red marker with chart) Focused (blue marker with chart) Varied (observations inside and outside the classroom) (green marker with chart) Useful and Timely Feedback (orange marker with chart) During this activity, participants will count off by 4s with each number going to a different chart. The purpose of this facilitation strategy is to get participants out of their seats for a bit and interacting with people other than those in their school group. In some cases, it may not be feasible for this exact facilitation to work effectively. If the group is too large, such that it results in more than six people per group, the facilitator could divide the participants into eight groups and create eight pieces of chart paper (doubling up on each of the topics). If it is not possible to use chart paper in the space, the facilitator could consider having participants work in table groups. Activity detail: Have participants identify which group they are in by counting off by fours. Explain: “Quickly, within your tables, I’d like everyone to count off by 4s. Okay? Go. “Each numbered group is going to be assigned to a chart labeled with one of the principles for brief observations. I’d like everyone to stand up and make your way over to your chart. 1s should head over to the chart titled “frequent.” 2s should gather around the chart labeled “focused.” 3s head over to the chart titled “varied,” and 4s will start at the chart labeled “feedback.” Allow participants about 2 minutes to re-sort into their new standing groups. Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Guiding Questions What does it mean to be FREQUENT in your observations? How does this principle of high-quality observations better support the improvement of teaching and learning? Explain: “With your new group, brainstorm responses to the following questions: What does it mean to be _______ in your observations? How does this principle of high-quality observations better support the improvement of teaching and learning? ”Have one person record your responses to each question on the chart paper. After a few minutes, I’m going to signal for each group to move the next chart, at which point you should review what the prior group has recorded. Circle any notes or phrases you feel are most important, and add any additional thoughts to the chart. Each group will have the opportunity to review each of the four principles. “Ready to begin? Let’s get started.” Signal groups to move to the second chart after 3-4 minutes. Participants will need less time at each subsequent chart, since they will be working with more existing information. Stay aware of participant pace and adjust the time they have at each station accordingly. Explain: “Now that you have returned to your original chart, take two minutes to synthesize the ideas and be prepared to report out 2 or 3 key ideas that were generated. On your charts star the two or three ideas you will be sharing.” Give participants two minutes to complete this task. “We will take the next four minutes for groups to share two key ideas that you have for your guiding principle.” Call on each group.  Common Facilitation Challenges and Solutions: Pacing of the activity You will need to really watch the time and keep groups moving. You won’t be able to monitor all groups at the same time.  SEE FACILITATOR GUIDE FOR POSSIBLE ANSWERS TO EACH QUESTION FOR THE FOUR TOPICS (P. 17) Learning Wrap-up/Debrief 1 (5 minutes) As you wrap up the group discussion, ask: “What is the bottom line about these four principles? What idea has really jumped out at you?” Explain: “While this methodology represents a change in the way observations have traditionally been approached, short frequent unannounced observations can yield a great deal of useful information and many more samples of practice to support powerful conversations about educator practice.” Transition and Explain: “In the next 30-40 minutes, we are actually going to conduct an observation using a short video. You will be able to see what it feels like to take notes during a short, unannounced classroom visit, and then generate targeted, evidence-based feedback for the teacher. As you’ll see, the quality of your feedback relates directly to what you capture during your observation, so we’ll start with how to approach the collection of high-quality evidence during an observation.” Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Guiding Questions What does it mean to be FOCUSED in your observations? How does this principle of high-quality observations better support the improvement of teaching and learning? Explain: “With your new group, brainstorm responses to the following questions: What does it mean to be _______ in your observations? How does this principle of high-quality observations better support the improvement of teaching and learning? ”Have one person record your responses to each question on the chart paper. After a few minutes, I’m going to signal for each group to move the next chart, at which point you should review what the prior group has recorded. Circle any notes or phrases you feel are most important, and add any additional thoughts to the chart. Each group will have the opportunity to review each of the four principles. “Ready to begin? Let’s get started.” Signal groups to move to the second chart after 3-4 minutes. Participants will need less time at each subsequent chart, since they will be working with more existing information. Stay aware of participant pace and adjust the time they have at each station accordingly. Explain: “Now that you have returned to your original chart, take two minutes to synthesize the ideas and be prepared to report out 2 or 3 key ideas that were generated. On your charts star the two or three ideas you will be sharing.” Give participants two minutes to complete this task. “We will take the next four minutes for groups to share two key ideas that you have for your guiding principle.” Call on each group.  Common Facilitation Challenges and Solutions: Pacing of the activity You will need to really watch the time and keep groups moving. You won’t be able to monitor all groups at the same time.  SEE FACILITATOR GUIDE FOR POSSIBLE ANSWERS TO EACH QUESTION FOR THE FOUR TOPICS (P. 17) Learning Wrap-up/Debrief 1 (5 minutes) As you wrap up the group discussion, ask: “What is the bottom line about these four principles? What idea has really jumped out at you?” Explain: “While this methodology represents a change in the way observations have traditionally been approached, short frequent unannounced observations can yield a great deal of useful information and many more samples of practice to support powerful conversations about educator practice.” Transition and Explain: “In the next 30-40 minutes, we are actually going to conduct an observation using a short video. You will be able to see what it feels like to take notes during a short, unannounced classroom visit, and then generate targeted, evidence-based feedback for the teacher. As you’ll see, the quality of your feedback relates directly to what you capture during your observation, so we’ll start with how to approach the collection of high-quality evidence during an observation.” Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Guiding Questions What does it mean to be VARIED in your observations? How does this principle of high-quality observations better support the improvement of teaching and learning? Explain: “With your new group, brainstorm responses to the following questions: What does it mean to be _______ in your observations? How does this principle of high-quality observations better support the improvement of teaching and learning? ”Have one person record your responses to each question on the chart paper. After a few minutes, I’m going to signal for each group to move the next chart, at which point you should review what the prior group has recorded. Circle any notes or phrases you feel are most important, and add any additional thoughts to the chart. Each group will have the opportunity to review each of the four principles. “Ready to begin? Let’s get started.” Signal groups to move to the second chart after 3-4 minutes. Participants will need less time at each subsequent chart, since they will be working with more existing information. Stay aware of participant pace and adjust the time they have at each station accordingly. Explain: “Now that you have returned to your original chart, take two minutes to synthesize the ideas and be prepared to report out 2 or 3 key ideas that were generated. On your charts star the two or three ideas you will be sharing.” Give participants two minutes to complete this task. “We will take the next four minutes for groups to share two key ideas that you have for your guiding principle.” Call on each group.  Common Facilitation Challenges and Solutions: Pacing of the activity You will need to really watch the time and keep groups moving. You won’t be able to monitor all groups at the same time.  SEE FACILITATOR GUIDE FOR POSSIBLE ANSWERS TO EACH QUESTION FOR THE FOUR TOPICS (P. 17) Learning Wrap-up/Debrief 1 (5 minutes) As you wrap up the group discussion, ask: “What is the bottom line about these four principles? What idea has really jumped out at you?” Explain: “While this methodology represents a change in the way observations have traditionally been approached, short frequent unannounced observations can yield a great deal of useful information and many more samples of practice to support powerful conversations about educator practice.” Transition and Explain: “In the next 30-40 minutes, we are actually going to conduct an observation using a short video. You will be able to see what it feels like to take notes during a short, unannounced classroom visit, and then generate targeted, evidence-based feedback for the teacher. As you’ll see, the quality of your feedback relates directly to what you capture during your observation, so we’ll start with how to approach the collection of high-quality evidence during an observation.” Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Guiding Questions What does it mean to be USEFUL and to give TIMELY FEEDBACK in your observations? How does this principle of high-quality observations better support the improvement of teaching and learning? Explain: “With your new group, brainstorm responses to the following questions: What does it mean to be _______ in your observations? How does this principle of high-quality observations better support the improvement of teaching and learning? ”Have one person record your responses to each question on the chart paper. After a few minutes, I’m going to signal for each group to move the next chart, at which point you should review what the prior group has recorded. Circle any notes or phrases you feel are most important, and add any additional thoughts to the chart. Each group will have the opportunity to review each of the four principles. “Ready to begin? Let’s get started.” Signal groups to move to the second chart after 3-4 minutes. Participants will need less time at each subsequent chart, since they will be working with more existing information. Stay aware of participant pace and adjust the time they have at each station accordingly. Explain: “Now that you have returned to your original chart, take two minutes to synthesize the ideas and be prepared to report out 2 or 3 key ideas that were generated. On your charts star the two or three ideas you will be sharing.” Give participants two minutes to complete this task. “We will take the next four minutes for groups to share two key ideas that you have for your guiding principle.” Call on each group.  Common Facilitation Challenges and Solutions: Pacing of the activity You will need to really watch the time and keep groups moving. You won’t be able to monitor all groups at the same time.  SEE FACILITATOR GUIDE FOR POSSIBLE ANSWERS TO EACH QUESTION FOR THE FOUR TOPICS (P. 17) Learning Wrap-up/Debrief 1 (5 minutes) As you wrap up the group discussion, ask: “What is the bottom line about these four principles? What idea has really jumped out at you?” Explain: “While this methodology represents a change in the way observations have traditionally been approached, short frequent unannounced observations can yield a great deal of useful information and many more samples of practice to support powerful conversations about educator practice.” Transition and Explain: “In the next 30-40 minutes, we are actually going to conduct an observation using a short video. You will be able to see what it feels like to take notes during a short, unannounced classroom visit, and then generate targeted, evidence-based feedback for the teacher. As you’ll see, the quality of your feedback relates directly to what you capture during your observation, so we’ll start with how to approach the collection of high-quality evidence during an observation.” Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Strategies for Collecting Evidence Identify a focus ahead of time Goals, specific Standards/Indicators Record evidence, not judgment Quotations, observed actions or movements by teacher and students, literal descriptors, etc. Be an efficient note-taker Establish abbreviations, paraphrase Learning Content 2 (5 minutes) Explain: “Before jumping into our first video observation, let’s go over a few key things to consider when collecting evidence from a short, unannounced observation. “First and foremost, identify a focus ahead of time. It’s critical to focus on one or two areas related to that educator’s practice during each observation. Before you enter that classroom or join that meeting, take a moment to re-familiarize yourself with that educator’s goals. What do you want or expect to see from this particular teacher at this point in the year? A targeted Standard or Indicator, or progress toward one or two of the educator’s goals becomes your “short mental checklist,” as Kim Marshall would say. Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Evidence versus Judgment T: “Explain your answer to me. Show me what you did.” Teacher does a good job getting students to explain their reasoning Students constructed sailboats using various types of materials independently Teacher wasn’t engaged enough with the students and just sat at his desk  Explain: “Next, remember to record evidence during your observation, not judgment. As on observer, it’s important to write concrete evidence statements that describe what the teacher and students are doing, not how you feel about it. We talked about the difference between fact and opinion in Module 5 and even practiced turning opinion statements into factual statements. There are a couple more examples here on slide 16. You will notice a direct quote from the teacher versus an opinion statement about what the teacher was doing – direct quotes are a great way to capture evidence. Opinion statements may be something you eventually use as feedback, but the evidence statement on the left is what you would record during the observation.” Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Collecting Evidence Through Observation Explain: “Finally, it’s helpful to adopt one or two note-taking strategies that facilitate quick short-hand. Here is a sample excerpt of observation notes. If we look at the slide, we see that the evaluator has used abbreviations such as: T (teacher), S (student), FFL (focus for learning), LD (lesson delivery), as well as time stamps at the bottom. “Choosing a few easy to decipher abbreviations, such as “S” for student and “T” for teacher, recording key conversations verbatim if possible, or paraphrasing around short quotes that convey the tone and spirit of the interaction are all good note-taking strategies during short observations.” Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Video I Background Kelli teaches 4th grade A math lesson is in progress Kelli’s goals focus on: Improving students’ understanding of place value and properties of operations in order to perform multi-digit arithmetic (Standard 4.NBT.4-6); and Using instructional practices that engage all students during independent or small group work time (Indicators II-A and II-B). http://www.youtube.com/watch?v=dRczDWKhwlg&feature=youtu.be Learning Activity 2: Video Practice – Short and Sweet (25 minutes) Purpose and intended outcomes: Practice gathering evidence during a brief, unannounced observation of a classroom lesson and provide guidance on organizing the evidence afterwards. Facilitation Notes: It is highly recommended that the facilitator do this activity him or herself before presenting it to participants. This will ensure better facilitation and allow the facilitator to give personal and authentic examples that are based on the video. Provide the following background about the video: Explain: “Now we’re going to stop by Kelli’s 4th grade math classroom for a quick 9-minute observation. Kelli is an experienced educator on a Self-Directed Growth Plan. Kelli’s goals focus on: Improving students’ understanding of place value and properties of operations in order to perform multi-digit arithmetic (Standard 4.NBT.4-6); and Using instructional practices that engage all students during independent or small group work time (Indicators II-A and II-B). “Remember, the first thing to do prior to an observation is to identify a focus. For this observation, we’re going to focus on evidence related to Kelli’s second goal, which is her professional practice goal. Go ahead and turn to Handout 3 in your packet. You can see it includes a 2-page excerpt from the Teacher Rubric showing the sections of the rubric related to this goal. See the two elements that are already circled? Take a second to read the descriptions or practice in the Proficient column for those two elements, just to remind you of what you might be looking for in Kelli’s instruction. While participants review Handout 3, facilitator should get the video clip ready. “Ready? We’re going to pretend we are Kelli’s principal now, walking into her room for a second unannounced observation this year. Grab a blank piece of paper, an iPad, or your laptop—whatever is available to you—and be ready to capture evidence as you enter the room. This video runs for about nine minutes and starts in the middle of Kelli’s lesson. Watch, listen, and write. You’ll see immediately that she’s conducting a lesson that involves small group work, so be prepared to take notes on her second goal: engaging all students during independent or small group work time. “You’ve just familiarized yourself with two relevant elements from the rubric, so you have an idea of what to look for. Feel free to jot down other things you observe that jump out at you as relevant or compelling, but remember—having a focus is important in brief observations.” “Everyone ready? Okay, here we go . . .” Show the queued-up video segment (approximately 9 minutes). During the video, the facilitator should also take notes by watching the video and writing evidence. Explain: “Okay, before we start discussing what we just observed, I’d like everyone to locate Handout 4 in your packet. This is the Observation Evidence Collection Tool that Kelli’s principal used during this observation. This may look familiar to you from Module 5. The Observation Evidence Collection Tool is designed to help evaluators document key evidence from observations in a way that facilitates the organization of that evidence, as well as the development of feedback based on that evidence. “You can see that Kelli’s principal was able to record the following information: Details of the observation (number, date, time, location), notes collected during the observation, and the Standards and Indicators of Effective Practice to which various aspects of Kelli’s practice align. There’s also space at the bottom to construct written feedback. While it is recommended that feedback be face-to-face, it will still be important to the overall evaluation process to formally record feedback.” Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Recording Evidence From An Observation Compare/discuss the notes on the completed Observation Evidence Collection Form to evidence you collected Identify points of agreement as well as evidence that was not included Explain: “At this point, I’d like you to work in a small group—two to three people—and start to compare and discuss the principal’s notes on the completed Observation Tool versus what you jotted down. Identify those points where your evidence matches as well as any different or additional evidence you may have collected. Don’t worry about the Standard/Indicator notations in the right-hand column—we’ll get to those later.” Common Facilitation Challenges and Solutions: Participants may get distracted or hung up on how the principal took notes, especially if their format, shorthand or notes took a different form than that of the teacher. Remind participants that the purpose of the activity is to focus on what we saw. The principal’s notes are not meant to be an exemplar, but rather an example. There are many effective ways to take observation notes and the focus should be on the participant finding a strategy that works for him/her. Participants may have missed things that the example principal chose to write down or feel that they were not able to capture in writing everything that they saw during the observation. Tell participants that conducting observations is a skill that must be practiced, especially when using a new rubric. As they become more familiar with the rubric and more comfortable taking observation notes during these short observations, they will feel more confident. In order to gain confidence, they can always practice observing classrooms in their school for a week without including those observations in the teachers’ evaluation (of course sharing this information with the teachers). Guiding Questions: Did you see the same things her principal saw? What did her principal miss that you noticed? What evidence did you see related to Kelli’s second goal, on a collaborative learning environment? Did you note any evidence related to her first goal or other Standards and Indicators? Facilitator Note: Facilitator should roam during this small group discussion and provide support and guidance to groups as their exchanges unfold. Some evidence that participants may have seen in the video clip related to Kelli’s second goal on a collaborative learning environment includes: Students were working in groups throughout the observation. Students were instructed to ask other classmates in their group for help before asking the teacher, encouraging collaboration. The teacher explicitly reminded one student to ask three group members first before asking her. The teacher paired two students up, facilitating them working together, when they were approaching the problem using a similar strategy. Students were in same-gender groups—deliberate? Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Observation Debrief What evidence did you see related to Kelli’s second goal, on a collaborative learning environment? Did you see the same things her principal saw? What did her principal miss that you noticed? What evidence did you see of other Standards, Indicators or goals? Learning Wrap-up/Debrief 2 (10 minutes) Call for the group’s attention after 5 minutes. Ask the participants the following questions, allowing time for individuals or groups to raise their hands and respond. What evidence did you see related to Kelli’s second goal, on a collaborative learning environment? Did you see the same things her principal saw? What did her principal miss that you noted? What evidence did you see of other Standards, Indicators or goals? “You probably noticed in the principal’s observation notes that she found evidence of three indicators during the observation. Although a focus is critical to have when conducting a short observation, you’ll often see evidence of practice related to other goals or Indicators. Don’t shy away from noting them.” Explain: “We’ve now had an opportunity to experience a brief classroom observation and collect some evidence of practice. The next step is to turn this evidence into feedback that’s targeted and helpful to the teacher. ” Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Value of Feedback Targeted, Ongoing Feedback Improved Educator Practice Improved Student Achievement Learning Content 3 (5 minutes) Explain: “The purpose of feedback is to help educators improve their practice. Providing feedback to educators on a regular basis supports frequent opportunities for conversation between the educator and their evaluator around instruction, and promotes a culture of continuous improvement. “When done well, this process can have marked, long-lasting changes on teacher practice and student outcomes. According to a multi-year study of Cincinnati Public School’s evaluation framework, in which teachers were observed 4-6 times per year and provided targeted feedback each time, teachers learned new information about their own performance and developed new skills. Not only did evidence show teachers sustaining these behavioral changes from year to year, these teachers were more effective at raising student achievement during the school year when they were being evaluated than they were previously, and even more effective in subsequent years. “Researchers attributed these outcomes to the focused observations and feedback routines of the evaluation process, as well as increased opportunities for self-reflection and conversations regarding effective teaching practice. Perhaps most illuminating was the suggestion that regular, targeted feedback was actually more important to lasting performance improvements than the final, overall [evaluation] scores at the end of the year (Taylor & Tyler, 2012). Bottom line: we can’t emphasize the importance of feedback to the continuous improvement of educators enough.” Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Characteristics of Effective Feedback FOCUSED: feedback should focus on what was observed EVIDENCE-BASED: feedback should be grounded in evidence of practice CONSTRUCTIVE: feedback should reinforce effective practice and identify areas for continued growth TIMELY: feedback should be provided shortly after the observation Explain: “If we want to make sure to give teachers effective feedback, what does that look like? “Effective feedback is FOCUSED: feedback should zero in on what you just observed and how that practice reflects the educator’s progress toward his/her goals or performance within the four standards of practice. “Effective feedback is EVIDENCE-BASED: feedback should be grounded in evidence related to what was just observed. “Effective feedback is CONSTRUCTIVE: Feedback should both reinforce evidence of effective practice and identify areas for continued growth, with suggestions for improvement and/or the identification of additional resources or supports if needed. “Finally, effective feedback must be provided in a TIMELY manner to be effective. It is not helpful to receive constructive feedback weeks after the observation. Ideally feedback should be provided within two or three days after the observation. “Let’s go back to Kelli in order to see how her principal might have constructed his feedback based on this recent observation.” Learning Activity 3: Providing Useful and Timely Feedback (15 minutes) Purpose and intended outcomes: Provide an opportunity for participants to utilize multiple sources of evidence, and to identify two areas of feedback based on their observation and knowledge of Kelli’s goals. Facilitation Notes: The facilitator should make sure to complete this activity him or herself before doing it with participants, identifying two points of feedback for Kelli. It is a critical step that will help strengthen activity facilitation. Activity Detail: Explain: “As you think about Kelli’s goals in her Educator Plan, you will want to provide feedback to Kelli that reinforces evidence of effective practice related to one or more of her goals so it’s relevant to her work going forward. Linking this feedback to specific Standards and Indicators from the teacher rubric will help to ground your feedback in concrete descriptions of effective practice and focus her in self-reflection about next steps. “Let’s go back to Handouts 3 and 4 in your packet, the Observation Collection Tool and the 2-page excerpt from the Teacher Rubric.” “During the observation, we all focused on practice related to Kelli’s professional practice goal: to increase the engagement of students in independent and small group work time. We definitely saw evidence of practice related to this goal. Using the evidence we collected from our observation, the next step is to ground our feedback in what we saw, and what effective practice looks like. “This is where the rubric comes in again. It’s the guidepost we use to assess practice based on what we just observed. What should it look like to support a collaborative learning environment? Is Kelli effectively motivating and engaging students through small groups? The 2-page excerpt you have from the Model Teacher Rubric includes descriptions of practice related to her goal that can help to frame your feedback. For example, we observed a couple examples of routines or rituals that Kelli used to maintain a safe, productive learning environment throughout the lesson, practice that is encapsulated in Indicator B, element 1. Remember “Mona. [Lisa.] Mona. [Lisa.]?” Another tactic might have been the same-gender grouping of students, but it’s hard to know. If this was something I wanted to highlight, I might construct feedback like, ‘I noticed some ways in which you captured the students’ attention and controlled behaviors that might have interfered with learning. The “Mona/Lisa” technique was particularly effective in re-capturing the students’ attention—nice job. I’m also curious about the same-gender grouping—do you find this approach to be helpful in promoting a safe intellectual environment? “Notice how I situated evidence from my observation in the description of practice from the rubric? This is just one example of how you might use the rubric to help anchor your feedback. Let’s try it ourselves.” Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Constructing Feedback Kelli’s Goal: Using instructional practices that engage all students during independent or small group work time (II-A-2, II-B-2.). Using your observation notes and the rubric: Draft two points of feedback for Kelli Focus on feedback that is Focused Evidence-based, and Constructive Explain: “Remember that this feedback might be a “stand-up conversation” followed by a written note or simply an email. I would like you to focus on written feedback for this activity. On your own, you’re going to review your notes from the observation and start flagging evidence of practice that stands out in relation to Kelli’s goal: using instructional practices that engage all students during independent or small group work time. Remember to identify points of strength as well as areas that may need follow-up or improvement. “Then, using the rubric excerpt in Handout 3, see if you can locate what you saw in Kelli’s classroom in the descriptors of practice. You’ll see two elements already circled: II-A-2 (student engagement) and II-B-2 (collaborative learning environment). These are the elements most directly related to Kelli’s goal, but don’t be afraid to utilize other elements to ground your feedback, where relevant. The purpose of feedback is to help Kelli see where her practice lies along the continuum, and what she can do to improve.” “You will have 10 minutes to draft two points of feedback for Kelli. Feel free to write directly into the box on Handout 4 titled ‘Feedback to the Educator.’” Cue individuals at eight minutes that they have two minutes left. Common Facilitation Challenges and Solutions: It will be difficult to monitor participants to ensure that they are completing the exercise. Rotate among as many tables as possible to ensure participants are completing the work and prompt individuals as needed. By this time you may also be able to observe school teams who are comfortable with the content that has been introduced versus teams who are really struggling. Groups may have trouble starting. If it looks as though teams are struggling to get started, suggest that they work in pairs to draft two points of feedback, rather than individually. Guiding Questions: Is the feedback focused, evidence-based, and constructive? Learning Wrap-up/Debrief 3 (5 minutes) Explain: “By now, you’ve hopefully drafted a couple points of feedback for Kelli, using your own observation notes as well as the rubric. Take a few minutes in your teams and share your feedback statements. Individuals may have zeroed in on different aspects of Kelli’s practice around which to give feedback, or you might discover similar ideas for feedback among your group. Remember to ask yourselves, ‘Is the feedback focused, evidence-based, and constructive?’” After a few minutes of small group discussion, bring the entire group back together for a whole group discussion about what they identified as feedback for Kelli. Transition and Explain: “As we have worked through the three learning activities we have focused on the four principles of high quality brief observations; we gained practice conducting a short, unannounced observation and collected and recorded key evidence; and we also practiced providing focused, evidence-based and constructive. You probably noted that we did not rate Kelli’s practice today. Although you might have situated your feedback to Kelli in the language of the rubric, you did not at any point in time record or submit a rating of “proficient” or “needs improvement.” This is an important point to take home with you: the rubric is there as a guide for your observations and can help you situate an educator’s practice within the four performance levels at any given time. “Remember, the purpose of conducting several observations over the course of time is to construct a more comprehensive picture of practice, rather than rating a point-in-time snap shot. Eventually, an evaluator will sit down with multiple pieces of evidence, from artifacts to observation notes, and assess them in their totality in order to arrive at an informed performance rating.” Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Video II Background Chuck teaches 10th grade physics Chuck is a 3rd year, non-PTS teacher Chuck’s goals focus on: Helping students to analyze, interpret and communicate results of scientific investigations and translate numerical or technical information into words. Creating a safe and collaborative learning environment (II-B-1 and II-B-2) http://www.youtube.com/watch?v=AxBavxlDC9s Explain: “We’re about to meet Chuck, a 10th grade physics teacher. It’s the third week of school, and this is the principal’s first visit to Chuck’s classroom this year. “Another important note about Chuck is that he is a non-PTS teacher in his third year of teaching. This is a critical year for him—he must be rated Proficient overall in order to receive professional teaching status. “Let’s talk a little bit about Chuck’s goals for this year. The science department at Chuck’s school is focusing on scientific literacy this year as part of their school’s implementation of the revised MA Curricular Frameworks. For Chuck, this means helping his 10th grade students learn how to analyze, interpret and communicate results of scientific investigations that use multiple variables, with a specific emphasis on teaching students how to translate quantitative or technical information expressed visually (e.g., a table or chart) into words using appropriate terminology and phrases. Chuck’s student learning goal uses writing assessments to gauge students’ ability to meet this standard. “Chuck’s professional practice goal focuses on creating a safe learning environment and collaborative learning environment. As a non-PTS teacher, Chuck is still working on strengthening his classroom culture.” “We are going to watch a nine minute video clip from Chuck’s classroom. Before we enter the room, however, imagine you’re his principal and you’re putting together materials for a day that will include several observations. Before entering his classroom, you’ll want to review his goals, and take a quick look at where they exist in the rubric, so you have a focus for when you walk into Chuck’s room. What exactly are you going to be looking for?” “I’d like everyone to take out Handout 7, which is an excerpt from the Teachers Rubric, and turn to an elbow partner and do this thinking together. This excerpt includes the first two Standards for Effective Teaching Practice and their related Indicators. Once again, you’ll see a few elements that have been circled for you—these elements align pretty closely to each of Chuck’s two goals. “As a pair, take a few minutes to familiarize yourselves with the relevant Indicators, and jot down two areas of focus for this observation. You’ll see a space for this on the Observation Collection Tool just above the note taking box, titled Intended Observation Focus. Make sure you agree on what you’re both going to look for. Your focus doesn’t have to be very detailed—just a general sense of what type(s) of practice you expect to see from Chuck, given one or both goals. Everyone take a few minutes to do this.” After three to five minutes, bring the group back together. “Okay. You’ve got your foci. Handout 6 is a blank observation collection sheet on which to jot notes. You can use another method for note-taking if you’d like – whatever is most comfortable to you. “Alright, let’s meet Chuck.” Play the video at this point. After the video has ended, give participants another minute or so to finish up their notes. Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Aligning Evidence to Standards and Indicators Review your notes with a partner Work together to assign each evidence statement to a Standard and Indicator, using the full Teacher Rubric in your handout packet. Explain: “Okay everyone. Quick show of hands—how many of you saw evidence of his student learning goal? How many of you saw evidence of his professional practice goal related to cultivating a safe and collaborative learning environment? “Now that you have your notes, take about 10 minutes and work with your partner to compare your notes from the video. You started with the same focus but probably noted additional things outside that focus as well. Using descriptions of practice related to Standards I and II, located in your rubric excerpt, work together to assign your evidence statements to a Standard and Indicator, like was shown on Handout 3.” Give participants about 10 minutes to complete this task. After nine minutes, give participants a one-minute warning. Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Evidence of Chuck’s Goals During the video observation, what evidence did you see of: Helping students to analyze, interpret and communicate results of scientific investigations and translate numerical or technical information into words. Creating a safe and collaborative learning environment Explain: “Let’s have a couple of groups share out some evidence they saw of Chuck’s goals.” Ask: “What evidence did you see of Chuck’s student learning goal related to teaching students how to translate quantitative or technical information expressed visually (e.g., a table or chart) into words using appropriate terminology and phrases? Where in the rubric does this kind of instructional practice reside?” Allow a few groups to respond.   Ask: “What evidence did you see of Chuck’s professional practice goal related to creating a safe and collaborative learning environment? Which Indicators or elements in the rubric describe this practice?” Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Draft Feedback for Chuck Draft feedback for Chuck based on the evidence you recorded Come to consensus with your table on two to three pieces of feedback for Chuck Explain: “Last step. We’re still in the principal’s shoes. You’ve now returned to your office later in the day and are reviewing your notes from Chuck’s classroom. Using the evidence you compiled from this observation, I’d like you to work with your partner for the next 10 minutes to draft two to three pieces of feedback for Chuck. “Remember to consider the key components of effective feedback during this last step: feedback should be FOCUSED on what you observed, EVIDENCE-BASED, and CONSTRUCTIVE. You’ve already identified places in the rubric that align with what you observed, so use those descriptors of practice to help ground your feedback. Any questions?” Give participants 10 minutes to finish this task. Common facilitation challenges and solutions: Participants will struggle to align observation statements with standards and indicators on the rubric. Make sure as the facilitator you know the basic types of observation statements to be made about the video and the standards and indicators they align with. Guide the participants toward these kinds of observation statements/evidence and then toward the rubric. Participants create feedback that is still too general, unhelpful, or not constructive. Remember that giving good feedback takes a lot of practice. Invite groups to remember the 4 aspects of good feedback and for members of the group to practice giving feedback about feedback using these principles. After participants have had 10 minutes to draft their feedback for Chuck, bring the entire group back together for a brief whole group discussion around what kinds of evidence-based feedback would be constructive and helpful to Chuck going forward. Guiding questions for discussion: What are a two to three pieces of feedback you would give to Chuck about this observation? Learning Wrap-up/Debrief 4 (5 minutes) Ask: “Today’s module introduced the idea of conducting short, unannounced observations as part of teacher evaluation. You were even able to practice conducting a couple of shorter observations that had a targeted focus. How did this experience of conducting a short observation differ from what you previously did at your school?” Allow participants to answer. Ask: “How did our discussion of feedback help to expand your understanding of effective feedback?” Ask: “In today’s module we put together a lot of different pieces of the 5 Step Evaluation Cycle, including goals, evidence, the Model System Teacher Rubric and feedback. What is the main takeaway you are going to bring back to your school as a result of this Module?” Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Suggested Homework for School Leadership Teams Discuss and plan what an annual schedule of observations might look like in your school context. Consider the number of evaluators, number of teachers, and number of school days. Consider setting deadlines for completing the first observation of all teachers. Consider having evaluators set a goal for observation completion, like Kim Marshall did. Suggested Homework School leadership teams and evaluators can begin planning for implementation by creating an annual schedule of observations at the school. Explain: “While the homework is certainly not mandatory, we think it’s a valuable opportunity for school leadership teams to continue the discussion and work we started today. Your suggested homework is to begin planning an annual schedule for observations that takes into account the number of educators and evaluators at your school.” The homework assignment for school leadership teams to present to all school-based educators is described on Slide 31. This slide does not need to be covered with this group of participants. Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Suggested Homework for All School-Based Educators Try conducting a short observation of a colleague Ask your colleague to observe you as well Debrief/discuss the evidence that was gathered and what was challenging about the process. This slide does not need to be covered with this group of participants. Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Feedback and Questions Please take a moment to complete the feedback form being handed out. We appreciate your feedback! Questions? About this training: E-mail [FACILITATOR/VENDOR EMAIL HERE] About educator evaluation more generally: E-mail EducatorEvaluation@doe.mass.edu Come to Closure When the homework assignment has been covered, transition to the final wrap-up, hand out the feedback form for the session, direct participants as to where they can send questions via email, and dismiss the group. Keep this final slide on the screen as participants start to gather their things and leave the session. NOTE: Enter your email address on this final slide, so participants know where to send questions. Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Suggested Homework for School Leadership Teams Review and discuss your district/school improvement plan and compare it with the Standards and Indicators to determine alignment. Begin to identify the type of evidence and data that are currently being collected that could help inform progress toward Educator Plans and goals. Explain: “While the homework is certainly not mandatory, we think it’s a valuable opportunity for school leadership teams to continue the discussion and work we started today. Your suggested homework is to review and discuss your district/school improvement plan and compare it with the Standards and rubrics to determine alignment. Begin to identify the type of evidence and data that is currently being collected that could help inform progress toward Educator Plans and goals that align to your school improvement plan.” Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education

Suggested Homework for All School-Based Educators Work with a colleague, in a small group or at a faculty meeting, to select a particular Standard from the Model System Teacher Rubric, and identify potential sources of evidence (data) to inform performance decisions. Discuss whether these data are easily accessible and comparable across classrooms. The homework assignment for school leadership teams to present to all school-based educators is described on Slide 35. This slide does not need to be covered with this group of participants. Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education