1. EQuIP Student Work Protocol — Mathematics

2. Session Goals Develop reviewers’ ability to:  Use the EQuIP Student Work Protocol to examine student work and provide evidence-based feedback for both the task and also its lesson/unit. o Develop a common understanding among reviewers of task alignment and quality. o Develop a common understanding of the alignment rating descriptors for the EQuIP Student Work Protocol.

3. EQuIP Quality Review: Principles & Agreements 1.Alignment: Before beginning a review, all members of a review team are familiar with the Common Core State Standards for the grade band targeted. 2.Inquiry: Review processes emphasize inquiry rather than advocacy and are organized in steps around a set of guiding questions. 3.Respect & Commitment: Each member of a review team is respected as a valued colleague and contributor who makes a commitment to the EQuIP process. 4.Criteria & Evidence: All observations, judgments, discussions and recommendations are criterion and evidence based. 5.Respectful and Constructive Feedback: Lessons/units to be reviewed are seen as “works in progress.” Reviewers are respectful of contributors’ work and make constructive observations and suggestions based on evidence from the work. 6.Individual to Collective: Each member of a review team independently records his/her observations prior to discussion. Discussions focus on understanding all reviewers’ interpretations of the criteria and the evidence they have found. 7.Understanding & Agreement: The goal of the process is to compare and eventually calibrate judgments to move toward agreement about quality with respect to college- and career-readiness.

4. Introduction to the Student Work Protocol The objectives of the EQuIP Student Work Protocol are: To analyze student work from a task within a lesson or unit to establish evidence of task alignment with the targeted CCSS. To provide suggestions for improving the task and related instructional materials.

5. When selecting the task: Select a task from a CCSS-aligned lesson/unit. Make sure the task is significant to the central purpose of the lesson/unit. Collect several samples from a cross-section of the student group. If from a longer lesson or full unit, consider selecting multiple tasks that represent different aspects of the lesson/unit. Introduction to the Student Work Protocol

6. The collaborative process: Teams of reviewers are preferred. Work from individual to collective. Discuss and collaborate. NOTE: The lesson/unit developer may or may not be a member of the team. Introduction to the Student Work Protocol

7. STEP 1: Analyze the Task STEP 2: Examine Instructional Context and CCSS Alignment STEP 3: Analyze Individual Student Work STEP 4: Analyze the Collection of Student Work Step 5: Provide suggestions for Improving the Materials Steps for Reviewing Student Work

8. Without consulting the standards or the supporting materials in the lesson/unit, analyze the purpose and demands of the task as evidenced by the directions and/or prompt(s). Record the grade, lesson/unit, and task title on the EQuIP Student Work Protocol Form. Use only the directions and prompts to analyze the requirements of the task without consulting the instructional context and supporting materials in the lesson/unit. Study the task thoroughly, making notes about its purpose and demands and noting apparent aligned standards. [For mathematics this requires actually working the problem(s) and answering the question(s) included in the task.] STEP 1: Analyze the Task

9. Without consulting the standards or the supporting materials in the lesson/unit, analyze the purpose and demands of the task as evidenced by the directions and/or prompt(s). Guiding Questions What content and performance demands does the task make on students? What is the purpose of the task? Which Common Core standards seem to be targeted by the task? What types of student reasoning are required by the task? Which Standards for Mathematical Practice might be assessed by the task? STEP 1: Analyze the Task Record Notes & Observations

10. Notes & Observations Regarding the Demands of the Task: The task addresses whole number operations, particularly addition and multiplication. Students will need to multiply, or skip count, by 3s, 4s and 5s and add whole numbers. They need to be able to decipher and solve word problems and reason mathematically. Question #4 requires that students interpret their results – reason quantitatively. They need to be able to add whole numbers and, at a minimum, interpret multiplication as repeated addition and to navigate a missing factor problem. The task provides all units of measure. STEP 1: Analyze the Task Grade 3 – Cookie Dough

11. Scan the entire lesson/unit, noting its purpose, content, and organization. Notice the placement of the task within the lesson/unit. Identify the standards targeted in the lesson/unit and compare to those identified in Step 1. Examine the answer keys, scoring guidelines, and/or rubrics related to the task. STEP 2: Examine Instructional Context and CCSS Alignment of the Task

12. Scan the lesson/unit to see what it contains and how it is organized. Grade 3 Math: Cookie Dough* Overview and Standardspage 1 Performance Task: Cookie Doughpages 4–5 Rubrics and Scoring Guidespages 6–8 Annotated Student Workpages 9–18 Instructional Supportspages 19–88 Unit Outlinepages 20–26 Full Lesson Planpages 27–85 Referencespages 86–88 *Page number references match the online PDF version of the unit. STEP 2: Examine Instructional Context and CCSS Alignment of the Task – Grade 3: Cookie Dough

13. 13 Use the alignment descriptors to evaluate the alignment between the targeted standards and the task. STEP 2: Examine Instructional Context and CCSS Alignment of the Task * Partial alignment is possible when a standard contains multiple concepts and/or performances in a single standard. In these standards one part may play a more central role in the overarching requirements of the standard. It is possible, for example, that the requirements of one part subsume the requirements of the other part(s). **Limited alignment occurs when the most central aspects of standards containing multiple concepts and/or performances are not clearly addressed by the expectations of the task, but there is sufficient alignment to the less central aspects that continuing the review process is warranted. This could be the case if multiple standards are targeted and other standards have a strong alignment. Reviewers may want to suggest improvements to alignment.

14. Guiding Questions Where does the task occur within the instructional sequence? What have students already learned from the lesson/unit when they approach the task? What will they learn after? Does the lesson/unit include sufficient and effective instruction and scaffolding leading up to the task? Do the expectations described in the scoring guidelines correspond with your analysis of the task in Step 1? Is the task central to the learning goals of the lesson/unit? STEP 2: Examine Instructional Context and CCSS Alignment of the Task Cont.

15. Guiding Questions (cont.) Which standards targeted, including the Mathematical Practices, in the lesson/unit match the content and performance demands of the task? Do the directions, prompts, and/or scoring guidelines for the task adequately provide or indicate opportunities for students to demonstrate the requirements of the targeted standard(s) for the task? STEP 2: Examine Instructional Context and CCSS Alignment of the Task Record Notes & Observations

16. Notes & Observations Regarding Alignment: This task is the centerpiece of the unit and used as the summative assessment. Materials in the unit provide support for both the teacher and the student and include enough scaffolding to support success for the student. Answer keys and scoring rubrics are present and helpful, with extensive scoring information in the annotations. The standards that align with the performance task are 3.OA.1, 3, 4, and 5 and MP.1, MP.2, and MP.3. The task and relevant targeted standards both address operations with whole numbers. There are gaps in alignment between the demands of the task and several of the standards targeted for the overall unit. Several of those standards representing gaps in alignment are addressed elsewhere in the unit, including 3.OA.2, 6, 8, and 9 and MP.4. Only 3.OA.7 is not fully addressed in the task or in the unit. STEP 2: Examine Instructional Context and CCSS Alignment of the Task – Grade 3: Cookie Dough Cont.

17. STEP 2: Examine Instructional Context and CCSS Alignment of the Task – Grade 3: Cookie Dough Cont.

18. STEP 2: Examine Instructional Context and CCSS Alignment of the Task – Grade 3: Cookie Dough

19. Use the table provided to analyze each individual student sample of work, asking the following questions about each: – What does the student’s work demonstrate about his/her understanding of the task? – What does the student’s work demonstrate about his/her proficiency with the requirements of the targeted CCSS? – What does the student’s work demonstrate about the depth of his/her understanding and reasoning ability, including understanding the context of the question(s) and/or proficiency with the Standards for Mathematical Practice? – How does the application of the scoring guidelines/rubrics related to the task support an understanding of the student’s proficiency? STEP 3: Analyze Individual Student Work

20. Use the table provided to analyze each individual sample of work by asking the following questions about each: STEP 3: Analyze Individual Student Work Record Notes & Observations in the Rows of the Table

21. Notes & Observations Regarding the Student Responses to the Task: High-level achieving student (Level 4): Student responses meet the demands of the entire task with few errors and also include explanations that are clear and complete. This student was able to find the correct answers using the most efficient operation, multiplication or division, and make the connection between the two operations. At-standard achieving student (Level 3): For most of the task the students’ responses show the main elements of the performance demands of the task and is an organized attack on the core of the problem. In some cases there were errors in calculations or in interpretation of their own work. Work shown may be limited but appears to show mathematical understanding. In one case, the correct answer is given but not supported by the work shown. In most cases, the student was able to explain the path to his/her solution. STEP 3: Analyze Individual Student Work Grade 3 – Cookie Dough

22. Notes & Observations Regarding the Student Responses to the Task: Below-standard performing student (Level 2): Students show partial understanding of the problem but either misread or misinterpreted some parts of the problem. They were able to get a correct answer for parts 1 and 2 but found an unreasonable answer for part 3. In some cases students were unable to work backward — or understand the inverse relationship between multiplication and division. Explanations are missing, incomplete, or not helpful. Minimally-successful student (Level 1): Students at this level have minimal success. In some cases even when questions are answered correctly, the work shows misunderstandings. Their reasoning is flawed in part 2 of the task and does not support their correct answer. They misinterpreted the question in part 4. Students appear to be stuck at the level of conceptual understanding of multiplication. STEP 3: Analyze Individual Student Work Grade 3 – Cookie Dough

23. STEP 4: Analyze the Collection of Student Work Cont. Look for trends across the collection of samples of student work. Guiding Questions On what aspects of the task have students generally performed well? What are the most frequent and fundamental problems students appear to be having with the task? Are there common errors made across the collection of student work? What does the range of student work demonstrate about the clarity of the task, directions, and supporting materials? In what ways do the scoring guidelines/rubrics aid in the evaluation of student proficiency on the targeted standards?

24. Guiding Questions (cont.) What do the patterns across multiple student work samples indicate about alignment of the task to the targeted standards? In what ways does the task allow (or not allow) students to demonstrate various levels of proficiency* with the targeted standards? Is there evidence of consistent levels of reasoning and understanding across the samples of student work? What does the pattern of student responses show about their common understanding of the mathematical context of the task? STEP 4: Analyze the Collection of Student Work Record Notes & Observations *Note: A range of student understanding of the requirements of the task and its targeted standards, from merely “proficient” to “deep conceptual understanding and reasoning,” might be evident in the student work.

25. Notes & Observations Regarding the Students’ Responses to the Task: There is a full range of response levels in the student work associated with this task. Most students in the collection were able to calculate correctly at least some of the time, either by repeated addition or multiplication. Most were able to justify their results using either diagrams or by showing the calculations. However some explanations were incomplete. Students were least successful with part 4, some not able to reason quantitatively when the answer had to be rounded appropriately. The task provided all measurement units (dollars or tubs) for the student. The scoring guidelines were extensive and helpful. STEP 4: Analyze the Collection of Student Work Grade 3 – Cookie Dough

26. Implications for future task development: Prompts and directions for this type of task should be clear and clearly aimed at the requirements of the standards. For this task, that might include clearly asking for an equation or a specific operation to be used. MP2 would be better assessed if the units were not provided and were, instead, required to be included with the solution. As struggling students move forward, they need to be encouraged to think about using the most efficient strategy and to attend to their process. All students need practice with making clear and complete explanations and justifications. They might benefit from reading and critiquing other students’ (or sample) explanations. Students performing below the standard were often unable to demonstrate an understanding of the inverse relationships between operations. Further development of this concept would be beneficial. STEP 4: Analyze the Collection of Student Work Grade 3 – Cookie Dough

27. Use insights from the alignment process and examination of student work to suggest improvements to the instructional materials. Guiding Questions Are the task instructions clear to students? How could they be modified to increase student understanding of the task expectations? Is the task properly placed within the overall lesson/unit plan? If not, how might it be repositioned? Does the task allow a variety of students to demonstrate their own level of proficiency? What modifications might be made to the task to elicit evidence of various levels of proficiency? STEP 5: Provide Suggestions for Improving the Materials Cont.

28. Guiding Questions (cont.) Do the task prompts, directions, and requirements provide students with a clear opportunity to demonstrate proficiency for the targeted standards? What modifications to the task might elicit better evidence of proficiency on the targeted standards? Does the task allow students to demonstrate deep understanding and reasoning about the related concepts, topics or texts? What modifications to the task might allow students to demonstrate the deep reasoning and understanding ? What modifications to scoring guidelines/rubrics would improve guidance for evaluating student proficiency on the targeted standards? STEP 5: Provide Suggestions for Improving the Materials Record Suggestions for Improvement

29. Suggestions for Improvement for the Task and the Lesson/Unit: At all levels student performance would be improved by making complete explanations and justifications. Better and more detailed prompts could alleviate that problem somewhat. Interpretation of quotients, the most critical aspect of 3.OA.2, is not specifically required in the performance task. However, there is a possibility that students would use this type of thinking for parts 3 and 4. Revising the prompt to specifically require using a quotient would provide alignment with this standard. In this performance task students must use abstract reasoning to solve the problems (MP.2). They decontextualize the verbal description to manipulate the numbers and then re-contextualize when presenting their solution and reasoning. If the units were not provided with each answer line, but were instead expected as part of the answer, alignment to MP.2 would be improved. As it stands, this task does not require modeling (MP.4). If the directions “Write an equation to represent the problem” were added to “Show how you figured it out” on parts 2, 3, and 4, alignment to MP.4 would be improved. Scoring rubrics provided in the unit are strong and useful. STEP 4: Provide Suggestions for Improving the Materials – Grade 3: Cookie Dough

30. Suggestions for Improvement for the Task and the Lesson/Unit: At all levels student performance would be improved by making complete explanations and justifications. Better and more detailed prompts could alleviate that problem somewhat. Interpretation of quotients, the most critical aspect of 3.OA.2, is not specifically required in the performance task. However, there is a possibility that students would use this type of thinking for parts 3 and 4. Revising the prompt to specifically require using a quotient would provide alignment with this standard. In this performance task students must use abstract reasoning to solve the problems (MP.2). They decontextualize the verbal description to manipulate the numbers and then re-contextualize when presenting their solution and reasoning. If the units were not provided with each answer line, but were instead expected as part of the answer, alignment to MP.2 would be improved. As it stands, this task does not require modeling (MP.4). If the directions “Write an equation to represent the problem” were added to “Show how you figured it out” on parts 2, 3, and 4, alignment to MP.4 would be improved. Scoring rubrics provided in the unit are strong and useful. STEP 4: Provide Suggestions for Improving the Materials – Grade 3: Cookie Dough

31. Did we develop a common understanding among reviewers of:  How examining student work, using the Student Work Protocol, can provide another lens through which we can view and provide feedback for a task and its lesson/unit? o How to assess the quality and alignment of an individual task in a lesson/unit? o How the alignment descriptors are used in the Student Work Protocol? Reflect on Session Goals

32. http://www.achieve.org/EQuIP