NEA and AFT Item Review Boot Camp Working Session: Mathematics

Session Objectives Participants will: Learn the process for PARCC State Educator item review Practice reviewing and making recommendations for sample items This session will guide participants through an abbreviated training similar to the training given to new PARCC State Educator item reviewers

Mathematics Task Types Type I Based on Sub Claims A, B, and E: The student solves problems involving the Major, Additional, and Supporting Content for the grade/course with connections to the Standards for Mathematical Practice, and demonstrates fluency in areas set forth in the Standards for Content in grades 3-6. Type II Based on Sub Claim C: The student expresses grade/course-level appropriate mathematical reasoning by constructing viable arguments, critiquing the reasoning of others and/or attending to precision when making mathematical statements. Type III Based on Sub Claim D: The student solves real-world problems with a degree of difficulty appropriate to the grade/course by applying knowledge and skills articulated in the standards for the current grade/course

Mathematics Review Considerations/Criteria Does the task measure the intended evidence statement(s)? Does the task measure the intended mathematical practice(s)? Is the task mathematically correct and free from errors? Is the wording of the task clear, concise, and grade-level appropriate? Are the graphics/stimuli in the task clear, accurate, appropriate for the task, and appropriate for the grade? Do each prompt and all associated graphics/stimuli contribute to the quality of the task? Is the scoring guide/rubric clear, correct and aligned with the expectations for performance that are expressed in the task? General review criteria for math reviewers

Step 1 Read and work each task independently.

Review Tasks Independently Before Discussing Them as a Group Reviewers should: review each task independently record your comments in the comments section preliminary decision on how to proceed with the task (accept, accept with edits, reject) A task should be edited if it has a flaw that can be fixed or needs clarification. A task should only be rejected if it has a flaw that can not be addressed. Each task will be discussed as a group after individual reviews. Preliminary decisions may be changed at that time. Comments are not required for every item. You only need to add comments if you have concerns about an item.   6

Step 2 Verify that the task aligns to the intended evidence statement. For type I, the evidence statement is the standard or part of the standard. For types II and III, see the task models.

Alignment to Evidence Statements and the CCSS Each task should: assess the designated evidence statement conform to the content clarifications, limits, and emphasis associated with the evidence statement Reviewers should: note alignment issues in the comments section accept the task with edits if the task can easily be edited to make the task align to the evidence statement reject the task if the task can not easily be edited to make the task fit the evidence statement align with one or more of the Common Core State Standards associated with the evidence statement   8

Step 3 Determine if there are any mathematical flaws in the task (i.e. anything that makes the task unusable as written).

Flaws Each task should: contain content (text, stimuli, terminology, notation, art, etc.) that is mathematically correct, precise, and generally accepted by math educators be free from flaws not contain unintended mathematical errors, misconceptions, contradictions, or ambiguities

Step 4 Review answer keys and rubrics for accuracy. Is the key correct? TP: Rubrics only required for tasks of type II and III. Is the rubric clear enough to the scorer so that they will know how/where to assign points? Rubrics for type II and III should have an emphasis on the work (engaging in the practices of reasoning and modeling) and minor emphasis on the computation/answer.

Answer Keys and Scoring Rubrics Type I one-point tasks should: have the correct key Scoring Rubrics should: be clear enough so that the person scoring the response will know how to assign points based on different parts of the response assign at least 50% of the total points to the reasoning/modeling provided in the response and less than 50% of the points to a computations provided in the response for Type II and Type III tasks Is the key correct? TP: Rubrics only required for tasks of type II and III. Is the rubric clear enough to the scorer so that they will know how/where to assign points? Rubrics for type II and III should have an emphasis on the work (engaging in the practices of reasoning and modeling) and minor emphasis on the computation/answer.

Step 5 Advise to accept or reject the task.

Advise to Accept or Reject Reviewers as a group discuss any major comments determine how to proceed with the task (accept, accept with edits, reject) A task should be edited if it has a flaw that can be fixed or needs clarification. A task should only be rejected if it has a flaw that can not be addressed.   14

Questions?