1. NEA and AFT Item Review Boot Camp Working Session: Mathematics

2. Session Objectives Participants will:
Learn the process for PARCC State Educator item review
Practice reviewing and making recommendations for sample items
This session will guide participants through an abbreviated training similar to the training given to new PARCC State Educator item reviewers

3. Mathematics Task Types
Type I
Based on Sub Claims A, B, and E: The student solves problems involving the Major, Additional, and Supporting Content for the grade/course with connections to the Standards for Mathematical Practice, and demonstrates fluency in areas set forth in the Standards for Content in grades 3-6.
Type II
Based on Sub Claim C: The student expresses grade/course-level appropriate mathematical reasoning by constructing viable arguments, critiquing the reasoning of others and/or attending to precision when making mathematical statements.
Type III
Based on Sub Claim D: The student solves real-world problems with a degree of difficulty appropriate to the grade/course by applying knowledge and skills articulated in the standards for the current grade/course

4. Mathematics Review Considerations/Criteria
Does the task measure the intended evidence statement(s)?
Does the task measure the intended mathematical practice(s)?
Is the task mathematically correct and free from errors?
Is the wording of the task clear, concise, and grade-level appropriate?
Are the graphics/stimuli in the task clear, accurate, appropriate for the task, and appropriate for the grade?
Do each prompt and all associated graphics/stimuli contribute to the quality of the task?
Is the scoring guide/rubric clear, correct and aligned with the expectations for performance that are expressed in the task?
General review criteria for math reviewers

5. Step 1
Read and work each task independently.

6. Review Tasks Independently Before Discussing Them as a Group
Reviewers should:
review each task independently
record your comments in the comments section
preliminary decision on how to proceed with the task (accept, accept with edits, reject)
A task should be edited if it has a flaw that can be fixed or needs clarification. A task should only be rejected if it has a flaw that can not be addressed.
Each task will be discussed as a group after individual reviews. Preliminary decisions may be changed at that time.
Comments are not required for every item. You only need to add comments if you have concerns about an item. 6

7. Step 2 Verify that the task aligns to the intended evidence statement.
For type I, the evidence statement is the standard or part of the standard. For types II and III, see the task models.

8. Alignment to Evidence Statements and the CCSS
Each task should:
assess the designated evidence statement
conform to the content clarifications, limits, and emphasis associated with the evidence statement
Reviewers should:
note alignment issues in the comments section
accept the task with edits if the task can easily be edited to make the task align to the evidence statement
reject the task if the task can not easily be edited to make the task fit the evidence statement
align with one or more of the Common Core State Standards associated with the evidence statement 8

9. Step 3
Determine if there are any mathematical flaws in the task (i.e. anything that makes the task unusable as written).

10. Flaws Each task should:
contain content (text, stimuli, terminology, notation, art, etc.) that is
mathematically correct, precise, and generally accepted by math educators
be free from flaws
not contain unintended mathematical errors, misconceptions,
contradictions, or ambiguities

11. Step 4 Review answer keys and rubrics for accuracy.
Is the key correct?
TP: Rubrics only required for tasks of type II and III. Is the rubric clear enough to the scorer so that they will know how/where to assign points? Rubrics for type II and III should have an emphasis on the work (engaging in the practices of reasoning and modeling) and minor emphasis on the computation/answer.

12. Answer Keys and Scoring Rubrics
Type I one-point tasks should:
have the correct key
Scoring Rubrics should:
be clear enough so that the person scoring the response will know how to assign points based on different parts of the response
assign at least 50% of the total points to the reasoning/modeling provided in the response and less than 50% of the points to a computations provided in the response for Type II and Type III tasks
Is the key correct?
TP: Rubrics only required for tasks of type II and III. Is the rubric clear enough to the scorer so that they will know how/where to assign points? Rubrics for type II and III should have an emphasis on the work (engaging in the practices of reasoning and modeling) and minor emphasis on the computation/answer.

13. Step 5
Advise to accept or reject the task.

14. Advise to Accept or Reject
Reviewers as a group
discuss any major comments
determine how to proceed with the task (accept, accept with edits, reject)
A task should be edited if it has a flaw that can be fixed or needs clarification. A task should only be rejected if it has a flaw that can not be addressed. 14

15. Questions?