C H A P T E R 3 Rubrics Chapter 3 Rubrics

What Is a Rubric? The criteria used for judging students’ performance Guidelines by which a product is judged (Wiggins 1996) Explain the standards for acceptable performance (Herman, Aschbacher, and Winters 1996) Ways for teachers to communicate expectations for students’ performance

Benefits of Using Rubrics Increase the consistency of scoring. Rubrics can improve instruction. They let students know what is important. They help students make sense of the learning.

Criteria for Rubrics Clear criteria are essential in performance-based assessments so students’ work can be judged consistently (Arter 1996). Product criteria are what students produce or what they are able to do at a certain time. Process criteria are critical elements necessary for correct performance. Progress criteria indicate how much students have improved on a learning curve. When using product criteria to judge student performance also include process criteria.

Simple Scoring Guides Checklists (performance lists) contain a list of criteria that must be seen for performance or product to meet expectations. Good for peer or self-assessments. Scorers concerned only about demonstration of the characteristic. Keep items on a checklist to a minimum— too many make the checklist difficult to use. (continued)

Simple Scoring Guides (continued) When using peer checklists, teach the students how to observe. Make sure that students understand what all the terms mean. Show examples of the skill being done correctly. Show examples of what an error looks like. Emphasize that the student is trying to help the partner improve skill, not improve just for a grade. Younger students should have fewer items.

Point System Scoring Guides Similar to checklists but they award points for the various criteria on the list. There’s no judgment of quality. Offers students feedback based on points. Teachers can add up points earned and convert to a grade. More points can be awarded for a certain trait to provide some emphasis.

Analytic Rubrics Used with formative assessments. Quantitative analytic rubrics require the scorer to give a numerical score. Traits are evaluated for quality. Use words such as never, sometimes, usually, or always. Qualitative analytic rubrics provide verbal descriptions for each level of various traits to be evaluated. Numbers are easier to remember than verbal descriptions.

Holistic Rubrics Consist of paragraphs written to describe various levels of performance. Scorers decide which level best describes the overall quality of students’ work. Faster to use than analytic rubrics. Useful for assigning grades. Used for summative evaluations or large- scale testing. Give limited feedback to students.

Generalized and Task-Specific Rubrics Generalized rubrics are used to assess a variety of assessments. Tend to look at big-picture characteristics. Game play rubric looks at elements common to all games of same type (e.g., net games). Task-specific rubrics contain criteria unique to the assessment. Targets specific knowledge or behaviors teachers want to observe.

Deciding How Many Criteria to Include Simpler assessments require less complex rubrics. Include all essential elements and hold the extras. Descriptors, traits, characteristics are all terms used to indicate elements on a rubric. Criteria must distinguish between useful indicators and genuine criteria (Wiggins 1998a). (continued)

Deciding How Many Criteria to Include (continued) Include all important components of performance. Avoid details that are insignificant. Write in language that the user can understand. Take into account contextual variables. Link to instructional objectives. Reflect best practice or professional opinion. Address every task and component of the assessment.

Deciding How Many Levels to Write A rubric should have a minimum of four levels. Developmental rubrics are used for all grade levels and use several levels to show continuum of performances. Might be used for statewide assessment. Writing an even number of levels eliminates the temptation to select the middle score (Wiggins 1996).

How to Create Rubrics Identify what needs to be assessed. Envision desired student performance. Identify the main characteristics. Develop descriptors for a quantitative rubric. Pilot the assessment. Develop levels for the rubric. Create a rubric for students. Administer the assessment. Revise the rubric.

Step 1: Identify What Needs to Be Assessed What is the purpose of this assessment? What am I assessing? What information do I need from this assessment? Review instructional goals. If teachers fail to clearly identify content, purpose, and information the assessment is to provide, the resulting rubric will be deficient.

Step 2: Envision the Desired Student Performance Must decide what will be accepted as evidence that the objectives have been met. Develop mental picture of expected student performance. Helps establish a target for final product. Having a clear picture of final expectations will make subsequent steps easier.

Step 3: Identify the Main Characteristics Brainstorm several words that could become descriptors for rubric. Write short sentences or phrases that indicate student competency. Evaluate entire list and group similar criteria.

Step 4: Develop Descriptors (for Quantitative Rubrics) Develop descriptors for levels that apply to all statements in quantitative rubric. For example, the words sometimes, often, always, never describe various levels. (continued)

Step 4: Develop Descriptors (for Quantitative Rubrics) (continued)

Step 5: Pilot the Assessment and Rubric Will help determine if correct descriptors were chosen. Provides teacher with examples of students’ work for developing levels for qualitative or holistic rubrics. Use rubric to provide feedback to students, not for a grade during the pilot. Pilot is necessary for seeing if criteria were set at an appropriate level of difficulty.

Step 6: Develop Levels (for Qualitative Rubrics) Condense the list of descriptors. Group those alike and create essential categories. Target approximately six characteristics. Easiest if levels emerge from student products obtained from the pilot. (continued)

Step 6: Develop Levels (for Qualitative Rubrics) (continued) Levels for a rubric should do the following: Distinguish between best performance and least successful. Use descriptors that provide a true distinction between performance levels. Explain to students what is expected. Focus on quality, not quantity. Avoid comparison words such as more than, better, or worse.

Step 7: Create a Separate Rubric for Students (for Qualitative Rubrics) Only for qualitative rubrics. Depends on the complexity of the assessment. Separate rubric may need to be written more clearly for students without revealing information that is being assessed.

Step 8: Administer the Assessment Administer assessment to students. Use it to evaluate students’ achievement. Student rubric should be attached to assessment.

Step 9: Revise the Rubric Look at students’ performance. Look for ways to strengthen the rubric. Continue to revisit rubrics and criteria to refine and improve them.

Special Considerations to Address When Creating Rubrics Validity Reliability Transparency Subjectivity Without attention to these items, the value of assessment can be jeopardized.

Validity Valid assessments measure what they intend to measure. Performance-based assessments tend to have strong face validity. Validity problems arise if the rubric includes the wrong characteristics or omits items.

Reliability Assessments are reliable when they consistently produce the same results. Scores from different evaluators should be similar. The more clear and simple the rubric, the greater the reliability (Wiggins 1996). Drift is change of criteria between scorers. Reliable rubrics prevent problems of drift because scorer has something concrete to reference.

Transparency Rubrics have to have enough detail so that evaluators can score reliably. Often includes putting some answers in the scoring guide. Teacher should create a separate scoring guide for students when the scorer’s rubric contains too much information.

Subjectivity Subjectivity is the amount of judgment used in assigning a score to a student’s test performance. Subjective tests require judgment, analysis, and reflection on the part of the scorer (Fredrickson and Collins 1989). Involves making decisions based on professional judgment. Professional judgment is the more accurate term than subjectivity to describe evaluation of performance-based assessments. (continued)

Subjectivity (continued) With training, people can use professional judgment to evaluate performance-based assessments in a valid and reliable manner.

Rubric Guidelines Use samples of students’ work. Give rubrics with the assessment. Have students rewrite the rubric. Allow for multiple correct answers. Limit the scope of the assessment. Avoid hypergeneral rubrics. Write levels of difficulty for cognitive assessments. Adjust the rubric after the assessment has been scored, not during.

Use Samples of Students’ Work Exemplars are samples of student work that demonstrate acceptable or unacceptable performance. Help explain why a score was assigned. The scorer has concrete examples with which to compare other student’s work. Anchor is a student’s work that gives evaluators something to ground their evaluations on.

Give Rubrics With the Assessment Give students criteria that evaluate the assessment when the task is presented. Help students understand teacher expectations. Student achievement is greater.

Have Students Rewrite the Rubric Discussing rubrics helps students to clarify what is expected. Have students develop checklists to help understand the expectations. Students should not be expected to write the rubric for complex assessments.

Allow for Multiple Correct Answers Teachers must be willing to accept alternative interpretations and performance within the scope of the criteria established. Rubrics should never mandate the process, format, method, or approach that students are to use to complete the assessment (Wiggins 1998a).

Limit the Scope of the Assessment Rubric must capture the range of correct responses without allowing just any answer to be appropriate. If the rubric is complex and too difficult to use, the assessment should be narrowed.

Avoid Hypergeneral Rubrics If the criteria is too general to distinguish the levels, the rubric is of little value. They don’t help students learn and are susceptible to poor reliability by the person using it to evaluate (Popham 2003). The unacceptable level is where teachers struggle with hypergeneral descriptions.

Writing Levels of Difficulty for Cognitive Assessments Teachers must address each level when writing a rubric. The more levels written, the smaller the difference between levels. Do not have more levels than necessary. Use students’ work to help determine the number of levels needed.

Adjust the Rubric After the Assessment Has Been Scored, Not During Even if omissions or errors have been made on the rubric, continue to evaluate using the criteria given to the students. Adjust the rubric after the assessment and make any necessary changes for future use. Changes in the rubric must also be addressed in the criteria given to students for a specific task.