Designing Scoring Rubrics

What is a Rubric? Guidelines by which a product is judged Guidelines by which a product is judged Explain the standards for acceptable performance or work Explain the standards for acceptable performance or work Points out what is significant or important in learning Points out what is significant or important in learning Criteria for consistent evaluation Criteria for consistent evaluation Guide students towards improved performance Guide students towards improved performance Provide specific feedback to students on how to improve performance Provide specific feedback to students on how to improve performance Without a standard of evaluation, assessments are no more than instructional tasks Without a standard of evaluation, assessments are no more than instructional tasks

Criteria for Rubrics Descriptors, traits, characteristics are terms used to indicate the elements contained in rubrics. Descriptors, traits, characteristics are terms used to indicate the elements contained in rubrics. Reflect the elements of good performance and become the source for the criteria used to judge the performance. Reflect the elements of good performance and become the source for the criteria used to judge the performance. Three types of criteria to determine grades: Three types of criteria to determine grades: Product, Process, Progress Product, Process, Progress

Product Criteria What students produce (pamphlet, brochure, play a game, create a game, perform a routine). What students produce (pamphlet, brochure, play a game, create a game, perform a routine). Performance and projects are examples of products. Performance and projects are examples of products. Rubric contains critical elements necessary for required content. Rubric contains critical elements necessary for required content. Stating performance outcomes in terms of products says nothing about application or correct form. Stating performance outcomes in terms of products says nothing about application or correct form. These are process criteria. These are process criteria.

Process Criteria Critical elements necessary for correct performance. Critical elements necessary for correct performance. Outlines the processes that students use to learn. Outlines the processes that students use to learn. In Physical Education process criteria refers to the quality of physical performance. In Physical Education process criteria refers to the quality of physical performance. Self-check, peer evaluation, teacher evaluation. Self-check, peer evaluation, teacher evaluation. Evaluating process says nothing about how much a student has improved. Evaluating process says nothing about how much a student has improved. These are progress criteria. These are progress criteria.

Progress Criteria Determine how much a student has improved or progressed. Determine how much a student has improved or progressed. Determines the mastery of the critical elements of performance. Determines the mastery of the critical elements of performance. In order to measure progress teachers must administer assessments more than once. In order to measure progress teachers must administer assessments more than once.

Types of Rubrics The type of rubric chosen for assessment depends on the task being evaluated and the needs of the assessor. The type of rubric chosen for assessment depends on the task being evaluated and the needs of the assessor. Analytic and holistic rubrics are very complex and require time to generate. Analytic and holistic rubrics are very complex and require time to generate. Checklists are the easiest to create. Checklists are the easiest to create. Point System rubrics are easy to convert to grades. Point System rubrics are easy to convert to grades.

Checklists Characteristics or behaviours that are basically scored as Yes/No ratings. Characteristics or behaviours that are basically scored as Yes/No ratings. A list of characteristics regarding the performance or product is written and the scorer decides whether the trait is present or not. A list of characteristics regarding the performance or product is written and the scorer decides whether the trait is present or not. Evaluate critical elements or process criteria of motor skills where quality is not a factor. Evaluate critical elements or process criteria of motor skills where quality is not a factor. Journals or portfolios can be assessed to see if they have the presence of various points without making judgments about the quality of the response. Journals or portfolios can be assessed to see if they have the presence of various points without making judgments about the quality of the response. Checklists generally contain more traits or descriptors than usually found in other scoring guides. Checklists generally contain more traits or descriptors than usually found in other scoring guides. This detail is helpful in guiding student learning. This detail is helpful in guiding student learning.

Example: Checklist Rubric YesNo ________Plays within the rules of the game. ________Does not argue with others. ________Respect’s the other team’s efforts. ________Accepts the calls of officials.

Example: Self-Assessment ____I play within the rules of the game. ____I don’t argue with others. ____I offer encouragement and support. ____I accept the outcome of the game.

Point System Rubrics Similar to checklists they award point for the various criteria on the list. Similar to checklists they award point for the various criteria on the list. Judgment of quality is not required. Judgment of quality is not required. Provides students with feedback. Provides students with feedback. Points can be added up and converted to a grade. Points can be added up and converted to a grade. Certain traits or characteristics can be weighted giving them greater emphasis. Certain traits or characteristics can be weighted giving them greater emphasis.

Example: Scoring Rubric Fitness Portfolio ___ Fitness Evaluation (8 points) Cardiovascular ___ Cardiovascular Assessment (4 points) ___ Abdominal Endurance (2 points) ___ Flexibility Assessment (2 points) ___ Fitness Plan (18 points) ___ Calculate target heart rate (2 points) ___ Needs analysis (4 points) ___ Workout Plan (12 points) ___ Warm-up and recording chart (3 pts + 1 point for chart) ___ Aerobic plan and chart (3 pts + 1 point for chart) ___ Strength plan and chart (3 pts + 1 point for chart)

Analytic Rubrics Requires the scorer’s judgment to determine the degree of quality. Requires the scorer’s judgment to determine the degree of quality. Evaluate the strength or weakness of a trait or element. Evaluate the strength or weakness of a trait or element. Students can easily see areas on which they must improve. Students can easily see areas on which they must improve. These rubrics take more time to score. These rubrics take more time to score. Words give a verbal indication of the degree of quality. Words give a verbal indication of the degree of quality. Always – Frequently – Sometimes – Never Always – Frequently – Sometimes – Never

Quantitative Analytic Rubric Game Play Assessment for Tennis The Student: 1.Uses correct form on the forehand. 2.Places shots. 3.Moves into position quickly. 4.Returns to base position. 5.Calls shots correctly. 6.Knowledge of the rules is evident. Never Sometimes Frequently Always Useful for providing feedback to students and assessing their ability to utilize skills in an applied setting.

Qualitative Analytic Rubric Rules for Tennis 1Is unfamiliar with the rules. Depends on opponents or partner for instruction. Struggles with most questions. Is unsure of serving order and rotation. 2Shows some evidence of knowing rules. Struggles with some questions. Serving order and rotation are correct. May have a few errors. 3Shows evidence of usually knowing and applying rules. Can answer most questions when asked. Serving order and rotation are correct. 4Shows evidence of thoroughly knowing and applying rules. Can answer any question when asked. Serving order and rotation are correct. Provide verbal descriptions of teacher expectations and are useful for providing formative feedback about several elements important to playing a game or performing well.

Holistic Rubrics – Useful for Summative Evaluations Proficient Level Consistently… Demonstrates the ability to select and successfully use the appropriate skills. Demonstrates good skill form. Moves to cover the appropriate space. Anticipates the offensive and defensive play and selects the appropriate response. Applies and follows the appropriate rules and scoring. Follows the rules of etiquette and fair play. Competent Level Demonstrates the ability to select and use the appropriate skills the majority of the time. Demonstrates good skill form a majority of the time. Attempts to move to cover the appropriate space a majority of the time. Selects the appropriate offensive and defensive strategy and the appropriate response most of the time. Consistently applies and follows the appropriate rules and scoring. Demonstrates good sporting behaviour consistently and follows game etiquette most of the time. Novice Level Inconsistently selects and uses appropriate skills. Frequently demonstrates incorrect skill form. Does not move to cover the appropriate space on the court. Little evidence of applying the appropriate strategy and response to a situation. Applies and follows appropriate rules and scoring. Inconsistent demonstration of rules of etiquette or good sporting behaviour.

Considerations to Address when Creating Rubrics Validity – Does the rubric measure what it claims to measure. Validity – Does the rubric measure what it claims to measure. Reliability – Does the assessment consistently produce the same results. Reliability – Does the assessment consistently produce the same results. Transparency – Criteria are clear enough to students so that they can assess themselves and others with roughly the same reliability. Transparency – Criteria are clear enough to students so that they can assess themselves and others with roughly the same reliability. Subjectivity – The amount of judgment used to assign a score to a student’s performance. Subjectivity – The amount of judgment used to assign a score to a student’s performance.

Rubric Hints and Guidelines Use samples of student work - samples show what is possible but should not limit the potential for excellence. Use samples of student work - samples show what is possible but should not limit the potential for excellence. Share rubrics with students - criteria helps students understand teacher expectations. Share rubrics with students - criteria helps students understand teacher expectations. Let students create some rubrics - students are more likely to understand important criteria. Let students create some rubrics - students are more likely to understand important criteria. Allow for multiple correct answers - sometimes the correct response is “it depends”. Allow for multiple correct answers - sometimes the correct response is “it depends”. Frequency is not the sole indicator of quality – too often when frequency is included in a rubric, the quality of the performance is not addressed. Frequency is not the sole indicator of quality – too often when frequency is included in a rubric, the quality of the performance is not addressed.

Rubric Hints and Guidelines Limit the scope of the assessment – complex rubrics are cumbersome and often capture non- critical elements. Limit the scope of the assessment – complex rubrics are cumbersome and often capture non- critical elements. Consider the levels of difficulty – give credit for attempting more difficult tasks. Consider the levels of difficulty – give credit for attempting more difficult tasks. Determine the number of levels to develop – the more levels, the smaller the difference between levels, and reliability decreases. Determine the number of levels to develop – the more levels, the smaller the difference between levels, and reliability decreases. Adjust the rubric after, not during the assessment – changing the rules while the game is being played is not fair practice. Adjust the rubric after, not during the assessment – changing the rules while the game is being played is not fair practice.