RUBRIC DEVELOPMENT Why use rubrics? Characteristics of good rubrics Definition & types of rubric Strategies for rubric development SOLO taxonomy Phenomeno- graphic sorting Using rubrics with students

 Definition  A scoring tool that lays out the specific expectations for an assessment task (Stevens & Levi, 2005)  A set of clear explanations or criteria used to help teachers and students focus on what is valued in a subject, topic, or activity (Russell, & Airasian, 2012).  Components of a rubric:  Criteria/Indicator  aspects of an assessment task which the assessor takes into account when making their judgment  May use different weightings for different criteria  Level of Attainment  often use grade level descriptors  Types of rubrics:  Descriptive (Analytic), Holistic  Why and when we use particular types of rubrics SCORING RUBRIC

WHY USE RUBRICS? For teachers: Prompt a criterion-referenced assessment Provide students with detailed and timely feedback Encourage critical thinking Facilitate communication with others involved in scoring Help to refine teaching skills/learning activities For students: Clarify the teacher’s expectations of student performance Provide informative descriptions of expected performance Help to monitor and critique own work

 Well defined  Clearly describe the expected level of student performance for each criterion in a rubric  Avoid general evaluative words (poor, excellent, etc.)  Use specific objective terms (correctly identifies, uses only basic vocabulary, chooses incorrect formula… )  Context specific  Describe what teachers expect from student for a given performance or work product on a particular subject domain  Viable for instruction CHARACTERISTICS OF GOOD RUBRICS (1)

 Finite and exhaustive  Every response must be scorable  Too many score levels is confusing for students and causes disagreement among teacher scores  Ordered  Represent the different levels of learning targets as defined by LP  Related to Common Core theme/strand  Coherent with the cognitive complexity of the standards CHARACTERISTICS OF GOOD RUBRICS (2)

Components: (1)Task description (2)Assessment criteria (3)Performance levels Task Description: Criteria 1 Criteria 2 Criteria 3 Criteria 4 Total Level 5 Level 4 Level 3 Level 2 Level 1 ANALYTIC (DESCRIPTIVE) RUBRICS Advantage:  Provides judgment on each criterion Disadvantage:  Time consuming to make

Task Description: Criteria Level 5Overall description of Level 5 Level 4Overall description of Level 4 Level 3Overall description of Level 3 Level 2Overall description of Level 2 Level 1Overall description of Level 1 HOLISTIC RUBRICS  A single scale with all criteria to be included in the evaluation being considered together  Based on an overall judgment of student work Advantage:  Saves time in developing and scoring Disadvantage:  Does not provide specific feedback for improvement

 On performance-based tasks:  extended response items  projects  presentations  portfolios WHEN TO USE RUBRICS

 Reflecting on the task & content  Learning outcomes of the unit and the particular assessment  What we want from the students, why we created this assessment, what our expectations are  Listing the learning outcomes & expectations  Focus on the particular details of the task and what specific learning objectives we expect to see in the completed task  Grouping & labeling the outcomes & criteria  Organize the results of reflections, group similar expectations together to become the rubric Indicators  Application of a rubric format  Apply the templates & descriptions to the final form of the rubrics 4 KEY STAGES IN CREATING RUBRICS

 Relationship between the intended Learning Progression (LP) and rubrics  Direct use of the LP as standard reference  Use of a general strategy (i.e. modified Bloom’s taxonomy or SOLO taxonomy)  Assign the target outcome as mapped onto the Cognitive Rigor Matrix as the maximum level of the rubric  Use the SOLO taxonomy as scoring  Phenomenographic sorting STRATEGIES FOR RUBRIC DEVELOPMENT

SOLO TAXONOMY Possible Score LevelResponses that … 4Extendednot only include all relevant pieces of information, but also extend the response to integrate relevant pieces of information not in the stimulus. 3Relationalintegrate all relevant pieces of information from the stimulus. 2Multistructuralrespond to several relevant pieces of information from the stimulus. 1Unistructuralrespond to only one relevant piece of information from the stimulus. 0Pre-structuralconsist only of irrelevant information. * Modified from Wilson (2005, p. 75)  A possible value of 0 – 4 can be used to score each question

1.Sort student responses into performance level piles (excellent, good, …, and poor), or in terms of the levels of understanding of the responses 2.Describe similarities within a pile and differences between  These similarities and differences can inform the different level of rubrics 3.Do sorting in pairs  To reconfirm matches & mismatches  Allow dialogue to maximize the effectiveness of rubric development PHENOMENOGRAPHIC SORTING

 Explain what the test will emphasize  Emphasis and expectations will be delineated in the assessment criteria in the rubrics  Inform students how the assessment will be scored  Explain what each of the assessment criteria defined in the rubrics means  Explain how the results will be used  Explain the importance of the test scores in the student’s learning progression USING RUBRICS WITH STUDENTS

 Make sure that the wording of the rubrics is understandable to students (simplify wording for lower grades).  Works best with holistic rubrics (or with a combined version of analytic descriptions).  Provide rubrics to students in advance of the administration of the assessment.  Alternatively, students can be provided with a general rubric and a small number of papers (names removed). Students can score the papers in small groups; groups are required to come to consensus on the grade to be assigned. HOW TO USE RUBRICS WITH STUDENTS

 Nitko, A. J., & Brookhart, S. (2007). Educational assessment of students. Upper Saddle River, NJ: Pearson Education, Inc.  McMillan, J. H. (2007). Classroom assessment. Principles and practice for effective standard-based instruction (4th ed.). Boston: Pearson - Allyn & Bacon.  Oregon Department of Education. (2014, June). Assessment guidance.  Popham, W. J. (2014). Criterion-referenced measurement: A half-century wasted? Paper presented at the Annual Meeting of National Council on Measurement in Education, Philadephia, PA.  Popham, W. J. (2014). Classroom assessment: What teachers needs to know. San Francisco, CA: Pearson  Russell, M. K., & Airasian, P. W. (2012). Classroom assessment: Concepts and applications. New York, NY: McGraw-Hill.  Stevens, D. & Levi, A. (2005). Introduction to rubrics. As assessment tool to save grading time, convey effective feedback, and promote student learning. Sterling: Stylus Publishing, LLC  Wihardini, D. (2010). Assessment development II. Unpublished manuscript. Research and Development Department, Binus Business School, Jakarta, Indonesia.  Wilson, M. (2005). Constructing measures: An item response modeling approach. Mahwah, NJ: Lawrence Erlbaum Associates. BIBLIOGRAPHY

Rubrics PPT by the Oregon Department of Education and Berkeley Evaluation and Assessment Research Center is licensed under a CC BY 4.0.Oregon Department of EducationBerkeley Evaluation and Assessment Research CenterCC BY 4.0 You are free to:  Share — copy and redistribute the material in any medium or format  Adapt — remix, transform, and build upon the material Under the following terms:  Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.appropriate creditindicate if changes were made  NonCommercial — You may not use the material for commercial purposes.commercial purposes  ShareAlike — If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original.same license Oregon Department of Education welcomes editing of these resources and would greatly appreciate being able to learn from the changes made. To share an edited version of this resource, please contact Cristen McLean, CREATIVE COMMONS LICENSE