1. Scoring Rubrics

2. The Outcomes To describe scoring rubrics (SR)
To explain the usefulness of SR
To differentiate types of SR
To discuss process of developing SR

3. What is Scoring Rubrics?
A descriptive scoring scheme developed by assessors to guide the analysis of the products or processes of students’ efforts (Brookhart, 1999).
Basically for performance assessment.
Usually in a matrix form.
Similar to Likert scales.
It contains a mutually agreed upon negotiated contract or criteria for success.

4. What is Scoring Rubrics?
Employed for a judgment of quality.
Used to assess a broad range of subjects and activities.
It focuses on stated objectives and the educational standards.
As active participants, students are aware of the standards.
The assessment is transparent, and fair.

5. Example 1: The Cookie
Task: Make a chocolate chip cookie that I would want to eat.
Criteria: Texture, Taste, Number of Chocolate Chips, Richness
Range of performance:
Delicious(14-16 pts)
Tasty(11-13 pts)
Edible(8-10 pts)
Not yet edible(0-7 pts)

6. Assess The Cookie Overall score By criteria Delicious Tasty Edible
Not yet edible
By criteria
Number of chips
Texture
Taste
Richness

7. The Rubric Delicious 4 Tasty 3 Edible 2 Not yet edible 1 # chips
Chips in every bite
75% chips
50% chips
Less than 50% chips
texture
Consistently chewy
Chewy middle, crispy edges
Crunchy
Like a dog biscuit
color
Even golden brown
Brown with pale center
All brown
Or all pale
Burned
richness
Buttery, high fat
Medium fat
Low-fat flavor
Non-fat flavor

8. Holistic Example Cookie with Delicious level (4) Chips in every bite
Consistently chewy
Even golden brown
Buttery, high fat

9. Example of Scoring Rubrics
An assessment of writing sample
Criteria: linguistic structure and/or persuasiveness of the argument?
Four score levels; Meet Expectation, Adequate, Needs Improvement, and Inadequate
Each describes the characteristics of a response that would receive the respective score.
The description will increase raters’ reliability.
The description helps students understand the basis for their scores and what they need to do to improve their future performances.

10. SR for Report Writing L4: Meet Expectation L3: Adequate
Easily followed
Effective transitions
Professional format
Supportive graphics explained
Appropriate grammar
Clear and concise
L3: Adequate
Basic transitions
Structured format
Supportive graphics not clearly explained
Minimal distractions on flow, presentation, grammar
L2: Needs Improvement
Difficult to follow
Inadequate transitions
Rambling format
Insufficient/irrelevant information
Ambiguous graphics
Numerous distractions
L1: Inadequate
No organization
Difficult to read and understand

11. Learning Outcomes
Cognitive
Affective
Psychomotor
Social

12. Bloom’s Cognitive Domain
Level
Learning Outcomes
Evaluation
Making judgment based on a pre-established set of criteria
Synthesis
Producing something new or original from component parts
Analysis
See interrelationships/ hierarchy of ideas
Application
Using a concept or principle to solve a problem
Comprehension
Explaining/interpreting the meaning of material
Knowledge
Remembering facts, terms, concepts, definitions, principles

13. Krathwohl’s Affective Domain
Level
Learning Outcomes
Characterization
Behave in a way that is consistent with the values being adopted
Organization
Organizing ideas into a value system
Valuing
Judge & express commitment to the idea
Responding
Active participation in the activity
Receiving
Awareness of stimulus activity

14. Harrow’s Psychomotor Domain
Level
Learning Outcomes
Non-discursive communication
Communication through movement e.g. Posture nonverbal, expressive & interpretive movement
Skilled Movements
The result of learning, often complex learning, resulting in efficiency in carrying complex task
Physical Abilities
Individual’s physical self, enable smooth and efficient movements, e.g. endurance, strength
Perceptual Abilities
All the abilities that send input to the brain for interpretation, which affects motor movements
Basic-Fundamental Movements
Inherent in more complex or skilled motor movements e.g. manipulative movements
Reflex Movements
Involuntary movements, evident at birth or develop with maturation

15. Simpson’s Psychomotor Domain
Category
Learning Outcomes
Perception
The ability to use sensory cues to guide motor activity. This ranges from sensory stimulation, through cue selection, to translation
Set
Readiness to act. Includes mental, physical, and emotional sets. These three sets are dispositions that predetermine a person’s response to different situations – sometimes called mindsets.
Guided Response
The early stages in learning a complex skill that includes imitation and trial and error. Adequacy of performance is achieved by practicing.
Mechanism
The intermediate stage in learning a complex skill. Learned responses have become habitual and the movements can be performed with some confidence and proficiency

16. Simpson’s Psychomotor Domain
Category
Learning Outcomes
Complex Overt Response
The skillful performance of motor acts that involve complex movement patterns. Proficiency is indicated by a quick, accurate, and highly coordinated performance, requiring a minimum of energy. Includes performing without hesitation, and automatic performance. E.g. Players utter sounds of satisfaction (grunts) as soon as they hit a ball.
Adaptation
Skills are well developed and the individual modify movement patterns to fit special requirements. Responds effectively to unexpected experiences.
Origination
Creating new movement patterns to fit a particular situation or specific problem. Learning outcomes emphasize creativity based upon highly developed skills. Constructs a new theory.

17. Social Domain (Leise, 2004) Category Learning Outcomes Communication
Receiving, preparing, delivering a message
Teamwork
Involves relating culture by accepting constraints, living in society, and demonstrating cultural competence. It also involves relating with others by inviting interaction, relating for meaning, performing a team and performing in an organization
Management
Managing people, building and maintaining teams, managing communication, and managing resources
Leadership
Envisioning, building a following, maintaining commitment, and empowering

18. Are Scoring Rubrics Appropriate?
The purpose of evaluation:
Learning
Instruction
Grading
Selection
Placement
Counseling & Guidance
Program or Curriculum
Administrative Policy

19. Benefits of SR
Support the “Examination” - ensured that the specified criteria has been reached. Establish clear benchmarks for success.
Provide feedback on how to improve students performances. Because they are aware of the standards.

20. Types of SR
Analytic versus Holistic
General versus Task-specific

21. Analytic vs Holistic Analytic:
Like a checklist, allows for a separate assessment of each of the factors – the criteria
Example, SR on quality of written sample based on linguistic structure
If our objective is for persuasiveness of the argument, a separate set of criteria is needed
Each criterion is scored on a different descriptive scale

22. Analytic vs Holistic Holistic:
Used when there is an overlap between the criteria set for the different factors
The criteria are considered together on a single descriptive scale
Holistic SR supports broader judgment concerning the quality of the process or product

23. General vs Task-Specific
General SR:
For the assessment of a broader tasks category
E.g. If the purpose is to develop a student’s oral communication skills, a general SR is used to evaluate each of the given oral presentations
The student will use the feedback to improve their next presentation

24. General vs Task-Specific
Used to evaluate student’s performances on a single assessment event
E.g. Evaluating student’s knowledge on a different historical event

25. SR may contain both general and task-specific
An analytic SR that contains both general and task specific components could be designed to evaluate students’ oral presentation skills and their knowledge of the historical event being discussed.

26. Developing SR
1. Identify the qualities – the criteria - that need to be displayed. Refers to the learning outcomes
2. Define the criteria for lowest and top-level performance; resulting in three score levels
3. If greater distinctions are desired, comparisons can be made between the criteria for each existing score level, till the desired number of score levels is reached or no further meaningful distinctions can be made.

27. Developing SC
4. Each score category should be defined using descriptions of the work (quantifiable) rather than judgments about the work – “Student’s grammar contain no errors” is preferable over “Student’s grammar are good”
5. Ask other evaluators to use the SR to evaluate a subset of student responses; inter-raters reliability
6. Differences will suggest how the rubric may be further clarified

28. Thank You