1. Creating Analytic Rubrics April 27, 2017
Cathy Sanders Director of Assessment Office of Assessment & Accreditation

2. Workshop Objectives Introduce you to analytic and holistic rubrics
Go through steps in creating an analytic rubric
Provide you with the opportunity to create a simple analytic rubric
Explain how to use an analytic rubric in conjunction with SLO assessment data

3. What is a Rubric?
A scoring tool that uses a set of evaluation criteria that are directly tied to the student learning outcome to assess student learning
Used with subjective assessments
Promotes inter-rater reliability
When the content of the rubric is communicated prior to students completing the work, the grading process is very clear and transparent to everyone involved.

4. Two Types of Rubrics
Analytic rubric provides cell descriptors for each evaluation criterion for each level of performance.
Holistic rubric provides 1 cell descriptor for all evaluation criteria for each level of performance.
UNC Charlotte uses analytic rubrics
More useful in pinpointing areas of strengths and weaknesses in student learning
Aids in deciding needed improvements in student learning

5. Sample Analytic Rubric
Evaluation
Criteria
1 = Needs Improvement
2 = Acceptable
3 = Accomplished
Criterion Score
Ability to develop a rubric scale
Is unable to develop a minimum of a 3 point scale that progresses from least proficient to most proficient
Is able to develop a minimum of a 3 point scale that progresses from least proficient to most proficient
Is able to develop a scale that is greater than 3 points that progresses from least proficient to most proficient 3
Ability to develop evaluation criteria
Is unable to identify at least 3 relevant evaluation criteria
Is able to identify at least 3 relevant evaluation criteria
Is able to identify more than 3 relevant evaluation criteria 2
Ability to develop cell descriptors
Some of the cell descriptors are missing and cell descriptors(s) present do not clearly differentiate the levels of proficiency.
Cell descriptors are present for all evaluation criteria but some do not clearly differentiate the levels of proficiency.
Cell descriptors are present for all evaluation criteria that clearly differentiate the levels of proficiency. 1

7. Elements of an Analytic Rubric
Rubric Scale
Evaluation Criteria
Cell Descriptors
Criterion Score

8. Sample Analytic Rubric
Evaluation
Criteria
1 = Needs Improvement
2 = Acceptable
3 = Accomplished
Criterion Score
Ability to develop a rubric scale
Is unable to develop a minimum of a 3 point scale that progresses from least proficient to most proficient
Is able to develop a minimum of a 3 point scale that progresses from least proficient to most proficient
Is able to develop a scale that is greater than 3 points that progresses from least proficient to most proficient 3
Ability to develop evaluation criteria
Is unable to identify at least 3 relevant evaluation criteria
Is able to identify at least 3 relevant evaluation criteria
Is able to identify more than 3 relevant evaluation criteria 2
Ability to develop cell descriptors
Some of the cell descriptors are missing and cell descriptors(s) present do not clearly differentiate the levels of proficiency.
Cell descriptors are present for all evaluation criteria but some do not clearly differentiate the levels of proficiency.
Cell descriptors are present for all evaluation criteria that clearly differentiate the levels of proficiency. 1

9. Prior to Creating an Analytic Rubric
Develop the Student Learning Outcome (SLO) statement
The knowledge, skills, abilities, behaviors, and dispositions that students will demonstrate as a result of the educational program
Decide the tasks that students will perform and the artifact they will create to demonstrate proficiency on the SLO
Identify the criteria that will be used to evaluate proficiency on the SLO

10. Examples Students will demonstrate competence in programming skills.
Students will demonstrate oral communication skills
Artifact: An oral presentation
Criteria: Body language, eye contact, introduction and closure, pacing, poise, voice, and organization.
Artifact: A programming project
Criteria: Understands requirements, uses appropriate data structures, uses appropriate algorithms, modularizes solution, designs appropriate user interface, tests program for correctness, and documents program

11. Steps in Creating an Analytic Rubric
Determine a progressive scale and insert it in the columns across the top of the rubric template
Insert the evaluation criteria in the rows down the left side of the rubric template
At the intersection of each row and column, insert a description of what proficiency on each criterion at each scale level looks like
It may be easier to first describe what proficiency at the highest and lowest scale levels look like
Use a meta-rubric to evaluate the rubric.
Norm the rubric: multiple scorers use the rubric to score student artifacts.
Revise the cell evaluation criteria and/or cell descriptors to reflect needed changes

12. Workshop Exercise: Creating an Analytic Rubric

13. Instructions for Analytic Rubric Exercise
Develop an analytic rubric for an oral presentation that includes:
a 3-point rubric scale
3 evaluation criteria that you feel are the most important
cell descriptors for each criterion at all 3 scale levels (a total of 9 cell descriptors)

14. SLO Assessment Plan Template
Student Learning Outcome 1: the knowledge, skill, ability, behavior, or disposition that students will demonstrate proficiency in.
Effectiveness Measure: Student artifact that will be used to gauge acquisition of the student learning outcome.
Methodology: A description of how, what, when, and where…
Performance Outcome: the percentage of students that will demonstrate proficiency and the level of proficiency expected.

15. Using the Analytic Rubric with Assessment Data
Identify SLOs on which students did not meet performance expectations
Compare assessment data with the Performance Outcome
Example: PO states “ 80% of students will score a 2 “Acceptable” or higher on each of the rubric criteria” but only 75% did. That is an unmet SLO
Look at the average criterion scores on the rubric associated with the unmet SLO
Identify the evaluation criteria on which students did not meet the Performance Outcome (e.g., did not score 2 “Acceptable” or higher)
Those are the criteria to focus improvement efforts on

16. Analytic Rubric 3 2 1 1 = Needs Improvement 2 = Acceptable
 Criteria
1 = Needs Improvement
2 = Acceptable
3 = Accomplished
Average Criterion Score
Ability to develop a rubric scale
Is unable to develop a minimum of a 3 point scale that progresses from least proficient to most proficient
Is able to develop a minimum of a 3 point scale that progresses from least proficient to most proficient
Is able to develop a scale that is greater than 3 points that progresses from least proficient to most proficient 3
Ability to develop evaluation criteria
Is unable to identify at least 3 relevant evaluation criteria
Is able to identify at least 3 relevant evaluation criteria
Is able to identify more than 3 relevant evaluation criteria 2
Ability to develop cell descriptors
Some of the cell descriptors are missing and cell descriptors(s) present do not clearly differentiate the levels of proficiency.
Cell descriptors are present for all evaluation criteria but some do not clearly differentiate the levels of proficiency.
Cell descriptors are present for all evaluation criteria that clearly differentiate the levels of proficiency. 1

17. Collaborative Discussion by Program Faculty After Problem Areas of SLO Have Been Identified
how the SLO is taught
whether there is sufficient time to teach the SLO
how assignments lead to acquisition of the SLO
if assignments need to be revised or replaced
if additional assignments are needed to reinforce the learning
if a different assessment is needed (to achieve better alignment between what is being taught and what is being assessed).

18. Questions