Approaches to Effective and Efficient Assessment and Grading Carol A. Kominski, Ph.D. Assessment Specialist

Authentic Assessment If you must take a chauffeur-driven trip, what do you prefer when you have only two choices? – A driver who passed the written portion of the licensing exam but failed the driving portion – A driver who passed the driving portion of the licensing exam but failed the written portion

Efficiency*Continuum Low High *Efficiency = ease and speed of grading. Portfolios, clinical applications, reflection, research reports Multiple choice, matching, T-F Case-based analyses, oral presentations, demonstrations Short written or oral responses Fill in the blank questions

Authenticity*Continuum Low High *Authenticity = degree to which requirement is similar to that likely to be encountered in workplace or professional practice. Multiple choice, matching, T-F Portfolios, clinical applications, reflection, research reports Fill in the blank questions Short written or oral responses Case-based analyses, oral presentations, demonstrations

Assessment Challenges How can efficient assessments be more authentic? (Strategies to Assess Large Classes) How can authentic assessments be more efficient? (Approaches to Effective and Efficient Assessment and Grading)

Potential Authentic Assessments Short answer or essay test/assignment Research report Case Study Analysis Written Work Class presentation or speech Debate Conversation Oral Communication Specific research procedure or technique Clinical procedure, e.g. blood sampling Client or patient interaction Demonstration Surveys Records of participation Unobtrusive measures of behavior Disposition or Behavior E-Portfolio Instructional unit Reflection paper Comprehensive/Hybrid

Web Resource on Authentic Assessment Authentic Assessment Toolbox – provides background, explanations, examples. atisit.htm

How to make authentic assessment manageable and efficient Rubrics (otherwise known as Standards, Criteria, Measures) – Multiple dimensions – Scale Placement Categories Usually 3 to 5 choices on continuum Only numbers, e.g, 1 to 5 scale – Poor, Fair, Good, Excellent – Novice, Developing, Accomplished – Doesn’t Meet, Meets, Exceeds Expectations – Your own creation Checklists (Two Categories) – Yes or No/True or False/Off or On – Can be complemented with Student Involvement Peer assessment Teaching assistants

What are rubrics? “A scoring tool that lays out the specific expectations for an assignment. Rubrics divide an assignment into its component parts and provide a detailed description of what constitutes acceptable or unacceptable levels of performance for each of those parts.” Parts are – Task Description. – Scale. – Dimensions. – Dimension Descriptions. Source: Stevens, Danielle D. and Levi, Antonia J. (2005). Introduction to Rubrics, 3. “A scoring tool for qualitative rating of authentic or complex student work. It includes criteria for rating important dimensions of performance, as well as standards of attainment for those criteria. The rubric tells both instructor and student what is considered important and what to look for when assessing.” Source: Jonsson, A. & Svingby, G. (2007). The use of scoring rubrics: Reliability, validity, and educational consequences. Educational Research Review. 2,

Why rubrics are good assessment practice Timeliness Detailed Feedback Critical Thinking Communication Refinement of teaching skills Level playing field Source: Stevens, Danielle D. and Levi, Antonia J. (2005). Why use rubrics? Introduction to Rubrics,

Steps in Constructing a Rubric Determine learning objectives. Outline expectations. – Use results of previous assignments, if possible. What’s good? What’s acceptable? What’s unacceptable or poor? – Start with what others have done. Develop rubric dimensions, e.g., – Organization – Persuasiveness of arguments – Research support – Quality of writing Fill in the rubric.

Rubric Features Rubrics can be created in a variety of forms and levels of complexity, however, they all contain three common features which: – focus on measuring a stated objective (performance, behavior, or quality). – use a range to rate performance. – contain specific performance characteristics arranged in levels indicating the degree to which a standard has been met. Source: Pickett, N. and Dodge, B. (2007). Rubrics for Web Lessons.

Rubrics Do’s and Don’ts Describe performance levels clearly. Solicit advice from others (other faculty, teaching assistants, even students). Use anchor examples from previous assessments, if possible. Measure consistency (over time and raters). Revise as needed for consistency, clarity, comprehensiveness, ease of use. Make sure relative emphasis (or points) is placed on high priority criteria. Do Assume that students understand the rubric without discussion and explanation. Make rubric a counting tool which eliminates professional judgment. Allow rubric to set too low a limit on student achievement. Use surprise rubrics shared with students only after assignment submission. Don’t

Checklists A simple type of rubric For either-or classification. Good for determining compliance with format and content requirements. Typically used for lower order items. Examples Abstract APA format Two examples On time submission

Example of Combination Rubric

Student Involvement Models just sharing Presentation rubric can be changed Feedback students help fill in dimensions Pass-the-Hat students create dimensions and fill them in Post-it students participate in all stages of rubric creation 4 X 4 Model Source: Stevens, Danielle D. and Levi, Antonia J. (2005). Rubric Construction and the Classroom. Introduction to Rubrics,

Peer Assessment Designed for mature students. Engages learners in authentic learning. Students develop assessment criteria. Students assess others’ learning, grade work, and receive feedback. Tutor checks to ensure quality. Source: Juwah, C. (2003). Using Peer Assessment to Develop Skills and Capabilities. Journal of the United States Distance Learning Association, January 2003,

Teaching Assistants Collaboration helps in Understanding what instructor is trying to accomplish. Modeling good teaching practice. Obtaining insight into common student misunderstandings and issues.

Web Resources: Rubric Examples from Higher Education American Association for Learning in Higher Education (AALHE) – California State University at Bakersfield – Packet_Jan06.pdf Packet_Jan06.pdf Loyola Marymount University – ssessment_Resources/Rubrics/Example_Rubrics.htm ssessment_Resources/Rubrics/Example_Rubrics.htm California State University at Fullerton Business School – /RubricDirectory/other_rubrics.htm /RubricDirectory/other_rubrics.htm

Rubrics: Examples from Pharmacy, Physician’s Assistant, Public Health programs Pharmacy SOAP note rubric – =Z8WACB&sp=true& =Z8WACB&sp=true& Physician’s assistant physical history rubric 2&sp=true 2&sp=true Public Health Rubric ( QEP.pdf) See next page. QEP.pdf

Other Rubrics: A Few Examples Ethics – n458/sample_case_rubric.htm n458/sample_case_rubric.htm Case analysis – ment/ksom/case-analysis-rubric.pdf ment/ksom/case-analysis-rubric.pdf Critical thinking – 20Thinking-long.pdf 20Thinking-long.pdf

What’s wrong with this rubric?

Assignment 1. Describe consequences of high sodium consumption on the human body and its possible relationship to health problems and morbidity. Assignment 2. Demonstrate procedures for determining whether a patient has a potential heart problem. Assignment 3. Describe, evaluate, and compare at least two potential solutions to reduce the proportion of the population who do not have health insurance. Challenge: Design rubrics for assignments like the following.

More Support Needed Contact Center for Learning and Development (CLD) for – Assessment Design (Carol Kominski Ext. 2942) – Instructional Design (Kun Huang Ext. 2941) – Blackboard (AZ Bashet Ext. 2943) – Psychometric analysis (Kevin Kalinowski Ext. 2543) – Other support (Director Vanneise Collins Ext. 5056)