1. Approaches to Effective and Efficient Assessment and Grading Carol A. Kominski, Ph.D. Assessment Specialist

2. Authentic Assessment If you must take a chauffeur-driven trip, what do you prefer when you have only two choices? – A driver who passed the written portion of the licensing exam but failed the driving portion – A driver who passed the driving portion of the licensing exam but failed the written portion

3. Efficiency*Continuum Low High *Efficiency = ease and speed of grading. Portfolios, clinical applications, reflection, research reports Multiple choice, matching, T-F Case-based analyses, oral presentations, demonstrations Short written or oral responses Fill in the blank questions

4. Authenticity*Continuum Low High *Authenticity = degree to which requirement is similar to that likely to be encountered in workplace or professional practice. Multiple choice, matching, T-F Portfolios, clinical applications, reflection, research reports Fill in the blank questions Short written or oral responses Case-based analyses, oral presentations, demonstrations

5. Assessment Challenges How can efficient assessments be more authentic? (Strategies to Assess Large Classes) How can authentic assessments be more efficient? (Approaches to Effective and Efficient Assessment and Grading)

6. Potential Authentic Assessments Short answer or essay test/assignment Research report Case Study Analysis Written Work Class presentation or speech Debate Conversation Oral Communication Specific research procedure or technique Clinical procedure, e.g. blood sampling Client or patient interaction Demonstration Surveys Records of participation Unobtrusive measures of behavior Disposition or Behavior E-Portfolio Instructional unit Reflection paper Comprehensive/Hybrid

7. Web Resource on Authentic Assessment Authentic Assessment Toolbox – provides background, explanations, examples. http://jfmueller.faculty.noctrl.edu/toolbox/wh atisit.htm

8. How to make authentic assessment manageable and efficient Rubrics (otherwise known as Standards, Criteria, Measures) – Multiple dimensions – Scale Placement Categories Usually 3 to 5 choices on continuum Only numbers, e.g, 1 to 5 scale – Poor, Fair, Good, Excellent – Novice, Developing, Accomplished – Doesn’t Meet, Meets, Exceeds Expectations – Your own creation Checklists (Two Categories) – Yes or No/True or False/Off or On – Can be complemented with Student Involvement Peer assessment Teaching assistants

9. What are rubrics? “A scoring tool that lays out the specific expectations for an assignment. Rubrics divide an assignment into its component parts and provide a detailed description of what constitutes acceptable or unacceptable levels of performance for each of those parts.” Parts are – Task Description. – Scale. – Dimensions. – Dimension Descriptions. Source: Stevens, Danielle D. and Levi, Antonia J. (2005). Introduction to Rubrics, 3. “A scoring tool for qualitative rating of authentic or complex student work. It includes criteria for rating important dimensions of performance, as well as standards of attainment for those criteria. The rubric tells both instructor and student what is considered important and what to look for when assessing.” Source: Jonsson, A. & Svingby, G. (2007). The use of scoring rubrics: Reliability, validity, and educational consequences. Educational Research Review. 2, 130-144.

10. Why rubrics are good assessment practice Timeliness Detailed Feedback Critical Thinking Communication Refinement of teaching skills Level playing field Source: Stevens, Danielle D. and Levi, Antonia J. (2005). Why use rubrics? Introduction to Rubrics, 17-28.

11. Steps in Constructing a Rubric Determine learning objectives. Outline expectations. – Use results of previous assignments, if possible. What’s good? What’s acceptable? What’s unacceptable or poor? – Start with what others have done. Develop rubric dimensions, e.g., – Organization – Persuasiveness of arguments – Research support – Quality of writing Fill in the rubric.

12. Rubric Features Rubrics can be created in a variety of forms and levels of complexity, however, they all contain three common features which: – focus on measuring a stated objective (performance, behavior, or quality). – use a range to rate performance. – contain specific performance characteristics arranged in levels indicating the degree to which a standard has been met. Source: Pickett, N. and Dodge, B. (2007). Rubrics for Web Lessons. http://webquest.sdsu.edu/rubrics/weblessons.htm http://webquest.sdsu.edu/rubrics/weblessons.htm

13. Rubrics Do’s and Don’ts Describe performance levels clearly. Solicit advice from others (other faculty, teaching assistants, even students). Use anchor examples from previous assessments, if possible. Measure consistency (over time and raters). Revise as needed for consistency, clarity, comprehensiveness, ease of use. Make sure relative emphasis (or points) is placed on high priority criteria. Do Assume that students understand the rubric without discussion and explanation. Make rubric a counting tool which eliminates professional judgment. Allow rubric to set too low a limit on student achievement. Use surprise rubrics shared with students only after assignment submission. Don’t

14. Checklists A simple type of rubric For either-or classification. Good for determining compliance with format and content requirements. Typically used for lower order items. Examples Abstract APA format Two examples On time submission

15. Example of Combination Rubric

16. Student Involvement Models just sharing Presentation rubric can be changed Feedback students help fill in dimensions Pass-the-Hat students create dimensions and fill them in Post-it students participate in all stages of rubric creation 4 X 4 Model Source: Stevens, Danielle D. and Levi, Antonia J. (2005). Rubric Construction and the Classroom. Introduction to Rubrics, 49-64.

17. Peer Assessment Designed for mature students. Engages learners in authentic learning. Students develop assessment criteria. Students assess others’ learning, grade work, and receive feedback. Tutor checks to ensure quality. Source: Juwah, C. (2003). Using Peer Assessment to Develop Skills and Capabilities. Journal of the United States Distance Learning Association, January 2003, 39-50.

18. Teaching Assistants Collaboration helps in Understanding what instructor is trying to accomplish. Modeling good teaching practice. Obtaining insight into common student misunderstandings and issues.

19. Web Resources: Rubric Examples from Higher Education American Association for Learning in Higher Education (AALHE) – http://course1.winona.edu/shatfield/air/rubrics.htm http://course1.winona.edu/shatfield/air/rubrics.htm California State University at Bakersfield – http://www.csub.edu/TLC/options/resources/handouts/Rubric_ Packet_Jan06.pdf http://www.csub.edu/TLC/options/resources/handouts/Rubric_ Packet_Jan06.pdf Loyola Marymount University – http://www.lmu.edu/about/services/academicplanning/assessment/A ssessment_Resources/Rubrics/Example_Rubrics.htm http://www.lmu.edu/about/services/academicplanning/assessment/A ssessment_Resources/Rubrics/Example_Rubrics.htm California State University at Fullerton Business School – http://business.fullerton.edu/centers/CollegeAssessmentCenter /RubricDirectory/other_rubrics.htm http://business.fullerton.edu/centers/CollegeAssessmentCenter /RubricDirectory/other_rubrics.htm

20. Rubrics: Examples from Pharmacy, Physician’s Assistant, Public Health programs Pharmacy SOAP note rubric – http://www.rcampus.com/rubricshowc.cfm?code =Z8WACB&sp=true& http://www.rcampus.com/rubricshowc.cfm?code =Z8WACB&sp=true& Physician’s assistant physical history rubric http://www.rcampus.com/rubricshowc.cfm?code=W4A5C 2&sp=true http://www.rcampus.com/rubricshowc.cfm?code=W4A5C 2&sp=true Public Health Rubric (http://www.tamhsc.edu/about/accreditation/sacs/CARE- QEP.pdf) See next page.http://www.tamhsc.edu/about/accreditation/sacs/CARE- QEP.pdf

22. Other Rubrics: A Few Examples Ethics – http://www.personal.psu.edu/faculty/d/x/dxm12/ n458/sample_case_rubric.htm http://www.personal.psu.edu/faculty/d/x/dxm12/ n458/sample_case_rubric.htm Case analysis – http://academic.scranton.edu/department/assess ment/ksom/case-analysis-rubric.pdf http://academic.scranton.edu/department/assess ment/ksom/case-analysis-rubric.pdf Critical thinking – http://course1.winona.edu/shatfield/air/Critical% 20Thinking-long.pdf http://course1.winona.edu/shatfield/air/Critical% 20Thinking-long.pdf

23. What’s wrong with this rubric?

24. Assignment 1. Describe consequences of high sodium consumption on the human body and its possible relationship to health problems and morbidity. Assignment 2. Demonstrate procedures for determining whether a patient has a potential heart problem. Assignment 3. Describe, evaluate, and compare at least two potential solutions to reduce the proportion of the population who do not have health insurance. Challenge: Design rubrics for assignments like the following.

25. More Support Needed Contact Center for Learning and Development (CLD) for – Assessment Design (Carol Kominski Ext. 2942) – Instructional Design (Kun Huang Ext. 2941) – Blackboard (AZ Bashet Ext. 2943) – Psychometric analysis (Kevin Kalinowski Ext. 2543) – Other support (Director Vanneise Collins Ext. 5056)