Developing a Rubric as One Measurement Tool in the Assessment Process for Academic Advising Sharon A. Aiken-Wisniewski, Ph.D. University of Utah NACADA Executive Office Kansas State University 2323 Anderson Ave, Suite 225 Manhattan, KS Phone: (785) Fax: (785) © 2012 National Academic Advising Association The contents of all material in this presentation are copyrighted by the National Academic Advising Association, unless otherwise indicated. Copyright is not claimed as to any part of an original work prepared by a U.S. or state government officer or employee as part of that person's official duties. All rights are reserved by NACADA, and content may not be reproduced, downloaded, disseminated, published, or transferred in any form or by any means, except with the prior written permission of NACADA, or as indicated below. Members of NACADA may download pages or other content for their own use, consistent with the mission and purpose of NACADA. However, no part of such content may be otherwise or subsequently be reproduced, downloaded, disseminated, published, or transferred, in any form or by any means, except with the prior written permission of, and with express attribution to NACADA. Copyright infringement is a violation of federal law and is subject to criminal and civil penalties. NACADA and National Academic Advising Association are service marks of the National Academic Advising Association. The Global Community for Academic Advising

An Activity The Global Community for Academic Advising

Participant Learning Outcomes Behavioral Describe a rubric. Develop levels of performance for a rubric. Develop criterion points for a rubric. Cognitive Understand the purpose of using a rubric for measurement. Explain the use of descriptors when considering a level of performance for a criterion point. Affective Values reflection within the process of developing a rubric.

A rubric is a scoring scale utilized to measure a student’s or advisor’s performance against a predetermined set of criteria Assessment of Academic Advising Institute

The Global Community for Academic Advising A Rubric... Divides a desired objective or outcome into measurable pieces Identifies what are acceptable and unacceptable degrees of performance for a specific outcome The component parts are called –criterion points and –degrees of performance Descriptors are the intersection of criterion points and degrees of performance that assist with scoring

Rubrics typically have at least two criterion points and two levels of performance on separate axes The Global Community for Academic Advising

For example, if you wanted to measure a student’s level of knowledge regarding a specific learning outcome, such as knowing departmental requirements, your vertical axis may contain the criterion points for *Knowledge *Understanding *Behavior Assessment of Academic Advising Institute

While your horizontal axis may contain the degrees of performance. Points can include: Adequate/not adequate Excellent/competent/developing Range of numbers The Global Community for Academic Advising

Such that the beginning of your rubric would look like: ExcellentCompetentImprovingDeveloping Knowledge Criterion Understanding Criterion Behavior Criterion The Global Community for Academic Advising

You would then add descriptors for the intersection of each criterion point and each degree of performance For example, a descriptor for the criterion Knowledge may be “Ability to communicate the required courses for the major” For the criterion Understanding, it may be “Ability to comprehend the reasoning behind the requirements” And for the criterion Behavior it may be “Ability to schedule/enroll in appropriate major courses” Assessment of Academic Advising Institute

Adding these descriptors for your criterion points, your rubric would evolve into ExcellentCompetentImprovingDeveloping Knowledge Ability to communicate the required courses for the major Understanding Ability to comprehend the reasoning behind the requirements Behavior Ability to correctly schedule/enroll in appropriate major courses The Global Community for Academic Advising

Descriptors would also be added for the degrees of performance for each criterion as well for example, the descriptor: For Excellent for the criterion Knowledge may be “Can delineate all requirements for the major” For Competent regarding the criterion Knowledge, it may be “Can delineate most of the requirements for the major” For Improving it may be “Can delineate some of the requirements for the major” For Developing for the criterion Knowledge it may be “Cannot delineate any requirements for the major” Assessment of Academic Advising Institute

Adding these descriptors to the degrees of performance for the criterion Knowledge would result in the rubric looking like ExcellentCompetentImprovingDeveloping Knowledge Ability to communicate the required courses for the major Can delineate all requirements of the major Can delineate most requirements for the major Can delineate some requirements for the major Cannot delineate any requirements for the major Understanding Ability to comprehend the reasoning behind the requirements Behavior Ability to correctly schedule/enroll in appropriate major courses The Global Community for Academic Advising

You then continue to add descriptors for each degree of performance relative to each of the remaining criterion points of Understanding and Behavior Your final rubric may look like: Assessment of Academic Advising Institute

ExcellentCompetentImprovingDeveloping Knowledge Ability to communicate the required courses for the major Can delineate all requirements of the major Can delineate most requirements for the major Can delineate some requirements for the major Cannot delineate any requirements for the major Understanding Ability to comprehend the reasoning behind the requirements Can clearly describe reasons/rational e for required courses Can clearly describe many reasons for the required courses Can clearly describe several reasons for the required courses Cannot describe any reasons for the required courses Behavior Ability to correctly schedule/enroll in appropriate major courses Enrolls in appropriate courses to fulfill major requirements Enrolls in several appropriate courses to fulfill major requirements Enrolls in a few appropriate courses to fulfill major requirements Has no appropriate courses on schedule The Global Community for Academic Advising

Rubrics may also include just one criterion point for a very specific task (e.g., attend a workshop) and just one dichotomous level of performance (e.g., the student did or didn’t attend) Assessment of Academic Advising Institute

Construction of a rubric requires: - reflection on the overall objectives, goals, and outcomes of your program, and - identification of what you hope will be accomplished via academic advising The Global Community for Academic Advising

Evaluating Rubric Data One way to evaluate data from a rubric –The % of students that fit into each degree of performance for each criterion. Another way to evaluate data from a rubric –develop a score range to see what the numerical average is for each criterion point. Excellent = 4 Competent = 3 Needs More Info = 2 Not Aware = 1 –This could be tracked longitudinally to establish a benchmark

The Global Community for Academic Advising Rubric – Withdrawal (732) Completed by advisors after appointments – What are the % of students in each degree of performance for the first criterion – explaining withdrawal? – What is the mean (average) degree of performance on students communicate withdraw deadline? Average is 3.09 on 4.0

Rubrics are a good way to allow a quick evaluation of a performance or demonstration of knowledge, but should be utilized as just one of multiple measures for any given performance or demonstration of knowledge Assessment of Academic Advising Institute

Resources Maki, P. L. (2004). Assessing for Learning; Building a Sustainable Commitment Across the Institution. Sterling, VA: Stylus Mueller, J. (2006). Authentic Assessment Toolbox. Retrieved 1/26/06. Stevens, D. D., and Levi, A. J. (2005). Introduction to Rubrics. Sterling, VA: Stylus. The Global Community for Academic Advising