USING RUBRICS TO ASSESS THE PERFORMANCE OF NEW PROFESSIONALS IN STUDENT AFFAIRS Tony Ribera & Sarah Fernandez A Presentation at the 2009 NASPA Region IV-E Conference, Chicago, IL

AGENDA State of assessment in student affairs Integrating rubrics into assessment efforts New professionals in student affairs and using rubrics to assess their learning/performance Going forward

PARTICIPANTS SHOULD GAIN… An appreciation for rubrics as a way to communicate expectations and assess performance An understanding of steps to develop a descriptive rubric in various functional areas of student affairs Experience creating a descriptive rubric with knowledgeable colleagues from various institutions

ASSESSMENT IN STUDENT AFFAIRS Who is responsible (ACPA, 2007) Questioning student learning (Creamer, Winston, & Miller, 2001; Hanson, 1991; King, 2003; Love, 1995) Systematic inquiry (Pascarella & Whitt, 1999) Looking toward the future (Torres & Walbert, 2009)

“One can argue there is a large gap between what student affairs practitioners say about the value and importance of research and assessment to their work and the extent to which such activities are actually conducted and used” (Pascarella & Whitt, 1999, p. 103).

WHAT IS A RUBRIC? Assessment tool often used to grade assignments Divides a larger assignment into smaller tasks Describes criteria and sets expectations Allows for consistent scoring Provides appropriate feedback (Stevens & Levi, 2005; Suskie, 2009)

TYPES OF RUBRICS (Suskie, 2009) Checklists List of goals/outcomes Rating Scales List of goals/outcomes plus rating scale Descriptive Rubrics Includes description for each level of the rating scale Holistic Scoring Guides Narrative description of the ratings possible. Does not include list of goals/outcomes

DESCRIPTIVE RUBRICS (Stevens & Levi, 2005) Most commonly used in higher education Made up of 4 parts: Task Description Description of overall assignment Scale Provides labels for the rating of performance Dimensions Goals/outcomes. Skills that should be achieved Description of Dimensions Expectations for each dimension at each rating scale

ASSESSMENT AND NEW PROFESSIONALS “Learning must be at the center of the student affairs profession. We as student affairs professionals require the consistent development of our own learning. Thus, what we learn and how we learn it becomes critical to developing our role as facilitators of the student learning and development process” (ACPA, 2007, p.2).

WHAT DO WE KNOW ABOUT NEW PROFESSIONALS? Graduate preparation (Waple, 2006) Competencies and responsibilities (Burkard, Cole, Ott, & Stoflet, 2005) Values in recruiting and hiring (Kretovics, 2002) Desire for feedback (Renn & Jessup-Anger, 2008)

CHALLENGES FACING NEW PROFESSIONALS Graduate preparation (Erwin & Sivo, 2001) Time/lack of resources (Johnson & Steele, 1984) Individual interest and motivation

ExemplaryCompetentBasic Application of Professional Literature in program and intervention development Use literature to understand new assessment techniques. Contribute to literature on assessment and evaluation. Use literature to understand new assessment techniques. Use literature to understand the importance of assessment and evaluation techniques Facilitation of data collection to measure student learning Design and implement high quality assessment and evaluation strategies appropriate for staff, departmental, and institutional efforts. Facilitate training of staff to participate in assessment techniques. Recognize importance of assessment, and participate in training of colleagues. Identify most appropriate instruments to be used in assessment efforts for staff and departmental efforts. Construct basic assessment and evaluation tools with supervision. Collect data to assess department wide educational efforts. Evaluate staff, collecting data from multiple sources. Interpretation of assessment data Facilitate the development of assessment and evaluation reports, which use data to create goals and plans of action for use of results. Use departmental resources to support high quality evaluation. Participate in the analysis of collected data. Effectively interpret results to be reported to department. Interpret data in order to use results of reports and studies. Application of results to improve programs and practices Ensure the use of results in improving and/or creating new departmental and institutional efforts, programs, activities. Emphasize importance of assessment to departmental and institutional efforts. Ensure the use of results in improving and/or creating new departmental efforts, programs, activities. Emphasize importance of assessment to departmental efforts. Use findings of assessment and evaluation in improving educational efforts. Comprehension of qualitative research methods Supervise the design of qualitative assessment and evaluation. Understand needs to be flexible in qualitative studies. Have mature understanding of transferability and trustworthiness. Create qualitative sampling designs with supervision. Participate in analysis of qualitative data. Use techniques to ensure trustworthiness. Understand qualitative research methods. Understand how qualitative findings can be used in work settings. Judge trustworthiness of study. Comprehension of quantitative research methods Supervise the design of quantitative assessment and evaluation. Understand how to disseminate results. Assess instruments for validity and reliability. Learn to use mixed methods of data collection. Create quantitative designs with supervision. Participate in the analysis of quantitative data. Understand more complex methods, including multivariate techniques. Understand quantitative research methods. Recognize issues of validity and reliability in quantitative methods.

STEPS FOR CREATING A DESCRIPTIVE RUBRIC (Suskie, 2009) Identify learning outcomes-what do you want them to take away? List criteria-what do you want them to accomplish? Focus on most significant skills. These will become your dimensions Developing rating scale At minimum levels for adequate and inadequate Write descriptions Test it out- make sure the standards and descriptions are appropriate

RUBRIC DEVELOPMENT MODELS (Stevens & Levi, 2005) Presentation Model Rubric presented without outside feedback Feedback Model Rubric can be changed with feedback after presented Pass-the-Hat Model & Post-it Model Both parties work together to develop rubric 4 X 4 model Students take full responsibility of development

SMALL GROUP WORK ACTIVITY AND LARGE GROUP DISCUSSION

MOVING FORWARD Need for more scholar-practitioners (Hossler, 2001; Komives, 1998; Schroeder & Pike, 2001) Engaging in a scholarship of teaching and learning (Huber & Hutchings, 2005) Questioning Gathering and exploring evidence Trying out and refining new insights Going public