SURVEYS, OBSERVATIONS, AND RUBRICS OH MY! ASSESSING CAREER SERVICES Jessica M. Turos Bowling Green State University Career Center

OUTLINE Brief review of assessment/evaluation literature and the assessment cycle Discussion of assessment practices used including successes and challenges Ways we can use assessment to inform our work Question and Answer

LEARNING OUTCOMES Decipher pertinent assessment/evaluation literature Identify the assessment cycle and potential successes and challenges Determine quantitative and qualitative methods used to conduct assessment/evaluation

LEARNING OUTCOMES CONT. Recognize potential assessment practices used in career services Distinguish pros and cons of each method of assessment/evaluation Name potential assessment/evaluation practices and methods

ASSESSMENT/EVALUA TION Gathering information – measurement Using information for individual and institutional improvement – evaluation Student learning – assessment (Astin, 1993)

ASSESSMENT/EVALUA TION Assessment - “any effort to gather, analyze, and interpret evidence which describes institutional, divisional, or agency effectiveness” (Schuh, Upcraft, & Associates, 2001) Comprehensive process Culture of assessment

ASSESSMENT LITERATURE Accountability Summative Justification Satisfaction Improvement Formative Learning Accountability and improvement are not mutually exclusive

AUTHENTIC ASSESSMENT Involves real-world information and skills Uses critical thinking skills to explore unstructured problems Performed in situations as similar to the context as possible Allows for feedback, practice, and second chances Empowers students

TYPES OF AUTHENTIC ASSESSMENT Real-world projects Simulated environment Case studies Service-learning projects Performances Evaluation based on observation of work Portfolios

ASSESSMENT CYCLE (SUSKIE, 2004)

ASSESSMENT METHODS Indirect Questionnaire Evaluations Surveys Interviews Focus groups Direct Exams Performance measures Projects Presentations Case studies Portfolios

ASSESSMENT METHODS Quantitative – learning and satisfaction Evaluations Surveys Questionnaires Tests Rubrics

ASSESSMENT METHODS Qualitative – learning and satisfaction Observations Interviews Focus groups Rubrics Portfolios Mixed Methods – learning and satisfaction

ASSESSMENT PRACTICES USED Surveys/Evaluations Paper versus electronic Response rate Data entry Examples Feedback form Job and Internship Fair Student and Employer Workshops and Trainings Courses

ASSESSMENT PRACTICES USED Observations Formal versus informal Participation/Buy-in Collaboration Examples Job and Internship Fair Observation Form Staff and Students Student appointments Case Consultation Meetings

ASSESSMENT PRACTICES USED Rubrics Usage – staff and students Portfolios Product Process Examples Minimum Resume Requirements Resume Rubrics Teacher Portfolio Review E-Portfolios

ASSESSMENT PRACTICES USED Focus groups Participation Timing Interviews Participation Timing Examples WorkNet Usability Perfect Interview Learning

WAYS ASSESSMENT CAN INFORM OUR WORK Case Studies Discussion Case 1 – Your office is implementing a workshop series and you want to know if it is effective in terms of students learning key concepts and students feeling engaged with the material. How do you assess the series? Case 2 – Your office is interested in learning the impact of your services (drop-in critiques and individual meetings) and resources (web content and printed resources) on students’ resumes. How do you assess the impact?

WAYS ASSESSMENT CAN INFORM OUR WORK Examples from offices Successes Challenges Suggestions

QUESTION AND ANSWER

REFERENCES American College Personnel Association. (1996). The student learning imperative: Implications for student affairs. Retrieved July 10, 2009 from Astin, A. W. (1993). Assessment for excellence: The philosophy and practice of assessment and evaluation in higher education. New York: American Council on Education and the Oryx Press. Burley, H., & Price, M. (2003). What works with authentic assessment. Educational Horizons, 81 (4), 193 – 196. Gulikers, J., Bastiaens, T., & Kirschner, P. (2006). Authentic assessment, student and teacher perceptions: The practical values of the five- dimensional framework. Journal of Vocational Education and Training, 58 (3),

REFERENCES CONTINUED Keeling, R. P. (Ed.). (2004). Learning reconsidered: A campus-wide focus on the student experience. The National Association of Student Personnel Administrators and the American College Personnel Association. La Lopa, J. M. (2005). Developing a student-based evaluation tool for authentic assessment. New Directions for Teaching and Learning, 100, Lombardi, J. (2008). To portfolio or not portfolio: Helpful or hyped? College Teaching, 56 (1), Palomba, C. A., & Banta, T. W. (1999). Assessment essentials: Planning, implementing, and improving assessment in higher education. San Francisco: Jossey- Bass. Schuh, J. H., Upcraft, M. L., & Associates (2001). Assessment practice in student affairs: An applications manual. San Francisco: Jossey-Bass.

REFERENCES CONTINUED Shavelson, R. J. (2007). A brief history of student learning assessment: How we got where we are and a proposal for where to go next. Washington, DC: Association of American Colleges and Universities. Suskie, L. (2004). Assessing student learning: A common sense guide. Bolton, MA: Anker. Svinicki, M. D. (2005). Authentic assessment: Testing in reality. New Directions for Teaching and Learning, 100, U.S. Department of Education. (2006). A test of leadership: Charting the future of U.S. higher education. A report of the commission appointed by Secretary of Education M. Spellings. Washington, D.C. Pre- Publication copy, September Retrieved from: rts/pre-pub-report.pdf rts/pre-pub-report.pdf