Making Outcomes Assessment Useful and Used Laura murphy College-wide outcomes and assessment facilitator Spring 2017 Adopted with permission from linda suskie’s accjc “making SLO assessment useful and used”, ACCJC fall 2016

Who is Linda Suskie? Internationally recognized consultant, speaker, writer, and workshop facilitator on a broad variety of higher education assessment and accreditation topics. Taught graduate courses (assessment and educational research methods) and undergraduate courses (writing, statistics, and developmental mathematics); seven years as a Vice President at the Middle States Commission on Higher Education (an accreditor of colleges and universities in the mid-Atlantic region of the United States Identified Ten Big Ideas for Making Student Learning Assessment Useful and Used …

Student learning assessment is useful and used when… 1. You know how the results will be used, and by whom, and you plan your assessments accordingly.

Form Follows Function Why are you assessing student learning? How do you want to use student learning evidence? What decisions will student learning evidence inform? Who will make those decisions? What student learning evidence do you need for those people to make those decisions?

Student learning assessment is useful and used when… 2. Everyone focuses on what’s most important.

We are spread too thin Make great teaching a true, top college-wide priority Limit your goals 3-6 per program (1-2 for short certificates) 1-3 per course

Student learning assessment is useful and used when… 3. People have incentives to improve and change.

“Institutional values are communicated by what gets funded and what gets rewarded." Timothy Reese Cain and Pat Hutchings

Value Innovation… and Some Risk-Taking A great teacher is always improving. ▫ Promotion & tenure criteria Teaching excellence Scholarship of teaching Freedom to fail Rewards for using disappointing results ▫ Permanent fund for “curriculum development grants” Miramar: needs to encourage support of administration in funding/ rewarding.

Student learning assessment is useful and used when… 4. You have a culture of using good-quality, systematic evidence to inform decisions.

Visit the college’s Program Review/ Student Learning Outcome Assessment Subcommittees and/or the Research Subcommittee for assistance. Miramar: needs to continually improve our research capacity at the district and on campus.

Student learning assessment is useful and used when… 5. You have a culture of collaborating to knit the pieces together.

Why share results? Good News Attract students Attract support Bad News Get help solving the problem Support from others Resources

Audiences for Results Good News Good & Bad Bad News Admissions Alumni Employers Govt. policy makers Donors Leadership team Others working on this Assessment committee Accreditors Others who can help Student development staff Teaching/ learning center

Collaboration through… Assessment liaisons for every unit/program Assessment committee(s) synthesizing evidence and identify pervasive successes and issues ▫ Faculty, but also ▫ Student development & student support Flexible reporting expectations Miramar: needs to improve collaborative structures

Finding Time and Space to Discuss and Use Assessment Results Spend less time on less important work. ▫ Committees ▫ Activities with low cost-benefit ▫ Consolidate low-enrollment programs ▫ Consolidate low-enrollment courses Create occasions for collaborative discussion. Hold virtual, asynchronous discussions.

Student learning assessment is useful and used when… 6. Decision-makers see and understand the results.

Useful assessment findings “pop" Data are not information. Simple, user-friendly reports are more likely to be used than long, obtuse ones.

Averages have little meaning. Example- English 101 Average (4 pt scale) Thesis 2.5 Organization 2.7 Intro/conclusion Body paragraphs 2.6 Mechanics 3.0 Source material

Averages have little meaning. Look at percentages. 4 3 2 1 Thesis 25% 35% 17% 13% 9% Organization 26% 32% 24% 16% 1% Intro/conclusion 27% 37% 22% 12% 2% Body paragraphs 34% 23% 19% Mechanics 41% 33% 8% Source material 21% 29% 4%

Make results pop: Sort results from highest to lowest Meets or Exceeds (3 or 4) 4 3 2 1 Mechanics 74% 41% 33% 16% 8% 1% Intro/conclusion 64% 27% 37% 22% 12% 2% Thesis 60% 25% 35% 17% 13% 9% Organization 59% 26% 32% 24% Body paragraphs 57% 34% 23% 19% Source material 51% 21% 29% 4%

Student learning assessment is useful and used when… 7. Faculty are familiar with recent research on how to help students learn and succeed.

Offer evidence-based professional development to faculty. ▫ Including evidence-based teaching Offer mentoring. Create ways to share local good practices. Miramar: Needs to engage administration, faculty, staff and utilize committees.

Student learning assessment is useful and used when… 8. Learning outcomes, curricula, learning activities, and assessments are all clear, connected, and well-designed to help students achieve important outcomes.

The clearer your goals, the more useful your results. Learning goals

If you’re not teaching it, you can’t improve student learning of it. Learning goals Curricula

Course SLO Conceptualization This is what you’ll learn to do. This is what you’ll do to learn it. This is how you’ll show me that you’ve learned it. 1. 2. 3.

Course SLO Integration This is what you’ll learn to do. And this will help you learn how to… This is what you’ll do to learn it. This is how you’ll show me that you’ve learned it. 1. [Program SLO] [ISLO] 2. 3.

Construct meaningful activities. Learning goals Curricula Learning activities Construct meaningful activities.

Great learning activities have 3 transparent traits that connect to each other Mary-Ann Winkelmes

1. Purpose Explain what you want students to learn through the assignment and why. Learning Outcomes

2. Task What should the completed assignment look like? Guide students with a written “prompt.” Aim students in the right direction without giving anything away. What should the completed assignment look like? What resources can they use? Break large assignments into pieces.

3. Grading Criteria (Rubric) Give students the rubric. Create transparency. Spend more time giving the assignment and guiding it And less time grading it

Grading criteria (Rubrics) Learning goals Grading criteria (Rubrics) Curricula Learning activities Complete the circle of assessment

A little bit about rubrics… Rubrics have no widely accepted definition. The number of performance levels may affect validity. Use language that students understand. Use concrete, observable terms. ▫ Minimize subjective, evaluative terms.

Student learning assessment is useful and used when… 9. You have clear, defensible standards and targets for student performance.

What is a standard? The minimum level students need in order to: ▫ Succeed in the next course. ▫ Succeed in the workplace. ▫ Succeed in the next degree program. Passing work (C? C-? D?: The level that would not embarrass you) Defined clearly and defensibly.

What is a target? The proportion of students you want to meet your minimally adequate standard ▫ If essential for health & safety, 100% ▫ If essential for subsequent success, close to 100% ▫ If not essential… why is it a key outcome? Would you be happy if all your students were minimally adequate…but none were exemplary? ▫ Consider an additional target for outstanding or exceptional performance.

Student learning assessment is useful and used when… 10. You periodically reflect on what you’ve been doing and how useful it’s been.

If an assessment hasn’t been useful, stop doing it and do something else. If an assessment has been somewhat useful but has taken a lot of time and work, consider doing something different.

So what to do now? Develop a regular cycle of discussion for student learning in your department or unit. Use Course SLO Conceptualization and Development Checklist to guide discussions on improving the usefulness of outcomes assessment

Resources Suskie, L. (2014). Five dimensions of quality: A common sense guide to accreditation and accountability. San Francisco: Jossey-Bass. Kuh, G. D., Ikenberry, S. O., Jankowski, N. A., Cain, T. R., Ewell, P. T., Hutchings, P., & Kinzie, J. (2015). Using evidence of student learning to improve higher education. San Francisco: Jossey-Bass.