Assessment Day 2017; Triton College “Closing the Loop” Who tells your story? Jen Sweet, PhD Associate Director Office for Teaching, Learning & Assessment DePaul University Shannon Milligan, PhD Assessment Coordinator Faculty Center for Ignatian Pedagogy Loyola University Chicago Shannon Assessment Day 2017; Triton College

IMPORTANCE OF TELLING OUR STORY “Assessments are worthwhile only if the results are put to good use, and those uses can take place only after careful consideration and discussion. That consideration and discussion, in turn, can take place only if assessment results are communicated usefully, clearly, and accurately.” (Linda Suskie) Yet Banta (2009) [reinforced by McNeice-Stallard and Stallard (2012)], found that colleges struggle with using assessment results to drive continuous improvement, aka “closing the loop” We want to be the ones to tell and craft our stories! - If we don’t, someone else will Shannon

Part I. Telling Your Story Up to Present What is Happening? Part II. Crafting Your Story Moving Forward Where are we Going? Fidelity: Make Sure the Story We Tell is Nonfiction Shannon

TOOLS FOR UNDERSTANDING YOUR CURRENT STORY Assessment “Assessment is the systematic collection of information about student learning, using the time, knowledge, expertise, and resources available, in order to inform decisions about student learning.” Walvoord, B. (2010). Assessment Clear and Simple. Jossey Bass: USA, p. 2. Program Review “Program Review is a rigorous, systematic, objective, impartial, expert-based examination, evaluation and self- evaluation of how effectively a program is working.” Macau University of Science and Technology (2017). http://www.must.edu.mo/en/qa-en/program-review Jen

HOW DO ASSESSMENT AND PROGRAM REVIEW RELATE? HOW DO THEY DIFFER? Focus Process Products Goal Improvement Involvement Faculty/Staff Faculty/Staff, Deans, Provosts Level Activity/Course/Program Program Purpose Assurance of quality; continuous improvement Make value judgments (strengths, weaknesses, opportunities for improvement, directions for future) Scope Narrow (Generally a single learning outcome) Broad (An entire program, department, college) Timing Ongoing (conceptually no end) One time event (generally repeated periodically) Jen *Results of Assessment are an important component of Program Review, but Program Review looks at additional metrics, such as: Enrollment Patterns Graduation Rates Demand in the Field Student Satisfaction

LEVELS OF ASSESSMENT Institutional Program Activity/Course Jen

TOOLS FOR CRAFTING YOUR STORY Where do we want to go? How can we improve/grow? Are there things we should change moving forward? Memorandum of Understanding/ Memorandum of Agreement Items Specific action items you’ve identified over the course or program review to make improvements to the program (generally focus on areas with potential for highest impact on program quality) Closing the Loop on Assessment Plans for improvement that include specific action items Jen

Are Students Achieving LOs? THE ASSESSMENT LOOP Are Students Achieving LOs? Gathering Evidence Making Meaning Plans to Improve Taking Action Learning Outcomes Shannon

WHAT IS “CLOSING THE LOOP”? Taking Action on the Results of our Assessment Projects Using assessment results to inform change Communicating results A Strategy to Ensure Assessment Projects are not “One Off” Are Students Achieving LOs? Gathering Evidence Making Meaning Plans to Improve Taking Action Shannon

EVIDENCE BASED STORY TELLING A Great Way to Think About “Closing the Loop” on Assessment “Evidence of student learning is used in support of claims or arguments about improvement and accountability told through stories to…a specific audience” (Jankowski, 2012) This is how we know WHAT and THAT our students are learning Communication of results to stakeholders serves to “tell an important story with a meaningful point” (Suskie, 2009). Details can include: Context of your course/program and students Information learned from previous assessments Process for assessment (e.g. work collected and method of analysis) Findings from assessments (evidence) Shannon

WHY WORRY ABOUT CLOSING THE LOOP? Our students, courses, programs, and even fields, are NOT static Improve our courses and programs Ensure our students are achieving optimal learning in our courses and programs Return on the time and effort you put into the assessment process Better integration across the institution Eliminating “silos” We have to...HLC requires that we “close the loop” on assessment 4.B. subcomponent 3: The institution uses the information gained from assessment to improve student learning. Jen

CHALLENGES TO CLOSING THE LOOP Inability to Make Causal Claims (tracking students post-grad, etc.) Assessment Results not Generally Associated with Additional Resource Allocation Underlying Assumption that there is Something Wrong? Lack of Perfection = Justification for not Taking Action Methodology, data, etc. not perfect Conflation of assessment with other inquiry processes (evaluation, research) Fear of Constantly Re-Inventing the Wheel Need to Make Meaning of the Data “Shortest Straw” Method of Faculty Participation in the Process Sometimes the Problem is with the Process Shannon

EXAMPLES Student stakeholders: During the first class meeting of the semester, a Psychology faculty member is reviewing the syllabus with her class. As she reviews the syllabus, she notes what changes she made from the last time she taught the course. For example, a new lesson on neuroscience was added to better prepare students for a course they will take later in the major. She also notes that changes were made to the structure of the final project based on constructive feedback received both in class and on course evaluations. Curriculum mapping: Upon assessing the learning outcome: ‘students will be able to cite sources appropriately,’ a history professor determines that all of the sources students are citing are secondary sources. Looking back at the course, the history professor realizes there is nowhere in the curriculum where he specifically teaches students to distinguish between primary and secondary sources. Shannon

At your tables, please read and discuss the Case Study. CASE STUDY ACTIVITY At your tables, please read and discuss the Case Study. Do the results help answer the burning question? Why or why not? Are the results actionable? If yes, what actions would you recommend the program consider taking? If no, what would make the results more actionable? What could have been done differently to produce more actionable results? What, if anything, has this case taught you about using assessment results for program improvement? Jen

STRATEGIES FOR CLOSING THE LOOP TAKING ACTION! We will address this in our breakout sessions! NA

Questions?

REFERENCES Banta, T. W., Jones, E. A., & Black, K. E.. (2009). Designing effective assessment: Principles and profiles of good practice. San Francisco: Jossey-Bass. Jankowski, N. (2014). Closing the loop: Using assessment results to enhance student learning [PDF document]. Retrieved from http://www.learningoutcomesassessment.org/Presentations/NCICU2014.pdf Macau University of Science and Technology (2017). Retrieved from http://www.must.edu.mo/en/qa-en/program-review McNeice-Stallard, B. E., & Stallard, C.M. (2012). Measuring sustainability of outcomes assessment. The Journal of Applied Research in the Community College 19 (1): 6–13. Retrieved from http://www.niu.edu/p20network/readiness-activities/ePortfolio- materials/Arcario-Eynon-Klages-Polnariev-Closing-the-Loop-MUJ-fall-2013.pdf Suskie, L. (2009). Assessing student learning: A common sense guide. San Francisco, CA: Jossey-Bass. Walvoord, B. (2010). Assessment Clear and Simple. Jossey Bass: USA, p. 2. Thank you to Monica Stitt-Bergh from the University of Hawai‘i at Mānoa for permission to use her case study!