actionable assessment Jen Sweet, PhD Associate Director Office for Teaching, Learning & Assessment DePaul University Shannon Milligan, PhD Assessment Coordinator Faculty Center for Ignatian Pedagogy Loyola University Chicago Jen Assessment Day 2017; Triton College

Are Students Achieving LOs? THE ASSESSMENT LOOP Are Students Achieving LOs? Gathering Evidence Making Meaning Plans to Improve Taking Action Learning Outcomes Jen

Complete Exercise 1on the Reflection Worksheet Jen

CLOSING THE LOOP AT EACH STEP IN THE ASSESSMENT PROCESS Planning Gathering Evidence Making Meaning of Information Taking Action Jen

STRATEGIES FOR CLOSING THE LOOP PLANNING Integrated Course Design Keeping it Simple Curriculum Mapping Shannon

INTEGRATED COURSE DESIGN Course design framework by L. Dee Fink Emphasis on integration of course materials, assessment, and learning outcomes Recognizes the role of context (e.g. type of course, student/instructor background) Context (Situational Factors) Learning Outcomes Feedback and Assessment Teaching and Learning Activities Shannon

KEEPING IT SIMPLE Assignment Design (Kuh, 2016) Rubrics (Kuh and Fink) Survey results show that course assignments are the main way students demonstrate achievement of learning outcomes (including program level and above) Alignment is important! Assignments created to provide evidence of whether students achieve learning outcomes Can also work with other faculty to align assignments across a program (scaffolding) Library of assignments http://assignmentlibrary.org/ Rubrics (Kuh and Fink) Create more transparency of expectations More objective assessments of student performance Formative process Standardize assessment without standardizing the assignments (Kuh, 2016) Shannon

CURRICULUM MAPPING Ensure alignment of: Course learning outcomes and assignments/assessments Program learning outcomes and courses/course learning outcomes Shannon

Complete Exercise 2 on the Reflection Worksheet Shannon

STRATEGIES FOR CLOSING THE LOOP GATHERING EVIDENCE Multiple Methods Combining Quantitative and Qualitative Sometimes Need to Follow-up on Results Bringing in Other Significant Metrics Student DWF Rates Enrollment Rates Graduation or Retention Rates Jen

EXAMPLE: MULTIPLE METHODS Need to Follow-Up: Dr. M. conducted an assessment project to determine students’ abilities to “identify the social and ethical components of both the causes and solutions to environmental issues” Three alternative Chicago River separation scenarios were described and students were asked to choose which one, if any, they would recommend and to consider the scientific, social and ethical implications of their recommendation. Finding: Students were weak in their abilities to describe the implications of their recommendations. Follow-up with Qualitative Interview: Students let Dr. M. know that the reason they didn’t describe the implications of their recommendations is because they’d never actually been asked to do so (only to consider the implications). In the follow-up interviews, students did an excellent job describing the implications of their recommendations. WITHOUT FOLLOW-UP WITH A QUALITATIVE ASSESSMENT METHOD, DR. M. MAY HAVE REACHED THE FALSE CONCLUSION THAT HIS STUDENTS WERE NOT ACHIEVING THIS OUTCOME Jen

Complete Exercise 3 on the Reflection Worksheet Jen

STRATEGIES FOR CLOSING THE LOOP MAKING MEANING OF INFORMATION Need for broad participation Bringing Important Stakeholders into the Process Contextual What’s most important Priorities Consideration of Stakeholders Shannon

BRINGING IMPORTANT STAKEHOLDERS INTO THE PROCESS Students Other Faculty In the program/departments Who teach the course Program or Departmental Staff University Administrators Provost/President Deans Department Heads University Mission More Widely Sharing Results with Stakeholders Shannon

Complete Exercise 4 on the Reflection Worksheet Shannon

STRATEGIES FOR CLOSING THE LOOP TAKING ACTION! ADDRESSING ORIENTATION TOWARDS ASSESSMENT BEING REALISTIC AND CONCRETE IS THERE A NEED FOR PROFESSIONAL DEVELOPMENT? Jen

ORIENTATION TOWARDS ASSESSMENT CONTINUOUS QUALITY IMPROVEMENT Assumptions: No matter how great you are, you can always improve (even if it’s a little bit) Delivering the highest quality programs/courses requires constant monitoring There is a need to constantly ask: How are we doing? How can we do better? Jen

BEING REALISTIC AND CONCRETE Probably won’t be getting that extra faculty line during a budget crunch Build in flexibility Concrete Action Plan Who will be Involved? Identify Resources Needed Solidify a Timeline Jen

PROFESSIONAL DEVELOPMENT Internal: “The campuses that have made the most progress have invested in…professional development and have hosted venues where faculty and others come together to formulate and (to) explore questions about their students’ learning” –Kuh (2016) Encourage more focus on making assessment actionable Example: Assessment Certificate Program External: Chicago Area Assessment Group (CAAG)-local assessment community of practice The Assessment Group of Illinois Community Colleges- local assessment community of practice Association for the Assessment of Learning in Higher Education (AALHE)-national organization Higher Learning Commission (HLC)-annual conference Assessment Institute-annual conference Jen

CLOSING THE LOOP WHEN EVERYTHING IS GOING WELL Great News!! Move onto assessment of another learning outcome Plan to cycle back to assessing the learning outcome later Prioritize other learning outcomes where faculty/staff perceive there may be more opportunities for improvement! Jen

Complete Exercise 5 on the Reflection Worksheet Jen

REFERENCES Fink, L. D. (2013). Creating significant learning experiences: An integrated approach to designing college courses. San Francisco, CA: Jossey-Bass. Kuh, G. D. (2016). Getting assessment back on the right track. 2016 CAEL Forum and News (December 2016). Retrieved from: http://degreeprofile.org/wp- content/uploads/2017/02/Kuh- 2016_CFN_Assessment_Back_on_Track.pdf N/A

CONTACT INFORMATION Jen Sweet, PhD Associate Director Office for Teaching, Learning & Assessment DePaul University jsweet2@depaul.edu Shannon Milligan, PhD Assessment Coordinator Faculty Center for Ignatian Pedagogy Loyola University Chicago smilligan@luc.edu N/A