Taking Institutional Level Assessment to the Course Level Journey – Challenges – Discoveries Mesa Community College DIVING INTO DEEPER LEARNING

MEET YOUR PRESENTERS Diana Bullen Faculty – Business Director Business Management Programs Assessment Coordinator Marjorie Leta Faculty – Librarian Past Assessment Coordinator Past Chair Student Outcomes Committee Assessment Coach Lindsey Pedersen Faculty – Art History Student Outcomes Committee Champion Guided Pathways Lead

ABOUT MCC  21,900+ Students  Multiple Sites  Online Classes  International Students

TODAY’S AGENDA  Brief history of assessment at MCC  The Journey – Creating a new culture of assessment  The Challenge – Motivating faculty to embrace the new culture of assessment  The Discovery – Collaborating is energizing

TODAY’S LEARNING OUTCOMES  Review the assessment journey at MCC  Examine your college’s assessment practices  Develop an action plan to take back to your college

ASSESSMENT HISTORY AT MCC CIRCA 1996  Faculty driven via Student Outcomes Committee (SOC)  Supported by VPAA & Institutional Effectiveness  Faculty clusters created/selected assessment tools

THE PROCESS FROM  Outcomes Assessed annually on a rotating basis  Assessment Week – last week in February  Faculty volunteers  Compare pre & post groups  Winner of 2007 CHEA Award for Institutional Progress in Student Learning Outcomes

THE PROCESS WAS DISCONNECTED  Assessment not relevant to individual courses  Students never saw scores  Faculty participation was limited  ‘Closing the Loop’ was difficult

FIVE STAGES OF DEVELOPMENT  Stage 1: Review the Process  Stage 2: Develop Broad ISLOs  Stage 3: Develop Broad Scoring Guidelines  Stage 4: Implementation  Stage 5: Gather Valid Data at the Institutional Level

STAGE 1: THE REVIEW INFORMED IMPROVEMENT (ii) SOC began the ii process in fall 2012 to research assessment options, gather evidence and implement an action plan for change. Ask a Question: How can the SOCii Team facilitate the effective integration of general education outcomes assessment throughout all MCC programs and departments?

STAGE 1: THE REVIEW – MCC’S ISLOs – Past and Present MCC’s 4 Cs Communication Critical Thinking Civic Engagement Cultural & Global Engagement From 18 ISLOs to 4

Change Driven by...  SOCii Team  Faculty Commitment  New Assessment Criteria from HLC:  Clearly stated goals for student learning and effective processes  Uses the information gained from assessment to improve student learning  Substantial participation of faculty and other instructional staff members

STAGE 2: CREATING BROAD ISLOs In fall 2013, SOC held a series of open, drop-in workshops inviting all faculty to participate and collaborate in the revision and creation of our new ISLOs. The intent being to develop ISLOs that were:  relevant to all subjects  meaningful to faculty and students  supported by faculty senate And that could be assessed at the course level

STAGE 2: CREATING BROAD ISLOs - THE PROCESS Consider… College Vision, Mission & Values Current/Previous SLOs Curriculum Community Consult…  Bloom’s Taxonomy  AAC&U VALUE Rubrics Information Literacy Numeracy Problem Solving Critical Thinking

AAC&U VALUE RUBRICS V alid A ssessment of L earning in U ndergraduate E ducation The VALUE rubrics were developed by teams of faculty experts representing colleges and universities across the United States through a process that examined many existing campus rubrics and related documents for each learning outcome and incorporated additional feedback from faculty. - AAC&U

MCC’S 4C S WERE BORN… The Keys to Deeper Learning

STAGE 3: CREATING SCORING GUIDELINES FOR ASSESSING ISLOs Achievement Level 4 (Expert) Achievement Level 3 (Proficient) Achievement Level 2 (Developing) Achievement Level 1 (Needs Improvement)

STAGE FOUR: IMPLEMENTATION 1.Introduction  Mapping a course and/or program  Identifying an assessment artifact and matching it to a Scoring Guideline 2.Pilot Program:  Taking it to the course level Your Course ISLO

STAGE FOUR: IMPLEMENTATION 1. INTRODUCTION - LESSONS LEARNED Competencies do not match what is happening in the classroom!  Many competencies need updating to reflect current trends  Many competencies are written below Bloom’s application level  Some faculty resistance on process

Scoring Guidelines: Critical Thinking (CT) Communication (CO) Civic Engagement (CE) Cultural & Global Engagement (CG) Data Collection via Canvas Course or PrefixOutcome Faculty/Departments/ProgramsOffice of Institutional Effectiveness 1. Identify one ‘C’ that best aligns with your course 2. Identify a cumulative assignment that assesses ALL criteria of the aligned ‘C’ 3. Attach ‘C’ Scoring Guidelines to the cumulative assignment rubric in Canvas & score mastery upon completion Course Level Assessment Pilot : Spring 2015 EXS101=CT SPA115 =CG Art116 =CO EDU282 =CE

STAGE 4: IMPLEMENTATION 2. PILOT IMPLEMENTATION SPRING 2015  871 students were assessed in 33 different courses  358 students were assessed in Communication  289 students were assessed in Civic Engagement  168 students were assessed in Critical Thinking  88 students were assessed in Cultural and Global Engagement 34 Residential Faculty – 2 Adjunct Faculty 64 sections (Assessment Coaches were selected to assist faculty throughout the college)

STAGE 4: IMPLEMENTATION FALL Different Courses assessed (More than double the pilot) 52 Residential Faculty – 14 Adjunct Faculty (94 faculty were expected to participate – actual 71) 117 sections (Assessment Coaches were selected to assist faculty throughout the college)

NILOA REPORT 2014 NATIONAL INSTITUTE This level of alignment suggests more careful attention to integrating assessment activities on campus.

CONCLUSIONS The Challenge Academic Freedom The Discovery College-wide collaboration The Journey Innovation Excitement MCC’s 4Cs: Keys to Deeper Learning

WHY DO WE ASSESS?  Assess to Assist  Assess to Adjust  Assess to Advance COLLABORATION  Assess to Assist  Assess to Adjust  Assess to Advance COLLABORATION

STAGE 5: GATHER VALID DATA AT THE INSTITUTIONAL LEVEL  Formed a dedicated committee to develop guidelines to:  Engage in evidence-based decision making that:  Enhances the quality of programs and services  Enhances ongoing assessment processes  Informs planning and resource management  Collect evidence through a variety of methods and multiple sources  Create consistency throughout the assessment process  Interpret evidence and its validity Though assessment can be useful for external reporting, its primary purpose is as a systematic method for making decisions that enhance the learning environment

Activity  Examine your college’s assessment practices:  What processes are in place?  What processes need to be revised?  Develop an action plan to take back to your college:  Who should be involved?  Create a timeline of events essential to improve the process

QUESTIONS?

SOURCES National Institute for Learning Outcomes Assessment (NILOA)  American Association of Colleges & Universities The Primer Series by Ruth Stiehl and Les Lewchuk  Bloom’s Taxonomy  Bloom, et al. (1956). Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain. New York: David McKay Company.