ASSESSMENT OF THE GEN ED PROGRAM – 2015-16 GEN ED 2.0 TASK FORCE 21 SEPTEMBER 2016

Overview Part 1: Why we assess. Part 2: How we currently assess the UMKC Gen Ed Core. Part 3: What we found in 2015-16. Part 4: What we are learning from the findings.

Three (or more) levels of assessment Course-level assessment Program-level assessment (degree, minor, certificate) Institutional-level assessment – the Gen Ed Program

Part 1: Why we assess. Why do we engage in assessment?

The fine print Accountability State regulations HLC criteria If we engage effectively in assessment, we will meet external demands for accountability.

Why we assess Assessment helps us make sure that we are fulfilling the promises we make to our students and to society (Suskie, 2010; Finley, 2014) Our promises to students: Institutional mission and values statements Program’s mission statement Program’s student learning outcomes What is an SLO? How do we know one when we see it? What student are able to do = <action verb> = <one something> Parallel with outcomes for advising/advisors – demonstration of abilty

Purposes of Assessment Ensuring that we are delivering on what we care about: Students get the best possible education Students have access to the best possible learning environments, co-curricular programs, and support systems Ensuring that learning inside and outside the classroom is of appropriate scope, depth, and rigor Asking – and answering - questions we care about concerning student learning and development Identifying and preserving good practices in supporting student learning

The Important Questions Three questions to answer through assessment of student learning outcomes (SLOs): What have our students learned? Are we satisfied with what they’ve learned? If not, what are we going to do about it? (Eder, 2010)

Part 2: How we currently assess the General Education Core1. 1We will continue to assess the core we have. We will continue to assess the core we have.

Three methods – two direct, one indirect Authentic Assessment of Gen Ed SLOs Review of student artifacts2 from gen ed courses ETS Proficiency Profile – standardized exam required of baccalaureate-seeking students prior to graduation Reading, critical thinking, writing, and mathematics National Survey of Student Engagement (NSSE) and the Experiences with Writing Module 2student work produced as a normal part of course requirements Eligible to sit for ETS PP after complete 90 hours

How we got to where we are.

Process – Artifact Collection Reviewed number of courses offered in Spring 2016 Discourse 200, Anchor 200, and Focus A grouping had more than others Drew random sample of 125 students for each course category Sent email to instructors with names of students drawn in sample with information on submitting student artifacts and assignment guidelines Docs sent to me, stripped of faculty and student identifiers

Process – Artifact Review Three rubrics developed based on AAC&U VALUE rubrics – reviewed by UAC and GECC (posted on Assessment Website) Review panel members recruited – Jerry Wyckoff and others Panels convened on May 24 and 25, 2016 18 faculty and staff panel members plus three facilitators Panel members received honoraria of $200 + meals Rubric norming – 3 groups (Anchor, Discourse, Focus) Each artifact reviewed by at least two panel members

Part 3: What we found in 2015-16.

Authentic Assessment: Results - Summary

General Education Core Assessment – Spring 2016: Anchor 200, Discourse 200, Focus A Summary Data Tables Outcomes for which students are meeting or exceeding expectations? Outcomes for which student are not meeting expectations?

Meeting expectations: Discourse 200: Context and Purpose Content Development Genre or Disciplinary Conventions Syntax and Mechanics Anchor 200: Contextualize Information 69% achieving 4 or higher OK for 200 level? Focus A: (Small Ns) Analyze Human Condition – Methods and Sources 68% OK? Also developed in Discourse 69%

Outcomes on which student not meeting expectations? Interdisciplinary Thinking – Anchor 200: Think beyond the discipline - 21% achieved a 4 or higher Focus A: Analyze connection with other areas of inquiry - 22% Culture and Diversity Anchor 200: Describe Global Culture – 49% Anchor 200: Describe Culture Identity Factors – 44% Discourse 200: Culture or Global Diversity – 56% Only courses in which specifically addressed in gen ed program Findings are a product of how the SLOs are written, how the educational experiences are designed to achieve the SLOs, and how the assignments are designed to elicit the desired learning outcomes – and how they are assessed.

Some Lessons Learned Some Gen Ed SLOs - revise to ensure measurable Version 2.0 must ensure outcomes are measurable Set targets for student achievement – What is good enough? Course design – Course SLOs must be mapped to identified Gen Ed SLOs Educational experiences must be present to support achievement of SLOs Assignments must be designed to elicit the identified SLOs Observation by Panel Members: The clearer and more detailed the assignment, the better the quality of student work

ETS® Proficiency Profile Administered to senior students Eligible after complete 90 hours Must sit for the exam prior to graduating Who has heard of the EPP? How long has UMKC required the EPP? Prior to today, who has seen the results?

About the ETS® Proficiency Profile (EPP) a nationally-normed, multiple-choice test designed to assess students’ abilities in four areas: mathematics, reading, writing, and critical thinking the learning outcomes are tested within the context of the humanities, social sciences, and natural sciences, although (disclaimer) the exam does not measure specific content in these areas, as all needed information is contained in the questions. results are reported annually to the Missouri Department of Higher Education, as required by State statute and the policies of the Coordinating Board of Higher Education.

What are the data telling us? What are the institutional trend date telling us? What are/should be our expectations for student achievement? What are the comparative data telling us? How can we use these data?

About the NSSE administered to first-year and senior students (every three year’s at UMKC) survey designed to evaluate students’ engagement in programs that support their learning and personal development, and the institution’s support of evidence-based good practices in undergraduate education, e.g., high-impact practices

Experience with Writing Module result of a collaboration between NSSE and the Council of Writing Program Administrators. Items touch on three aspects of good writing assignments— interactivity, meaning-making, and clarity. complements questions on the core survey about how much writing students do, the nature of their course assignments, and perceived gains in written expression.

What are the Experience with Writing data telling us? Are the results at a level you expected? Are the results at a level that is acceptable?

Part 4: What we are learning from the findings.

What are the implications of the data? Ensure Gen Ed SLOs are measurable Design of the Gen Ed Program: ensure that there are sufficient and appropriate educational experiences to support achievement of the desired outcomes; ensure that there are assignments designed to elicit the SLOs and that can be assessed across the program; and set appropriate targets for student achievement in the aggregate of the SLOs (what is good enough). Ensure there is a feedback loop for assessment results and accountability for making programmatic/course alternations to enhance student learning General Education is everyone’s responsibility