1. June 2008 Making Sense of Assessing Student Learning Outcomes Southampton, UK June 26, 2008 Randy Swing, Executive Director Association for Institutional Research

2. June 2008 Assessment – U.S. Context Not grading (in US) (but may use grades) Quality Assurance Program Improvement

3. June 2008 Successful Assessment.... To.... Continue Effective Practice or Initiate Change (improvement)

4. June 2008 FAQ – Why do we need assessments beyond grading? U.S. Context Grades are a composite measure Learning Attendance Compliance with class rules Seldom validated More

5. June 2008 Grades: A Valuable Data Source? Researcher’s Dream Vary across and among participants Continuous variables in standard increments Readily available Widely held to be important indicators of success Researcher’s Nightmare Volunteer effect Unique treatments No random assignment – not experimental design

6. June 2008 Grades Matter! Major Predictor Persistence into the Sophomore Year Graduation Source: Descriptive Summary of 1995-1996 Beginning Postsecondary Students: Six Years Later (NCES 2003-151) First Year GPABachelors in 6 yrs Less than 2.2526% 2.25 – 3.2564% Over 3.2575%

7. June 2008 How much failure should we tolerate? 33 4-yr colleges & universities 159 High Enrollment Courses 17 2-yr institutions/ systems 86 High Enrollment Courses 1.Identify the 5 courses that annually enroll the largest number of new students. 2.Determine the DFWI rate for each course.

8. June 2008 Average DFWI Rate (percent of students): 25% (4-yr) 35% (2-yr) Range of DFWI Rates (percent of students): 1% to 79% (4-yr) 9% to 58% (2-yr) Percent of Courses Over 33% DFWI: 25% (4-yr) 59% (2-yr) Five Courses with Highest Enrollment of New Students Source: Kathy Morley and Julie Alexander, Policy Center on the First Year of College – unpublished 2006 DFWI Rate

9. June 2008 Alternatives to Grades (Selected examples) Standardized tests –Admissions tests (SAT and ACT) –Collegiate Learning Assessment (CLA) –ETS Major Tests –State Junior/Exit testing Proxy Measures –National Survey of Student Engagement

10. June 2008 Student Engagement Outcomes Course Contents Course Delivery Structures Student Prior Experiences & Characteristics At Entry Randy L. Swing, 2003 - Adapted from works by Astin and Terenzini Model for Assessing Learning in a Course Course Goals Out of Class Environments

11. June 2008 Contact Information Randy L. Swing, Ph.D. Executive Director, Association for Institutional Research 1435 E. Piedmont Drive Tallahassee, FL 32309 rswing@airweb2.org