SUNY Oneonta’s CLA Results: 2008-09 College Senate SUNY Oneonta October 5, 2009

Presenter Patty Francis Associate Provost for Institutional Assessment & Effectiveness

Background Information Oneonta’s original SCBA plan proposed to use SUNY rubrics to assess Critical Thinking in 2008-09 Chairs’ proposal in Fall 2007 to replace rubrics with test such as Collegiate Learning Assessment (CLA) Concern about reliability and validity of rubrics Concern about large number of courses/faculty that would necessarily be involved in use of rubrics Development of formal proposal for consideration by College Senate in Spring 2008 Approval by College Senate of proposal on March 31, 2008

About the CLA Not a multiple choice test – requires students to actively engage with materials presented over a computer in 60- to 90-minute session Analyses adjust for students’ Entering Academic Ability and provide results for first-year students, graduating seniors, and “value added” Assesses Critical Thinking and Writing Two types of tasks Performance Analytic Writing Make-an-Argument (i.e., SUNY CT outcome #2) Break-an-Argument (i.e., SUNY CT outcome #1)

ADMINISTRATION DETAILS Student Recruitment and Sampling Used cross-sectional approach, comparing first-semester students in Fall 2008 (n=123) to graduating seniors (n=109) in Spring 2009 Senior sample included 73 native, 36 transfer students Preparation of sampling plan for CAE assuring comparability between sample and student populations Mean SAT scores for first-year participants = 1097 (compared to 1096 for entire class) Mean SAT scores for seniors = 1076 (compared to 1077 for entire class) Student Participation and Motivation Reliance on first-year seminar classes for freshmen and academic departments for seniors Emphasis to students on benefits to institution, programs, and themselves Provision of individual student results to programs and to students

Results First-Year Students Graduating Seniors Performed at above expected levels on Overall Score, Performance Task, Analytic Writing Task, and Make-an-Argument Performed at expected levels on Critique-an-Argument Performed higher than 76% of comparison institutions Graduating Seniors Performed at well above expected levels on Critique-an-Argument Performed higher than 89% of comparison institutions!

Results (cont.) Value Added Performed at expected levels on Overall Score, Performance Task, and Make-an-Argument Performed at above expected levels on Analytic Writing Task and Critique-an-Argument Performed higher than 68% of comparison institutions Native vs. Transfer Students (Senior Sample) Analytic Writing – 82% of native students scored at expected/above expected levels compared to 81% of transfers Make-an-Argument – 79% of native students scored at expected/above expected levels compared to 62% of transfers Critique-an-Argument – 72% of native students scored at expected/above expected levels compared to 81% of transfers

Who Were Our Peers? 190 institutions listed in CLA Final Report, ranging from very small privates (Alma College in Michigan) to large publics (Arizona State) Two SUNY institutions (Oneonta, Buffalo State) US News and World Report overall rankings 53 in top 50 for their regions 11 in top 10 (Truman State, Drake University) 8 in top 11-20 12 in top 21-30 9 in top 31-40

What’s Next? Results provided to GEAC for review and recommendations Approval of CLA by GEAR Group for SCBA in Spring 2009, with pilot scheduled for 2010-11 Most important, we need to use results Academic departments with at least 7 or more graduating seniors who took the CLA have received their students’ results CLA’s “CLA in the Classroom” initiative, either at workshop sites or on campus Campus-wide discussion about critical thinking? Other ideas?

SUNY Oneonta’s CLA Results: 2008-09 College Senate SUNY Oneonta October 5, 2009