Collegiate Learning Assessment Montclair State University

Assessing Learning Outcomes in General Education  The assessment of learning outcomes related to general education is essential for continued accreditation by the Middle States Commission on Higher Education  Standard 12 of Middle States’ “Characteristics of Excellence” asks institutions to provide evidence that their students “demonstrate college-level proficiency in general education”  Standard 14 asks institutions to continuously assess learning outcomes, including general education outcomes  The University has agreed to assess learning outcomes in general education through its participation in the Voluntary System of Accountability (VSA)  The General Education Committee reviewed available instruments, and determined that the Collegiate Learning Assessment (CLA) was most suitable for the University

The Instrument  The Collegiate Learning Assessment (CLA) assesses Analytic Reasoning and Evaluation, Writing Effectiveness, Writing Mechanics, and Problem Solving  It relies on constructed-response tasks rather than on multiple-choice items  Students are asked to complete one Performance Task, or a combination of one Make-an-Argument, and one Critique-an-Argument, prompt  Performance Tasks include a document library that students use to answer several open-ended questions about a hypothetical, but realistic, situation  Make-an-Argument prompts ask students to support or reject a position on some issue, while Critique-an- Argument prompts ask students to evaluate the validity of an argument made by someone else

CLA Administration Fall 2010  100 Freshmen Tested  Sampling Universe: First- time Freshmen with available SAT scores (N=2,101)  Invitations to random sample of students with participation voluntary  Incentive: $75 Red Hawk  Online test in a proctored setting in CADA Spring 2011  100 Seniors Tested  Sampling Universe: “Native” Seniors with available SAT scores (N=1,125)  Invitations to random sample of students with participation voluntary  Incentive: $75 Red Hawk  Online test in a proctored setting in CADA

Representativeness of Test-Takers  Freshman and Senior test-takers were generally representative of the entire population  Freshman test-takers had higher SAT scores, higher HS GPA, and lower HS rank, but none of the differences were statistically significant  Senior test-takers had lower SAT-M scores, higher SAT- CR and SAT-W scores, and higher college GPA, but only the GPA difference was statistically significant  Differences between test-takers and non-test-takers for both freshman and senior populations were statistically significant for race/ethnicity, but not gender  Differences between test-takers and non-test-takers, by College/School, were statistically significant for seniors, but not for freshmen

Means Comparisons

Freshman Representativeness

Senior Representativeness

CLA: Unadjusted Performance Freshman Means  Total Score: 1073  Performance Task: 1033  Analytic Writing: 1113  Make Argument: 1113  Critique Argument: 1110  SAT: 1026 Senior Means  Total Score: 1177  Performance Task: 1198  Analytic Writing: 1155  Make Argument: 1135  Critique Argument: 1174  SAT: 1013

CLA: Value-Added & Estimates Value-Added Score  Total Score: 0.71  Performance Task: 1.26  Analytic Writing:  Make Argument:  Critique Argument: 0.14 Expected  Total Score: Near  Performance Task: Above  Analytic Writing: Near  Make Argument: Near  Critique Argument: Near

CLA: Fr. Sub-Score Comparisons MSUAll Freshmen Difference Performance Task: Reasoning Writing Effectiveness Writing Mechanics Problem Solving Make an Argument: Reasoning Writing Effectiveness Writing Mechanics Critique an Argument: Reasoning Writing Effectiveness Writing Mechanics

CLA: Sr. Sub-Score Comparisons MSUAll Seniors Difference Performance Task: Reasoning Writing Effectiveness Writing Mechanics Problem Solving Make an Argument: Reasoning Writing Effectiveness Writing Mechanics Critique an Argument: Reasoning Writing Effectiveness Writing Mechanics

Performing Beyond Expectations

Next Steps?  Faculty analysis of results  Development of “actionable” items  Implementation of proposed enhancements  Follow-up assessment of effects of changes