1. Collegiate Learning Assessment 2010-11 Montclair State University

2. Assessing Learning Outcomes in General Education  The assessment of learning outcomes related to general education is essential for continued accreditation by the Middle States Commission on Higher Education  Standard 12 of Middle States’ “Characteristics of Excellence” asks institutions to provide evidence that their students “demonstrate college-level proficiency in general education”  Standard 14 asks institutions to continuously assess learning outcomes, including general education outcomes  The University has agreed to assess learning outcomes in general education through its participation in the Voluntary System of Accountability (VSA)  The General Education Committee reviewed available instruments, and determined that the Collegiate Learning Assessment (CLA) was most suitable for the University

3. The Instrument  The Collegiate Learning Assessment (CLA) assesses Analytic Reasoning and Evaluation, Writing Effectiveness, Writing Mechanics, and Problem Solving  It relies on constructed-response tasks rather than on multiple-choice items  Students are asked to complete one Performance Task, or a combination of one Make-an-Argument, and one Critique-an-Argument, prompt  Performance Tasks include a document library that students use to answer several open-ended questions about a hypothetical, but realistic, situation  Make-an-Argument prompts ask students to support or reject a position on some issue, while Critique-an- Argument prompts ask students to evaluate the validity of an argument made by someone else

4. CLA Administration Fall 2010  100 Freshmen Tested  Sampling Universe: First- time Freshmen with available SAT scores (N=2,101)  Invitations to random sample of students with participation voluntary  Incentive: $75 Red Hawk  Online test in a proctored setting in CADA Spring 2011  100 Seniors Tested  Sampling Universe: “Native” Seniors with available SAT scores (N=1,125)  Invitations to random sample of students with participation voluntary  Incentive: $75 Red Hawk  Online test in a proctored setting in CADA

5. Representativeness of Test-Takers  Freshman and Senior test-takers were generally representative of the entire population  Freshman test-takers had higher SAT scores, higher HS GPA, and lower HS rank, but none of the differences were statistically significant  Senior test-takers had lower SAT-M scores, higher SAT- CR and SAT-W scores, and higher college GPA, but only the GPA difference was statistically significant  Differences between test-takers and non-test-takers for both freshman and senior populations were statistically significant for race/ethnicity, but not gender  Differences between test-takers and non-test-takers, by College/School, were statistically significant for seniors, but not for freshmen

6. Means Comparisons

7. Freshman Representativeness

8. Senior Representativeness

9. CLA: Unadjusted Performance Freshman Means  Total Score: 1073  Performance Task: 1033  Analytic Writing: 1113  Make Argument: 1113  Critique Argument: 1110  SAT: 1026 Senior Means  Total Score: 1177  Performance Task: 1198  Analytic Writing: 1155  Make Argument: 1135  Critique Argument: 1174  SAT: 1013

10. CLA: Value-Added & Estimates Value-Added Score  Total Score: 0.71  Performance Task: 1.26  Analytic Writing: -0.11  Make Argument: -0.30  Critique Argument: 0.14 Expected  Total Score: Near  Performance Task: Above  Analytic Writing: Near  Make Argument: Near  Critique Argument: Near

11. CLA: Fr. Sub-Score Comparisons MSUAll Freshmen Difference Performance Task: Reasoning2.72.8-0.1 Writing Effectiveness2.93.0-0.1 Writing Mechanics3.03.1-0.1 Problem Solving2.82.9-0.1 Make an Argument: Reasoning3.53.20.3 Writing Effectiveness3.63.20.4 Writing Mechanics3.63.40.2 Critique an Argument: Reasoning3.12.80.3 Writing Effectiveness3.22.90.3 Writing Mechanics3.63.40.2

12. CLA: Sr. Sub-Score Comparisons MSUAll Seniors Difference Performance Task: Reasoning3.63.40.2 Writing Effectiveness3.83.50.3 Writing Mechanics3.83.50.3 Problem Solving3.63.40.2 Make an Argument: Reasoning3.6 0.0 Writing Effectiveness3.63.7-0.1 Writing Mechanics3.8 0.0 Critique an Argument: Reasoning3.3 0.0 Writing Effectiveness3.53.40.1 Writing Mechanics4.03.90.1

13. Performing Beyond Expectations

14. Next Steps?  Faculty analysis of results  Development of “actionable” items  Implementation of proposed enhancements  Follow-up assessment of effects of changes