Assessing SUNY General Education Learning Outcomes with April 27, 2005

Presenters  David Chadima Consultant, Postsecondary Assessment Services, ACT  Robert Ziomek Director of Education and Workforce Research Services, ACT  Frances Dearing Assistant Dean of Assessment, Academic Support and Placement, Westchester County Community College  Lanette Raymond Research Associate, Suffolk County Community College

Presentation Outline  CAAP Overview  SUNY Outcomes / CAAP Test Module Alignment & Report Customization  Critical Thinking Pilot Studies  Suffolk County Community College  Westchester County Community College  Q&A

What is CAAP? Collegiate Assessment of Academic Proficiency

The standardized assessment program from ACT that enables postsecondary institutions to measure, evaluate, and enhance the outcomes of their general education programs.

CAAP Features  Six independent, 40-minute test modules  Flexible, in-class administration  Comprehensive reporting at the cohort and individual student levels  Extensive user norms  Linkage to ACT, COMPASS or ASSET to measure “value added”

CAAP User Norms to

CAAP User Norms  National norms for each test module (and subscore)  Broken out by:  Institution type  Year of students tested  Institution ownership  SUNY system norms (2- year & 4-year if sufficient volume.

CAAP Test Modules

 Five objective tests  Writing Skills  Reading  Mathematics  Science  Critical Thinking  One writing essay test

SUNY Learning Outcomes Knowledge and Skills Areas  Critical Thinking  Basic Communications (Written)  Mathematics

Outcomes/Test Alignment

KEY ISSUE:  For each test module, provide CAAP subscores that map to SUNY learning objectives

Critical Thinking

 32 items / 4 passages  Subject matter: Social Sciences, Natural Sciences, Humanities  Formats:  Case Studies  Dialogues  Debates  Editorials  Statistical Arguments  Experimental Results  Overlapping Position

Critical Thinking Sample Question following passage It is obvious from Favor's speech that she believes which of the following? A.Most prescriptions are unnecessary. B.Senator Counter will oppose the bill. C.If the bill is passed into law it will greatly reduce the cost of all medical treatment. D.If the bill is passed the average costs for treatment of minor ailments would be reduced significantly

Basic Communication (Written)

Writing Essay  Two independent prompts  20 minutes each  Essays scored in a holistic manner  ACT can score  Local faculty can score

Writing Essay Sample Your college administration is considering whether or not there should be a physical education requirement for undergraduates. The administration has asked students for their views on the issue and has announced that its final decision will be based on how such a requirement would affect the overall educational mission of the college. Write a letter to the administration arguing whether or not there should be a physical education requirement for undergraduates at your college.

Writing Skills  72 items / 6 passages  Subscores:  Usage / mechanics  Rhetorical skills  Spelling, vocabulary, and rules of grammar not tested

Writing Skills Specs Content Category% of Test# of Items Usage/Mechanics.4432 Punctuation.08 6 Grammar.11 8 Sentence Structure.2518 Rhetorical Skills.5640 Strategy.2115 Organization.1410 Style.2115 Total1.0072

Writing Skills Sample By the fourth day I had to face the truth: my body was slowly changing to becoming dough. So I tied on my running shoes and loped out to the main road in search of a five-mile route. A. NO CHANGE B. become C. being D. OMIT the underlined portion

Mathematics

CAAP Administration  Select test modules that match the needs of your gen ed program  2-week test window  Convenient in-class administration  40 minutes per test  Pencil & paper  Proctored

CAAP Administration  Return materials to ACT for scoring & reporting  Objective tests - ACT proprietary scanning  Essay tests - two ACT-trained scorers  Three week turn-around time

CAAP Reports  Standard Reporting Package  SUNY Subscore Reports

Standard Reporting Package Includes:  Institutional Summary Report  Student Score Reports (2)  Student Roster Report  Student Certificates of Achievement  Subgroup Reports (first 3 included free)  Summary of Student Motivation  Data files available

SUNY Subscore Report  Subscores map to SUNY learning objectives (CT)  Groups students by  top 25%  middle 50%  bottom 25%  Compares local student performance with state or national norms

SUNY Criteria Compliance  Will it directly measure student learning? YES  Is the measure is reliable?  Will it measure the objective it is intended to measure?  Does the plan include externally referenced norms?  Is it capable of measuring value- added?  Will the data that are reported be representative? YES

CAAP Resources 

CAAP Resources  CAAP manuals  Planning & Forms  Tech Handbook  Reporting & Research Services  Motivating Students

Contact: David Chadima ACT Postsecondary Assessment Services 500 ACT Drive P.O. Box 168 Iowa City, Iowa Tel: (800) Fax: (319)