VSA & Assessment of Learning Outcomes Sally Frazee, Temple University Criss Gilbert, University of Minnesota-Twin Cities Susan Morgan – Appalachian State University Sally Frazee, Temple University Criss Gilbert, University of Minnesota-Twin Cities Susan Morgan – Appalachian State University

V S A ? Voluntary System of Accountability 2007 AASCU Association of American State Colleges & Universities “College Portrait” of Undergraduate Education APLU Association of Public & Land-grant Universities

College Portrait Source of basic, comparable information about public colleges and institutions presented in a common format. No rankings, no spin…just the facts! Designed to be a trustworthy source of reliable data for prospective students and their families.

College Portrait Student Characteristics Undergraduate Admissions Costs of Attendance & Financial Aid Undergraduate Success & Progress Rate Campus Life Degrees, Areas of Study & Future Plans Student Experiences Student Learning Outcomes

Measure improvement in student abilities in three broad areas General cognitive skills that cut across disciplines: critical thinking, analytical reasoning, and written communication VSA sponsored review of sixteen Outcomes Assessment instruments…3 were selected - CAAP- CLA- Proficiency Profile Pilot project – expectation that best practices will emerge as data are collected Measure improvement in student abilities in three broad areas General cognitive skills that cut across disciplines: critical thinking, analytical reasoning, and written communication VSA sponsored review of sixteen Outcomes Assessment instruments…3 were selected - CAAP- CLA- Proficiency Profile Pilot project – expectation that best practices will emerge as data are collected Student Learning Outcomes

How Does This Work? Institutional learning outcomes as students enter and exit Cross-sectional methodology Random sample of freshmen & seniors Assess within 4 years, & then administer every 3 years Assess within 4 years, & then administer every 3 years University High Stakes _____ Student Low Stakes University High Stakes _____ Student Low Stakes

Outcomes Measures Approved by VSA  Sampling  Costs  Administration  Student recruitment  Student motivation  Use of data  Sampling  Costs  Administration  Student recruitment  Student motivation  Use of data CAAP CLA Proficiency Profile

Proficiency Profile (formerly MAPP) ETS Integrated test of general education skills Paper-and-pencil or online delivery Proctored and unproctored administrations

Standard Form 108 items 2 hours Standard Form 108 items 2 hours Abbreviated Form 36 items 40 minutes Abbreviated Form 36 items 40 minutes No minimum of test takers to produce aggregate reports Score Reports include a total score and subscores and proficiency classifications for individual students total scores, norm-referenced subscores and criterion-referenced scores at the group level an alphabetical roster with 17 individual student scores No minimum of test takers to produce aggregate reports Score Reports include a total score and subscores and proficiency classifications for individual students total scores, norm-referenced subscores and criterion-referenced scores at the group level an alphabetical roster with 17 individual student scores Minimum of 50 test takers to produce aggregate reports Score Reports include a total score, but no subscores, for individual students total scores, norm-referenced (scaled) subscores and criterion-referenced scores (proficiency classifications) at the group level. Minimum of 50 test takers to produce aggregate reports Score Reports include a total score, but no subscores, for individual students total scores, norm-referenced (scaled) subscores and criterion-referenced scores (proficiency classifications) at the group level.

Outcomes Measure Journey at Temple Signed on for VSA in CAAP Pilot & 2008 MAPP Pilot ▫Opportunity to try out administration ▫Opportunity to try out recruitment ▫Honoraria to fund faculty workshop 2009: Faculty Workshop: Looking at Learning 2010: Pilot Administration of the Proficiency Profile, with no VSA obligation to publish results

Faculty Workshop Multiple Ways of Looking at Assessment (Periodic Program Review) VSA – How and Why (Vice Provost, MARC) Using Published Tests (Chair, Educational Psychology) A Look at Professional Exams (MARC, Institutional Research)

Fall 2010 Administration GOAL: Abbreviated version to 200 freshmen within 6 weeks of the start of the semester PLAN: ▫ to all 4,300 incoming freshmen on 9/16 ▫Lots of convenient test sessions ▫$20.14 in Diamond Dollars ▫Spread the word –  in-class & dorm presentations  posters and plasma screens  newspaper articles  information desk & giveaways Budget will be $15,000-$20,000 per year

VSA & Assessment of Learning Outcomes Questions?