VSA & Assessment of Learning Outcomes Sally Frazee, Temple University Criss Gilbert, University of Minnesota-Twin Cities Susan Morgan – Appalachian State University Sally Frazee, Temple University Criss Gilbert, University of Minnesota-Twin Cities Susan Morgan – Appalachian State University
V S A ? Voluntary System of Accountability 2007 AASCU Association of American State Colleges & Universities “College Portrait” of Undergraduate Education APLU Association of Public & Land-grant Universities
College Portrait Source of basic, comparable information about public colleges and institutions presented in a common format. No rankings, no spin…just the facts! Designed to be a trustworthy source of reliable data for prospective students and their families.
College Portrait Student Characteristics Undergraduate Admissions Costs of Attendance & Financial Aid Undergraduate Success & Progress Rate Campus Life Degrees, Areas of Study & Future Plans Student Experiences Student Learning Outcomes
Measure improvement in student abilities in three broad areas General cognitive skills that cut across disciplines: critical thinking, analytical reasoning, and written communication VSA sponsored review of sixteen Outcomes Assessment instruments…3 were selected - CAAP- CLA- Proficiency Profile Pilot project – expectation that best practices will emerge as data are collected Measure improvement in student abilities in three broad areas General cognitive skills that cut across disciplines: critical thinking, analytical reasoning, and written communication VSA sponsored review of sixteen Outcomes Assessment instruments…3 were selected - CAAP- CLA- Proficiency Profile Pilot project – expectation that best practices will emerge as data are collected Student Learning Outcomes
How Does This Work? Institutional learning outcomes as students enter and exit Cross-sectional methodology Random sample of freshmen & seniors Assess within 4 years, & then administer every 3 years Assess within 4 years, & then administer every 3 years University High Stakes _____ Student Low Stakes University High Stakes _____ Student Low Stakes
Outcomes Measures Approved by VSA Sampling Costs Administration Student recruitment Student motivation Use of data Sampling Costs Administration Student recruitment Student motivation Use of data CAAP CLA Proficiency Profile
Proficiency Profile (formerly MAPP) ETS Integrated test of general education skills Paper-and-pencil or online delivery Proctored and unproctored administrations
Standard Form 108 items 2 hours Standard Form 108 items 2 hours Abbreviated Form 36 items 40 minutes Abbreviated Form 36 items 40 minutes No minimum of test takers to produce aggregate reports Score Reports include a total score and subscores and proficiency classifications for individual students total scores, norm-referenced subscores and criterion-referenced scores at the group level an alphabetical roster with 17 individual student scores No minimum of test takers to produce aggregate reports Score Reports include a total score and subscores and proficiency classifications for individual students total scores, norm-referenced subscores and criterion-referenced scores at the group level an alphabetical roster with 17 individual student scores Minimum of 50 test takers to produce aggregate reports Score Reports include a total score, but no subscores, for individual students total scores, norm-referenced (scaled) subscores and criterion-referenced scores (proficiency classifications) at the group level. Minimum of 50 test takers to produce aggregate reports Score Reports include a total score, but no subscores, for individual students total scores, norm-referenced (scaled) subscores and criterion-referenced scores (proficiency classifications) at the group level.
Outcomes Measure Journey at Temple Signed on for VSA in CAAP Pilot & 2008 MAPP Pilot ▫Opportunity to try out administration ▫Opportunity to try out recruitment ▫Honoraria to fund faculty workshop 2009: Faculty Workshop: Looking at Learning 2010: Pilot Administration of the Proficiency Profile, with no VSA obligation to publish results
Faculty Workshop Multiple Ways of Looking at Assessment (Periodic Program Review) VSA – How and Why (Vice Provost, MARC) Using Published Tests (Chair, Educational Psychology) A Look at Professional Exams (MARC, Institutional Research)
Fall 2010 Administration GOAL: Abbreviated version to 200 freshmen within 6 weeks of the start of the semester PLAN: ▫ to all 4,300 incoming freshmen on 9/16 ▫Lots of convenient test sessions ▫$20.14 in Diamond Dollars ▫Spread the word – in-class & dorm presentations posters and plasma screens newspaper articles information desk & giveaways Budget will be $15,000-$20,000 per year
VSA & Assessment of Learning Outcomes Questions?