Positioning Students with the Multiple Measures Assessment Project Holly Kresch, Math Faculty Diablo Valley College Bridget Herrin, Research Analyst at MiraCosta College Tim Johnston, Dean of Enrollment Services at Shasta College Jenna Barry Highfield, Research Analyst at Shasta College

Overview of the Multiple Measures Assessment Project Holly Kresch, Math Faculty Diablo Valley College

Multiple Measures Assessment Project (MMAP) [adapted from the following presentation] Multiple Measures Assessment Project (MMAP) November 2, 2016 Mallory Newell The RP Group Director, Research and Planning, De Anza College Ken Sorey Educational Results Partnership http://www.rpgroup.org/projects/multiple-measures-assessment-project

MMAP Project Overview Collaboration Model Development Engagement CAI CCCCO Cal-PASS+ RP Group 60 CCCs Model Development English Math ESL Reading Non-cognitive Variables Self-reported transcript data Engagement Local replication Webinars Professional development Support Pilot results inform statewide implementation bit.ly/MMAP2015

Sequence Completion - Math Basic Skills Cohort Progress Tracker: http://datamart.cccco.edu/Outcomes/BasicSkills_Cohort_Tracker.aspx

Growing body of evidence Weak relationship between assessment tests and college course outcomes: bit.ly/CCRCAssessment Incredible variability in cut scores; CCCs often use HIGHER cutscores than 4-year institutions: bit.ly/NAGB2012 Underestimates students of color, women, first generation college students, low SES: bit.ly/DefiningPromise Long thread of research in the CCCs Willett, Hayward, & Dahlstrom, 2008 http://bit.ly/WIllett2008 Hetts, Fuenmayor, & Rothstein, 2012 http://www.lbcc.edu/PromisePathways Willett & Karanjeff, 2014 http://bit.ly/RPSTEPS

Why Multiple Measures? Tests have been under-placing students provides a more complete picture of student ability provides a way to increase the accuracy of placement, particularly reducing underplacement http://bit.ly/CCRCPlacementAccuracy are required by law (Title V) supported by statewide senate

Potential Statewide Impact on Enrollment

Potential Impact on Enrollment and Equity Math Transfer-Level

Projected Impact on Success Rates (completion of course with C or better)

Common Concerns about MMAP Students placed via MMs will not be successful Our courses will have lower pass rates Our test is different Students would be better off in remedial coursework We are only looking at GPA Students will only get a “C” in transfer-level work Students who get a “C” in transfer-level won’t be able to transfer High school GPA is only good for recent graduates    

The Models

Data Set for Models CCC students enrolled in an English, Math, Reading or ESL class with matching high school data in CalPASS ~1 M cases for Math & English; ~200k for Reading & ESL Bulk of first CCC enrollments from 2008 through 2014 Rules were developed with the subset of students who had four years of high school data (about 25% of total sample) Used rpart to create the trees uses a machine learning algorithm where the order of entry of the variables does not matter. All predictors are considered in the algorithm, the predictor with the greatest gain to the model is selected for the first branch of the tree, continues to split until the splits no longer improve the model.

Variables Explored in the MMAP Models High School Unweighted Cumulative GPA Grades in high school courses CST scores Advanced Placement course taking Taking higher level courses (math) Delay between HS and CCC (math) HS English types (expository, remedial, ESL) HS Math level (Elem Algebra, Integrated Algebra, Pre-Calculus)

Rules & Implementation The MMAP Decision Matrix Diablo Valley College Placement Tool

Some lessons learned (from pilot colleges) MMAP rules are performing as expected Implementation of MM rules is nuanced, needs to involve members from across the college Communication to students should be clear and consistent – 1 placement rather than 2 Student support should be embedded Outreach/communication with local high schools Should include a robust research agenda

Upcoming MMAP Events Webinar 3rd Annual -Pilot College Convenings Developing a Research/Evaluation Plan - Thursday, November 3, 2-3 p.m. This webinar will focus on considerations when developing a research and evaluation plan to assess the impact of MMAP on your campus. 3rd Annual -Pilot College Convenings Northern California De Anza College Friday, December 2 - 10am - 2pm 21250 Stevens Creek Blvd Cupertino, CA 95014 Campus Center, Conference Room B RSVP Here Southern California Irvine Valley College Friday, December 9 - 11am - 3pm 5500 Irvine Center Drive Irvine, CA 92618 Room B-209 (Park in Lot 10) RSVP Here

Research Team Contacts To join our mailing list, email: newellmallory@deanza.edu Loris Fagioli The RP Group lfagioli@ivc.edu Mallory Newell newellmallory@deanza.edu Terrence Willett twillett@rpgroup.org Craig Hayward chayward@rpgroup.org John Hetts Educational Results Partnership jhetts@edresults.org Ken Sorey ken@edresults.org Daniel Lamoree dlamoree@edresults.org   Peter Bahr University of Michigan prbahr@umich.edu ‎