1. Positioning Students with the Multiple Measures Assessment Project
Holly Kresch, Math Faculty Diablo Valley College
Bridget Herrin, Research Analyst at MiraCosta College
Tim Johnston, Dean of Enrollment Services at Shasta College
Jenna Barry Highfield, Research Analyst at Shasta College

2. Overview of the Multiple Measures Assessment Project
Holly Kresch, Math Faculty Diablo Valley College

3. Multiple Measures Assessment Project (MMAP)
[adapted from the following presentation]
Multiple Measures Assessment Project (MMAP)
November 2, 2016
Mallory Newell
The RP Group
Director, Research and Planning, De Anza College
Ken Sorey
Educational Results Partnership

4. MMAP Project Overview Collaboration Model Development Engagement
CAI
CCCCO
Cal-PASS+
RP Group
60 CCCs
Model Development
English
Math
ESL
Reading
Non-cognitive Variables
Self-reported transcript data
Engagement
Local replication
Webinars
Professional development
Support
Pilot results inform statewide implementation
bit.ly/MMAP2015

6. Sequence Completion - Math
Basic Skills Cohort Progress Tracker:

7. Growing body of evidence
Weak relationship between assessment tests and college course outcomes: bit.ly/CCRCAssessment
Incredible variability in cut scores; CCCs often use HIGHER cutscores than 4-year institutions: bit.ly/NAGB2012
Underestimates students of color, women, first generation college students, low SES: bit.ly/DefiningPromise
Long thread of research in the CCCs
Willett, Hayward, & Dahlstrom,
Hetts, Fuenmayor, & Rothstein,
Willett & Karanjeff,

8. Why Multiple Measures? Tests have been under-placing students
provides a more complete picture of student ability
provides a way to increase the accuracy of placement, particularly reducing underplacement
are required by law (Title V)
supported by statewide senate

9. Potential Statewide Impact on Enrollment

10. Potential Impact on Enrollment and Equity Math Transfer-Level

11. Projected Impact on Success Rates
(completion of course with C or better)

12. Common Concerns about MMAP
Students placed via MMs will not be successful
Our courses will have lower pass rates
Our test is different
Students would be better off in remedial coursework
We are only looking at GPA
Students will only get a “C” in transfer-level work
Students who get a “C” in transfer-level won’t be able to transfer
High school GPA is only good for recent graduates

13. The Models

14. Data Set for Models
CCC students enrolled in an English, Math, Reading or ESL class with matching high school data in CalPASS
~1 M cases for Math & English; ~200k for Reading & ESL
Bulk of first CCC enrollments from 2008 through 2014
Rules were developed with the subset of students who had four years of high school data (about 25% of total sample)
Used rpart to create the trees
uses a machine learning algorithm where the order of entry of the variables does not matter. All predictors are considered in the algorithm, the predictor with the greatest gain to the model is selected for the first branch of the tree, continues to split until the splits no longer improve the model.

15. Variables Explored in the MMAP Models
High School Unweighted Cumulative GPA
Grades in high school courses
CST scores
Advanced Placement course taking
Taking higher level courses (math)
Delay between HS and CCC (math)
HS English types (expository, remedial, ESL)
HS Math level (Elem Algebra, Integrated Algebra, Pre-Calculus)

16. Rules & Implementation
The MMAP Decision Matrix
Diablo Valley College Placement Tool

17. Some lessons learned (from pilot colleges)
MMAP rules are performing as expected
Implementation of MM rules is nuanced, needs to involve members from across the college
Communication to students should be clear and consistent – 1 placement rather than 2
Student support should be embedded
Outreach/communication with local high schools
Should include a robust research agenda

18. Upcoming MMAP Events Webinar 3rd Annual -Pilot College Convenings
Developing a Research/Evaluation Plan - Thursday, November 3, 2-3 p.m. This webinar will focus on considerations when developing a research and evaluation plan to assess the impact of MMAP on your campus.
3rd Annual -Pilot College Convenings
Northern California De Anza College Friday, December 2 - 10am - 2pm Stevens Creek Blvd Cupertino, CA Campus Center, Conference Room B RSVP Here
Southern California Irvine Valley College Friday, December 9 - 11am - 3pm Irvine Center Drive Irvine, CA Room B-209 (Park in Lot 10) RSVP Here

19. Research Team Contacts
To join our mailing list,
Loris Fagioli
The RP Group
Mallory Newell
Terrence Willett
Craig Hayward
John Hetts
Educational Results Partnership
Ken Sorey
Daniel Lamoree
Peter Bahr University of Michigan ‎