1. Multiple Measures Pilot College Convening - North
Multiple Measures Assessment Project: Math & English Models from Phase II
Multiple Measures Pilot College Convening - North
Cañada College
Terrence Willett
October 28, 2015

2. MMAP variables for predicting CC success
English
Math
Cumulative HS GPA
AP English class grade
Score on English CST
Grade in last English class
Cumulative HS GPA
Grade in Algebra I, Algebra II, Statistics, Trigonometry, Calculus
Taking a challenging CST
Score on math CST
Terrence:

3. Why Decision Trees? Provide easy-to follow and easy-to-code rule sets
Create discrete categories
No distributional or relational assumptions
Visual presentation of decisions/decision-making
They’re fun!

4. Increasing Homogeneity with each split
A B Z
Z A B
B Z A
A B A
B A B
AAA
BBB
ZZZ
Root Node
Branch
Internal Node
Explain basic terms like branch, leaf and node here.
Leaf Node

7. Statistics Tree – Direct Matriculants

8. Statistics Tree – NDM

9. Non-direct matriculants
English Decision Rules
Direct matriculants
Non-direct matriculants
Transfer-level
(criterion = 0.70 / 0.70)
HS 11 GPA >= 2.6
HS 12 GPA >= 2.6
One-level below
(criterion = 0.65 / 0.65)
HS 11 GPA >= 2.3
HS 12 GPA >= 2.2 &
HS 12 English course GPs >= 1.8
Two-levels below
(criterion = 0.60 / 0.55)
HS 11 GPA >= 2.0
HS 12 GPA >= 1.8 &
HS 12 English course GPs >= 1.1
HS 12 GPA >= 1.8 & CST >= 288
Three-levels below
(criterion = 0.55 / 0.55)
HS 11 GPA >=1.4
HS 12 GPA >=1.5 & CST>=268
HS 12 GPA >=1.7
Four-levels below
Everyone else

10. HS GPA>=3 AND CST>=340 HS GPA>=3 AND Calculus C or better
Level (CB21 code, criterion)
Hardcoded Pre-Requisite
Direct Matriculants
(Up through 11th grade, no CST)
Non-Direct
Matriculants
With CST
Without CST
Pre-Calculus
(criterion = 0.70)
Algebra II
(or better)
HS GPA>=3.4
HS GPA>=2.6 AND Calculus
HS GPA>=3.3
HS GPA>=3 AND CST>=340
HS GPA>=3 AND Calculus C or better
Statistics
(criterion=0.70)
Algebra I (or better)
HS_11_GPA >= 3.0
HS_11_GPA >= 2.3 AND Pre-Calculus with C or better
HS_12_GPA >= 3.0
General Education Math
(Transfer Level)
HS GPA >=3.3
HS GPA>=3.2 OR
HS GPA>=2.9 AND Stat C or better
College Algebra
Algebra II (or better)
HS GPA >=3.2
HS GPA >=2.9
AND
Pre-Calculus C or better

11. Higher level math rules
Level (CB21 code, criterion)
Hardcoded Pre-Requisite
Direct Matriculants
(Up through 11th grade, no CST)
Non-Direct
Matriculants
With CST
Without CST
Calculus I
(criterion = 0.70)
Pre-Calculus or Trigonometry (or better)
HS GPA>=3.6
HS GPA>=3.2 AND Pre-Calculus C or better
HS GPA>=3.1 AND Calculus C or better
HS GPA>=3.5
HS GPA>=3.1 AND Calculus

12. Basic Skills Math Decision Rules
Algebra II (CB21=A, criterion=0.65)
HS 11 GPA >=2.8
HS 12 GPA >=2.9
HS 12 GPA >=2.5
AND
Algebra II CST >= 302
HS 12 GPA >=2.5 AND
Pre-Calculus with C or better
Algebra I (CB21=B, criterion = 0.55)
HS 11 GPA>=2.4
HS 12 GPA>=2.5
HS 12 GPA>=2.0
Algebra I CST >= 302
HS 12 GPA>=2.3
Algebra II C or better
Pre-Algebra (CB21=C,
criterion 0.50)
HS 11 GPA>=2.0
HS 12 GPA >=2.1
HS 12 GPA >=1.6
Algebra 1 Cluster 1 (Number Properties, Operations, and Linear Equations) >= 8
Arithmetic (CB21=D)
Everyone else
Basic Skills Math Decision Rules

13. Correlations with College Grade: 11th grade Cumulative GPA vs
Correlations with College Grade: 11th grade Cumulative GPA vs. Accuplacer
English
11th Grade GPA
Accuplacer
Transfer
0.27
0.10
1 level below
0.24
0.12
2 levels below
0.25
3 levels below
0.18
4 levels below
0.21
0.07
Math
11th Grade GPA
Accuplacer
Transfer - STEM
0.24
0.19
Transfer – Stats
0.31
0.16
Transfer – LAM
0.26
0.09
1 level below
0.28
0.21
2 levels below
0.11
3 levels below
0.23
4 levels below
0.05
This is just a univariate comparison between the DV (CC Grade Points earned in first relevant CC class) and two potential IVs. It conveys the idea that GPA is a potent predictor – more predictive than Accuplacer assessment test scores. Accuplacer is overshadowed even more in a multivariate context. Note: the correlation between 11th grade cumulative GPA and 12th grade cumulative GPA is 0.95
ACCUPLACER scores did not add to the predictive power of the English models, though they did improve some math models, particularly sub-models predicting success in Pre-calculus and Calculus I at the community college.

17. How long is High School GPA good for?

18. MMAP Research Team Contacts
Mallory Newell
The RP Group
Terrence Willett
Craig Hayward
John Hetts
Educational Results Partnership
Ken Sorey
Daniel Lamoree

20. Predicting transfer-level grades
Test
Correlation with GPs
Corrected correlation
CTEP Grammar
.10
.16
CTEP Reading
.06
.09
CTEP Syntax
.08
.12
Math Assessment Test
.21
.29
11th Grade GPA - English
.38
.39
11th Grade GPA – Math
.37

21. Cost Old model Acceleration MIH + Acceleration Semesters 3 2 1 Units10 6 4
Cost to get all to Engl 1A
$1,677,552
$1,084,682
$208,851
Placement at Transfer level
18%
29%
57%