1. Setting a college-readiness cut score for the CRMT CRMT Working Group August 19, 2009 1:30-3:30  Background  Pilot test summary  Current MPT-I placement cut scores  Contrasting groups  Group discussion

2. Original Legislation Common college readiness test in math for WA public higher education with common performance standard Fall 2009: high schools must make CRMT available to high school students (subject to funding) CRMT Background http://apps.leg.wa.gov/documents/billdocs/2007- 08/Pdf/Bills/House%20Passed%20Legislature/1906-S2.PL.pdf System Agreements around Using CRMT Provosts’ agreement http://www.washington.edu/oea/services/testing_center/crmt/ProvostsAgreement.pdf CTC agreement

3. Purpose 2008-2009 CRMT Pilot Study 1 To relate MPT test scores to subsequent course grades to assist in setting CRMT college readiness cut score Research Questions  How does student performance on the MPT-G compare to performance on the MPT-I?  How well do student test scores predict student grades? 1 McGhee D., N. Lowell, J. Gillmore, and J. Peterson (2009) 2009 General Mathematics Placement Test (MPT-G) Pilot, OEA Report 09-03 [http://www.washington.edu/oea/pdfs/reports/OEAReport0903.pdf]

4. Method 2008-2009 CRMT Pilot Study  Students at four-year universities took the MPT-G (n = 557) or the MPT-I (n = 692) for placement into courses  Students at other campuses were offered the opportunity to use test scores for placement at four- year universities  Tests administered between October 2008 and June 2009 Edmonds CC (n = 89) Spokane Falls CC (n = 137) TESC (n = 34) 21 high schools (n = 2220)

5. Results 2008-2009 CRMT Pilot Study Test Reliability Both the MPT-G and MPT-I showed excellent internal consistency (  =.84 and.85, respectively) Test Difficulty The MPT-G was more difficult than the MPT-I (Mns = 18.4 and 20.4, respectively; percentage equivalents = 52.6% and 58.3%)

6. 2008-2009 CRMT Pilot Study Test Difficulty (continued) The difference in test difficulty was observed for all three educational sectors, but was most pronounced at four-year institutions Average total score by test type and institution type Results

7. 2008-2009 CRMT Pilot Study Test Difficulty (continued) Students enrolled or enrolling in college level courses scored 6 points higher than students taking courses below college level Average total score by test type and course level Results

8. 2008-2009 CRMT Pilot Study Course Grades MPT-G and MPT-I scores were significantly correlated with subsequent math course grades both at high schools and four-year schools (r .4) Results

9. 2008-2009 CRMT Pilot Study Course Grades (continued) Students who passed their math course (grade  2.0) tended to have scored significantly higher on the MPT than did students who did not pass Results Average total score by course level and grade (four-year schools)

10. 2008-2009 CRMT Pilot Study Course Grades (continued) The probability of passing a course generally increased with total test score Results Observed rates of success in non-precalculus/calculus college level courses as a function of total test score (four-year schools)

11. Conclusions 2008-2009 CRMT Pilot Study  Both tests show good discriminant validity in the superior performance of students taking college level math courses over those in pre-college level courses  Both tests show good predictive validity by the significant correlations between total test scores and mathematics course grades  Both the MPT-G and MPT-I show excellent reliability

12. Current Intermediate Math Placement Test (MPT-I) Placement Cutoffs (July 2009) Placement Cut ScoreCentralEasternUW-SWash StWestern 30206 202 Calculus for Arch. Business Calculus 25118Accel precalc 23211 200 105 Elem teachers Finite math Precalculus 107Elem functions 21120Precalculus 20251 201 Elem teachers Finite math 381 156 114 K-12 teachers Alg for business Precalculus 19154Precalculus 2 18153Precalculus 1 17115Liberal arts111 107 Business Liberal arts 240Stats intro 15114Alg concepts212 210 205 Stats intro Math intro Stat’l thinking 112 107 101 Alg functions Math reasoning Elem teachers 12104Intermed alg 4103Alg intro 0100.1Algebra100Comb 103/10498Intermed alg101Elem alg review106 99 Quant reasoning Alg intro

13. Definition Contrasting Groups A family of methods to use actual performance data of known groups to set cut scores Process 1. Define two groups (“Masters” and “Non-Masters”) 2. Analyze data to find score(s) that best differentiate(s) between groups a. Select the point of intersection of two frequency distributions (visual inspection) b. Find the midpoint between average scores (computation) c. Find score at which probability of group membership is.50 (using logistic regression) Options

14. Defining the Groups Contrasting Groups You want the groups to be meaningful, defensible, and clearly distinct Not distinct enough High school Algebra 1 students vs. Post-secondary Calculus students (All) high school students vs. (All) post-secondary students More extreme than necessary

15. Possible Groups Contrasting Groups 1.any non-college, < 2.0 vs. any college,  2.0 2.any non-college, < 2.0 vs. post-secondary, college,  2.0 3. post-secondary, non-college vs. post-secondary, college 4. post-secondary, non-college, < 2.0 vs. post-secondary, college,  2.0 5. high school, non-collegevs. post-secondary, college 6. high school, non-college, < 2.0 vs. post-secondary, college,  2.0

16. Comparison 1 Any non- college class, <2.0 Any college- level class, 2.0+ MID- POINT MPT-G Median132318.0 Mean13.5622.7118.1 SD4.846.04 Valid N162370 MPT-I Median15.52520.3 Mean15.9224.4920.2 SD5.175.8 Valid N182528

17. Comparison 2 Any non- college class, <2.0 Postsec, college-level class, 2.0+ MID- POINT MPT-G Median132117.0 Mean13.5621.6617.6 SD4.845.32 Valid N162131 MPT-I Median15.52520.3 Mean15.9223.8819.9 SD5.175.31 Valid N182199

18. Comparison 3 Postsec, non- college class Postsec, college-level class MID- POINT MPT-G Median122116.5 Mean12.2820.9816.6 SD3.95.29 Valid N60190 MPT-I Median142318.5 Mean14.5122.8918.7 SD4.925.29 Valid N68293

19. Comparison 4 Postsec, non- college class, <2.0 Postsec, college-level class, 2.0+ MID- POINT MPT-G Median122116.5 Mean11.721.6616.7 SD3.865.32 Valid N27131 MPT-I Median142519.5 Mean13.8223.8818.9 SD4.185.31 Valid N33199

20. Comparison 5 HS, non- college class Postsec, college-level class MID- POINT MPT-G Median162118.5 Mean16.4220.9818.7 SD5.895.29 Valid N530190 MPT-I Median182320.5 Mean17.9622.8920.4 SD5.435.29 Valid N636293

21. Comparison 6 HS, non- college class, <2.0 Postsec, college-level class, 2.0+ MID- POINT MPT-G Median132117.0 Mean13.9321.6617.8 SD4.945.32 Valid N135131 MPT-I Median162520.5 Mean16.3823.8820.1 SD5.275.31 Valid N149199