Compass Users Webinar: Positioning Students with the Multiple Measures Assessment Project Tim Johnston, Dean of Enrollment Services Jenna Barry Highfield, Research Analyst November 3, 2016

MMAP at Shasta College Excellent Support from the RP Group http://rpgroup.org/projects/multiple-measures-assessment-project Campus Dialog English Department Counseling Department Assessment Center Shared Governance Student Success Committee Instructional Council Student Services Council

English Department Dialog Local MMAP Pilot in Spring 2015 Tested MMAP with locally modified decision rules Transfer level (English 1A): Cumulative H.S GPA greater than 2.70; AND “B” or better in the last, non- remedial English course; OR “C+” or higher in AP English course; OR CST / EAP/ Smarter Balance “Ready”; OR Advanced Placement (English Language OR English Literature) score 3 or higher; OR Schedule an Assessment

English Department Dialog One Level Below Transfer (English 190): Cumulative H.S GPA greater than 2.0; OR CST / EAP/ Smarter Balance “Conditional”; OR Schedule an Assessment  Two Levels Below Transfer (English 280 or below): Counselor recommendation using multiple measures; OR Advisory: All Basic English students should be placed in English 190 or lower. If a student’s most recent English class was Modified or RSP, they should be strongly encouraged to take the English assessment.

Shasta College MMAP Pilot Counselors visited students at their high schools in Spring 2015 Conducted multiple measures assessments (evaluation of high school transcripts based on guidelines previously discussed), resulting in a math and/or English course placement for students 471 students assessed with multiple measures in Spring 2015, comprising the “Multiple Measures Cohort Group” Comparison groups: “Traditionally assessed” – students primarily assessed with placement tests “All Others” – students not in the multiple measures cohort group and not identified as primarily placed by placement tests (e.g., assessed by professional judgment, self- report, etc.)

Pilot Study: Placement into Courses Only students who received math and/or English course placements between March and August 31, 2015 were included in the placement data set Students who were placed during this time and then subsequently enrolled in a math or English course in the Fall 2015 semester made up the enrollment data set If a student was given more than one course placement from multiple assessments, the higher placement was retained (disjunctive placement approach; RP Group, 2015) Placement methods were captured by the following categories: High School Transcripts Placement Tests All Other Methods (including professional judgment)

English Placement Results: Course Level and Placement Method Students placed by High School Transcripts (80.53%) were significantly more likely to be placed in transfer level English courses than were students placed by Placement Tests (55.17%) and All Other Methods (67.7%), Chi-Square (df = 6) = 145.99, p < .001, Cramer’s V = .17.

English Course Enrollment by Course Level and Student Group Students from the Multiple Measures Cohort group (86.46%) were significantly more likely to enroll in transfer level courses than students from the Traditionally Assessed (56.56%) and All Others (61.66%) group, Chi-Square (df = 8) = 70.47, p < .001, Cramer’s V = .25.

Success & Retention in English Courses No significant difference in success rates among the three student groups Type of Success Rate Multiple Measures Cohort Traditionally Assessed All Others Total Average (all courses) 66.81% 64.44% 64.71% 65.04% Transfer level 67.17% 67.51% 70.67% 68.66% Below transfer 64.52% 60.44% 55.11% 58.35% Multiple Measures Cohort (93.01%) significantly more likely to be retained in English courses than Traditionally Assessed (84.49%) group, Chi-Square (df = 2) = 11.54, p = .003, Cramer’s V = .10. Neither group differed from All Others group (89.54%) Type of Retention Rate Multiple Measures Cohort Traditionally Assessed All Others Total Average (all courses) 93.01% 84.49% 89.54% 88.35% Transfer level 92.93% 83.97% 91.52% 89.42% Below transfer 93.55% 85.16% 86.36% 86.38%

Recommendations and Next Steps The Local MMAP Pilot English study serves as validation for use of multiple measures in successfully placing students in English courses Upon the sunset of Compass on Nov. 30, 2016, MMAP decision rules will be used to place students into English courses. Questions? Jenna Barry Highfield at jhighfield@shastacollege.edu Tim Johnston at tjohnston@shastacollege.edu RP Group Decision Rules: http://rpgroup.org/system/files/StateWideDecisionRules_5.18.16.pdf