Multiple Measures Assessment Project Welcome Webinar Spring 2014

Multiple Measures and Common Assessment Overview of analysis Student Transcript Enhanced Placement Study (STEPS) findings leading to Multiple Measures Project (MMAP) Working with K-12 partners Compass or Accuplacer data Role of the pilot colleges Projected timeline Questions and Answers Agenda

Multiple Measures and Common Assessment Initiative Role of the Common Assessment Advisory Group

Overview of Analysis Examine the value of using transcripts as part of the assessment process Create predictive models using intersegmental data to study students who had already taken community college courses Use models to analyze how well transcript data predicts the first English & math courses students take and how well they do in them Recruit colleges to do local analyses to generate campus-specific insights and trigger CCC/K-12 conversations 4 Harnessing the Power of Alternative Assessment to Advance the Completion Agenda

Student Transcript Enhanced Placement Study (STEPS) – Statewide Findings

In English tests predict tests, grades and courses matter but vary by college 6

7 Cox & Snell pseudo R-square ~ 0.35 Harnessing the Power of Alternative Assessment to Advance the Completion Agenda

In math tests predict tests, but high school level is also important 8

9 Cox & Snell pseudo R-square ~ 0.50 Harnessing the Power of Alternative Assessment to Advance the Completion Agenda

In English, grades predict grades 10

11 Cox & Snell pseudo R-square ~ 0.20 Harnessing the Power of Alternative Assessment to Advance the Completion Agenda

In math, success predictors vary by college 12

13 Cox & Snell pseudo R-square ~ 0.20 Harnessing the Power of Alternative Assessment to Advance the Completion Agenda

What Does This Mean? Using high school transcript data could help inform alignment/articulation and refine assessment policies The relative weight of variables might be influenced by local factors, due to considerations such as articulation and curriculum Barriers to implementation must be addressed such as processing transcripts 14 Harnessing the Power of Alternative Assessment to Advance the Completion Agenda

What Happens Next? Updating the statewide analysis Encouraging more colleges to replicate the study so they understand the value of transcript data for their own students Starting conversations about how the study relates to other aspects of developmental education reform—such as alignment, course offerings, and curriculum Helping to develop a tool that enables colleges to access high school data to inform multiple measures assessment 15 Harnessing the Power of Alternative Assessment to Advance the Completion Agenda

Working with K-12 partners to upload data Key aspect of analysis using high school transcript data is getting local K-12 data into Cal-PASS Plus Cal-PASS Plus can assist in outreach, though initially coming from a college helps Cal-PASS Plus and RP Group can help provide materials and templates Entire process to submit data is streamlined – drag and drop files they already create for the CA Dept. of Education

Compass or Accuplacer data -Does your college use Accuplacer or Compass for student placements? -If not, we ask that you submit you college’s data to Cal-PASS Plus to include in the analysis.

Projected timeline Jan – Mar: Build data warehouse for application to multiple measures analysis – collect various data sources May – July: Complete research on system wide multiple measures models and algorithms July – Sept: Develop and deploy online analytic tools and decision tree models Oct – Dec: Provide PD and direct support to pilot colleges to test multiple measures tools and application; develop and test local user interface to re-identify data for placement decisions Dec – Ongoing: Provide cohort tracking data to pilot colleges on course outcomes; integrate multiple measures project with Common Assessment Initiative

Role of the pilot colleges Test interactive online placement tools Provide feedback on user experience Compare student placements using MMAP analysis to your college’s current system – How are students placing when comparing the two? – What new conversations have occurred about improving the placement process? Work with Cal-PASS Plus to track cohort outcomes after applying multiple measures through the project

Questions?

Contacts: Mallory Newell The RP Group Ken Sorey, Cal-PASS Plus Amanda Avallone Cal-PASS Plus STEPS Report and Summary MMAP Project Site ple-measures-assessment-project Cal-PASS Plus