Student Success Scorecard Contra Costa Community College District Board of Trustees Study Session on the Student Success Scorecard November 2018

Board goal and state mandate DISTRICT GOAL 1 ENHANCE STUDENT LEARNING AND SUCCESS: Create opportunities for thoughtful reflection and organizational learning that use meaningful quantitative and qualitative data, dialogue with diverse member of the community, student feedback, and other information in order to improve student outcomes. Objective 1.1 Conduct activities that improve student performance in areas included in the Student Success Scorecard over time.

Some background and context State legislation is increasingly emphasizing completion & transfer Scorecard metrics are still evolving but becoming an important CCC accountability yardstick Programs and new legislation linked to Scorecard metrics College Promise Assessment reform Strong Workforce Program Guided Pathways Student Equity AB 19 AB 504 SB 539 AB 705 AB 1018

Goals in the Vision for Success increase the number of degrees, credentials, and certificates (20% in 5 years) increase the number of transfers to a UC or CSU (35% in five years) decrease the average number of units accumulated by students earning associate’s degrees (by 8 units in 5 years) increase the percent of CTE students employed in their field of study (by 9% in five years) Reduce equity gaps across all underrepresented student groups (by 40% in 5 years)

Some info about the Scorecard itself Each Scorecard contains nearly 400 data points per college The scorecard is a cohort tacking tool, not a series of static snapshots Profiles community colleges not community college districts Definitions are highly nuanced

What the Scorecard actually looks like… Source: https://scorecard.cccco.edu/scorecard.aspx 2018. Updated 10/12/18, JD. Source: https://scorecard.cccco.edu/scorecard.aspx

Our focus for this study session Two student populations and four distinct pathways College prepared students pursuing degree/transfer Underprepared students pursuing degree/transfer Credential seeking CTE students Non-credential seeking CTE students Pre-collegiate course sequences in math and English The link between remediation and student equity

Career-oriented Students Completion-oriented Students College level coursework 4-Years % completing Progress milestone # Prepared for college Pre-collegiate coursework College level coursework # Underprepared for college % completing English Progress milestone Math Pace milestone Career-oriented Students % completing # Credential-seeking CTE pathway course taking % wage gains achieved # Non credential-seeking “Skills Builder” course taking

Not captured in Scorecard How much of our student population is captured by the Scorecard? Ball park percentages of Scorecard populations as percent of first-time students Prepared Completion Seeking Not captured in Scorecard 10% 40% Under-prepared Completion Seeking 30% 10% 10% CTE Non credential Seeking CTE Credential Seeking

Completion-oriented students Definitions Five year trend Baseline comparisons

% Students completing (Completion Rate) The art of calculating completion More difficult to count Easy to Quantify % Students completing (Completion Rate) divided by = # Students pursuing completion # Students completing

# Students pursuing completion The Scorecard looks exclusively at course taking behavior Definition: The number of first-time students with a minimum of 6 units earned within six years who also attempted any Math or English in the first three years … … who then achieved any of the following outcomes within six years of entry: • Earned AA/AS or credit Certificate • Transfer to four-year institution • Achieved “Transfer Prepared” Status # Students pursuing completion Who get’s counted as a completion-oriented student? In the past a student’s self-identified educational goal proved to be an unreliable indicator of actual intent. # Students completing

So that’s who we count, but how long do we count? Each cohort is given six years to complete. We add up all those completing each year to get the total number completing See notes below. 3% 16% 27% 34% Greg, I don’t know the source of this information. How where these figures derived? All I did was update the years on the axis. 15% 5% 2011/12 Starting Cohort 2011/12 2012/13 2013/14 2014/15 2016/17 Total Number that Completed in Six Years ----------- Number completing each year ---------

Five year completion rates (prepared and underprepared combined) 4CD California State Average Source: 2018 Scorecard. COMPLETION RATE (STUDENT PROGRESS AND ATTAINMENT RATE) Definition: The percentage of first-time students with minimum of 6 units earned who attempted any Math or English in the first three years and achieved any of the following outcomes within six years of entry: • Earned AA/AS or credit Certificate (Chancellor’s Office approved) • Transfer to four-year institution (students shown to have enrolled at any four-year institution of higher education after enrolling at a CCC) • Achieved “Transfer Prepared” (student successfully completed 60 UC/CSU transferable units with a GPA >= 2.0) Outcome (Achieve the following outcome): • A student must successfully transfer to the following segments during an academic year. o University of California o California State University o In-State Private (ISP), Non-Profit and For-Profit o Out-of-State (OOS), Public, Non-Profit and For-Profit Definition: Completion (SPAR) – The percentage of degree and/or transfer seeking first-time students tracked for six years to determine who succeeded in completing a degree, certificate or transfer related outcome.

Completion rates for California’s 114 community colleges We can also look at performance within state economic region 2018 Scorecard Bay Area & Central Coast are highest performing regions

Cluster analysis produced 7 distinct peer groups The state also provide peer group comparisons based on college service area statistics and student demographics Cluster analysis produced 7 distinct peer groups 2018 Scorecard Each of our colleges are among the highest performers within their respective peer group

A quick pulse check Any questions ?

Let’s return to completion-oriented students Two student populations: Prepared for college-level work Underprepared for college-level work

# of 4CD students pursuing completion Underprepared Students Two pathways for completion-oriented students Prepared Students Complete 24% # of 4CD students pursuing completion N = 1,113 N = 852 Source: 2018 Scorecard, 2011/12 to 2016/17 Cohort. 76% Underprepared Students Complete N = 4,685 N = 3,572 N = 1,764 Note: the State Chancellor’s Office defines unprepared as any completion oriented student whose first course in math or English was below transfer level.

4CD completion rates for Prepared and Underprepared students Source: 2018 Scorecard. Completion definition with: • Lowest attempted English or Math course was ‘Prepared for College Level’ o CB 21 COURSE-PRIOR-TO-COLLEGE-LEVEL = A, Y in Math, if Y then Math attempted should be T ‘Transferable’ or D ‘Degree Applicable’ o CB 21 COURSE-PRIOR-TO-COLLEGE-LEVEL = Y in English and English =T ‘Transferable’ o If attempted both Math and English, lowest of Math or English cannot be remedial level for ‘college level’ group. Gap = 30.7 Gap = 27.1

Completion rates for prepared students at all 114 colleges 4CD Bay Area is the highest performing region 2018 Scorecard

Completion rates for underprepared students at 114 colleges Bay Area and Southern Boarder are the highest performing region 4CD 2018 Scorecard

pulse check Any questions ?

Completion milestones Examining the pathway progression for our completion-oriented students is revealing

…easy, peasy, lemon, squeezy … Completion viewed through a pathway framework # in Starting Cohort # Earning 30 Units # Completion Completion oriented students Transferable / Degree-Applicable Units Degree, Certificate, Transfer, Transfer Prepared …easy, peasy, lemon, squeezy … * The California Student Success Scorecard defines the starting cohort as The number of first-time students with a minimum of 6 units earned within six years who also attempted any Math or English in the first three years who then achieved any of the following outcomes within six years of entry: Earned AA/AS or credit Certificate, Transfer to any four-year institution, Achieved “Transfer Prepared” Status (earned 60+ transferable units).

Where are the leakages? # in Starting Cohort # Earning 30 Units See notes below. - 40% - 28% # in Starting Cohort # Earning 30 Units Double check my calculations for the gaps. I wasn’t certain what I should be using to convey the point you are trying to make. # Completing 4,685 3,338 2,616

Median Time to Degree = 4.9 Years Median Time to Degree = 3.5 Years See notes below. Median Time to Degree = 4.9 Years 4CD Starting Cohort 76% - 43% Median Time to Degree = 3.5 Years - 42% 24% - 33% 0.1% Please double check my calculations. 2018 Scorecard. Prepared Students Unprepared Students

Average for California Community Colleges Underprepared Students* time to degree is linked to excess unit accumulation Of students obtaining an AA/AS Degree See notes below. Average for California Community Colleges Prepared Students Underprepared Students* 4.1 Years 3.5 Years 4.9 Years Median Time to Degree: Didn’t update this slide for 2018 presentation Median Units Earned: 78 units 69 units 89 units Data Source: Campaign for College Opportunity (2016). * Figures are for students completing any basic skills course in math or English

Helping underprepared students complete their basic skills sequence Completion-oriented Students College level coursework 4-Years 3 out of 4 students start here. How many complete the given sequence and reach college level? % completing Progress milestone # Prepared for college Pre-collegiate coursework College level coursework # Underprepared for college % completing English Math

Defining basic skills sequence completion The basic skills math and English pathway to college level coursework # of students that take a basic skills course in math and English Who then pass a college-level course in the same discipline within 6 years Basic Skills Sequence Caveat: The level at which students start their sequence is not factored into the scorecard calculation 4 Levels Below 3 Levels Below 2 Levels Below 1 Level Below

Defining the basic skills improvement rate for English Basic Skills English Sequence 3 Levels Below 2 Levels Below 1 Level Below Transfer Level English The percent of students attempt any course in the basic skills English sequence Who then pass the transfer transfer-level English within 6 years

Five year trend in Basic Skills English Improvement Rates 4CD California State Average Source: 2018 Scorecard. REMEDIAL PROGRESS (MATH, ENGLISH, ESL) Definition: For five cohort years, the percentage of credit students who attempted a course designated at “levels below transfer” in: • Math and successfully completed a college-level course in Math within six years. • English and successfully completed a college-level course in English within six years. • ESL and successfully completed the ESL sequence or a college-level English course within six years. The cohort is defined as the year the student attempts a course at “levels below transfer” in Math, English and/or ESL at that college. Outcomes in Basic Skills Education are reported for Math, English and/or ESL Definition: For five cohort years, the percentage of credit students who attempted a course designated at “levels below transfer” in English and successfully completed a college-level course in English within six years. The cohort is defined as the year the student attempts a course at “levels below transfer” in English at that college.

Basic Skills: English improvement rates for all 114 colleges 4CD 2018 Scorecard Bay Area and Southern Boarder are the highest performing region

Defining the basic skills improvement rate for Math Basic Skills English Sequence 3 Levels Below 2 Levels Below 1 Level Below Transfer Level English Math Sequence 4 Levels Below 3 Levels Below 2 Levels Below 1 Level Below or Transfer Level Math Degree Applicable The percent of students attempt any course in the basic skills math sequence ESL Sequence Who then pass either the transfer transfer-level or degree-applicable math course within 6 years Transfer Level English 6 Levels Below 5 Levels Below 4 Levels Below 3 Levels Below 2 Levels Below 1 Level Below or

Five year trend in Basic Skills Math Improvement Rates 4CD California State Average Source: 2017 Scorecard. REMEDIAL PROGRESS (MATH, ENGLISH, ESL) Definition: For five cohort years, the percentage of credit students who attempted a course designated at “levels below transfer” in: • Math and successfully completed a college-level course in Math within six years. • English and successfully completed a college-level course in English within six years. • ESL and successfully completed the ESL sequence or a college-level English course within six years. The cohort is defined as the year the student attempts a course at “levels below transfer” in Math, English and/or ESL at that college. Outcomes in Basic Skills Education are reported for Math, English and/or ESL Definition: For five cohort years, the percentage of credit students who attempted a course designated at “levels below transfer” in Math and successfully completed a college-level course in Math within six years. The cohort is defined as the year the student attempts a course at “levels below transfer” in Math at that college.

Basic Skills: Math improvement rates for all 114 colleges 4CD 2017 Scorecard

We miss a lot by not looking at level of placement 4 Levels Below 3 Levels Below 2 Levels Below 1 Level Below The likelihood a student completes the basic skills sequence: See notes below. 70% 35% This slide was not updated along with the others using 2018 Scorecard. 10/15/18. JD 17% 8% * Data source: The Basic Skills Cohort Tracker. Data are averages for a Fall 2011 cohort pulled from a sample of California Community Colleges that included all colleges for which the Cohort Tracker had sufficient data over the examined period.

pulse check Any questions ?

One key to success is faster completion of basic skills coursework New Transfer Level Achievement Indicator: Definition: The percent of first-time students who complete 6 units and attempt any Math or English in their first year and complete a transfer-level course in Math or English within their first year. And a second measure. College level coursework 4-Years % completing Progress milestone # Prepared for college Pre-collegiate coursework College level coursework # Underprepared for college % completing English Progress milestone Math Pace milestone

One year transfer improvement rates for math Bay Area is the high performer 4CD 2018 Scorecard

One year transfer improvement rates for English Bay Area is the middle performer 4CD 2018 Scorecard

Quick summary of data on completion-oriented students 4CD is an above average or strong performer on all completion metrics 3 of 4 students wanting a degree/transfer require remediation Getting more students through the remediation sequence is key to improving completing rates and improving student equity We have multiple interventions that (a) improve student placement, (b) shorten the time needed to complete basic skills coursework and (c) set students up for higher levels of program completion

Let’s now turn our attention to our CTE-oriented students Completion-oriented Students Let’s now turn our attention to our CTE-oriented students College level coursework 4-Years % completing Progress milestone # Prepared for college Pre-collegiate coursework College level coursework # Underprepared for college % completing English Progress milestone Math Pace milestones Career-oriented Students % completing # Credential-seeking CTE pathway course taking % wage gains achieved # Non credential-seeking “Skills Builder” course taking

Simplified definition: The Scorecard CTE Completion rate definition Simplified definition: we look at students whose course taking pattern suggests an intention of getting a CTE credential Definition: For five cohort years, the percentage of students who completed a CTE course for the first-time and completed more than 8 units in the subsequent three years in a single discipline (2-digit vocational TOP code where at least one of the courses is occupational SAM B or C) and who achieved any of the following outcomes within six years of entry: • Earned any AA/AS or credit Certificate (Chancellor’s Office approved) • Transfer to four-year institution (students shown to have enrolled at any four-year institution of higher education after enrolling at a CCC) • Achieved “Transfer Prepared” (student successfully completed 60 UC/CSU transferable units with a GPA >= 2.0). Source: http://scorecard.cccco.edu/scorecard.aspx Definition: The percentage of first-time students who achieve transfer level math or English in their first and second year. Cohort: Look system-wide to determine first-time status defined as a student who took a credit course in the CCC system for the first time. Students with prior enrollments outside the CCC system are also excluded. SB01 STUDENT IDENTIFIER STATUS = S (only students with valid SSN) SX03 ENROLLMENT-UNITS-EARNED >= 6 at your college and/or anywhere in the CCC system during the first year of enrollment. Units earned include completed courses with grade of A, B, C, D, or P. Attempted a math or English course during their first year: CB03 COURSE-TOP-CODE = 1701.00, 1501.00, 1520.00 CB21 COURSE-PRIOR-TO-COLLEGE-LEVEL = A, B, C, D, Y Outcomes: Completed transfer math course in first or second year : CB03 COURSE-TOP-CODE = 1701.00 CB05 COURSE-TRANSFER-STATUS = A, B, C(N) Completed transfer English course in first or second year : CB03 COURSE-TOP-CODE = 1501.00

Five year trend in CTE Completion Rates California State Average 4CD Source: 2018 Scorecard. CAREER TECHNICAL EDUCATION (CTE) RATE Definition: For five cohort years, the percentage of students who completed a CTE course for the first-time and completed more than 8 units in the subsequent three years in a single discipline (2-digit vocational TOP code where at least one of the courses is occupational SAM B or C) and who achieved any of the following outcomes within six years of entry: • Earned any AA/AS or credit Certificate (Chancellor’s Office approved) • Transfer to four-year institution (students shown to have enrolled at any four-year institution of higher education after enrolling at a CCC) • Achieved “Transfer Prepared” (student successfully completed 60 UC/CSU transferable units with a GPA >= 2.0) Definition: For five cohort years, the percentage of students who completed a CTE course for the first-time and completed more than 8 units in the subsequent three years in a single discipline (2-digit vocational TOP code where at least one of the courses is occupational SAM B or C) and who achieved any of the following outcomes within six years of entry: • Earned any AA/AS or credit Certificate (Chancellor’s Office approved) • Transfer to four-year institution (students shown to have enrolled at any four-year institution of higher education after enrolling at a CCC) • Achieved “Transfer Prepared” (student successfully completed 60 UC/CSU transferable units with a GPA >= 2.0).

CTE Completion rate for all 144 colleges 4CD 2017 Scorecard Bay Area is a lower performing region Part of the reason is that Bay Area students tend to pursue non-credential CTE pathways at very high rates

Career-oriented Students Completion-oriented Students College level coursework 4-Years % completing Progress milestone # Prepared for college Pre-collegiate coursework College level coursework # Underprepared for college % completing English Progress milestone Math Pace milestones Career-oriented Students % completing # Credential-seeking CTE pathway course taking % wage gains achieved # Non credential-seeking “Skills Builder” course taking

New Scorecard metric: Skills Builder Skills Builder Definition: The median percentage change in wages for students who completed higher level CTE coursework in 2012-2013 and left the system without receiving any type of credential. not a pathways completion measure cannot create a district level equivalent

Wage improvement rates for Skills Builder students at 114 colleges 2018 Scorecard The median percentage change in wages for students who completed higher level CTE coursework in 2013-2014 and left the system without receiving any type of traditional outcome such as transfer to a four year college or completion of a degree or certificate

pulse check Any questions ?

Summary of 4CD performance relative to state average We need to keep in mind that most of our trends are flat or slow growing and that we are now being asked by the state to demonstate steady improvement

A word or two about student equity the equity challenge

4CD Completion Rates by Student Ethnicity 100% 4CD Completion Rates by Student Ethnicity 90% 89% Completion Rates for Prepared Students 80% 76% Asians 71% 70% 70% 69% 63% 60% White 59% Completion Rates for Unprepared Students 52% 50% Hispanics 48% 44% Afr Americans 42% 40% 38% 30% Source: 2018 Scorecard 20% Placement level explains far more of the variation in completion than student ethnicity 10% 0% Overall Completion Prepared Students Unprepared Students

What percent place 3 or 4 levels below? A big part of our equity challenge is lurking in the placement of students 4 Levels Below 3 Levels Below 2 Levels Below 1 Level Below See notes below. What percent place 3 or 4 levels below? 70% 35% Asian & White Af American & Hispanic This slide was not updated along with the others using 2018 Scorecard. 10/15/18. JD 17% 25% 55% 8% * This data reflect the distribution for the developmental math sequence only.

Degree obtaining students 2011/12 – 2014/15 Which links back to time-to-degree Degree obtaining students 2011/12 – 2014/15 Gap = ½ year Gap = ~2 Years

student equity is linked to remediation and placement The better we get at moving students quickly into college level work, the better we get at achieving equitable outcomes

Landscape summary of trend and gap performance 4CD Average 4CD Equity Gap Completion-oriented students See notes below. Update gap info. Inc shrinking Prepared student completion Widening Inc Underprepared student completion CTE-oriented students Credential-seeking Inc No chng I’m not sure how you got this. Is it year over year, or from first cohort year to last cohort? I updated the 4cd column using year over year difference. I’m not sure how you calculated the equity gap. Please update that column. Non-credential seeking N/A N/A Remediation Inc English No chng Math Flat shrinking

Final wrap up Any questions ?

Thank you And special thanks to the Contra Costa District research staff who demonstrated outstanding data collection and analysis in support of the findings included in this presentation