Doing What's Right Means Telling the Right Story: Reporting Student Equity Completion Outcomes Thursday, March 17, 2016 10:00 AM

Agenda Questions Background Cohort options and examples 5/10/2018 Agenda Background Cohort options and examples Methods and technology Questions

Rules of the Road

Who we are Brad Phillips, PhD Mary Kay Patton, MPPA 5/10/2018 Who we are Brad Phillips, PhD President/CEO Mary Kay Patton, MPPA Project and Data Quality Manager John Watson, PhD  Sr. Director Analytics

Poll: Who is with us today? 5/10/2018 Poll: Who is with us today? Which of the following best describes you… District/campus administrator Educational researcher Student equity coordinator IT or database personnel Faculty member Other

5/10/2018 What we do IEBC helps education stakeholders use data and information to make informed decisions, improve practice, and increase student success. IEBC has extensive experience with a collaborative, action research-based model that links data across educational segments.

5/10/2018 Takeaways Learn how some cohort definitions may exclude the very students you are trying to address Understand methods for building student cohorts for student equity analysis and why using one method over another really matters Review tools and techniques used for developing metrics for cohort analysis

5/10/2018 Background

Background We have our inclusive, equity hat on today 5/10/2018 Background We have our inclusive, equity hat on today The focus today is on the completion metrics Much of the source for this data, as recommended by the CCCCO is from the ARCC report (Student Success Scorecard) Never intended for this use If we want to assume that all students (or almost all) come to college hoping to achieve, behavioral metrics are not appropriate A way to check this: “Are all students equally represented in the cohort?” If not, there is a problem with how the cohort is constructed

Examples of Alternate Cohorts 5/10/2018 Examples of Alternate Cohorts

5/10/2018 Introduction Data set used comes from CA community colleges who have participated in our CA Transitions project We start with FTIC students in the 08-09 academic year We are developing the cohort with an eye on only completion Our base cohort references the Student Success design:

Altering Filters Had a minimum of 6 units 59,656 5/10/2018 Altering Filters Distinct Student Counts, FTIC, academic year 08-09 Had a minimum of 6 units 59,656

Altering Filters Had a minimum of 6 units 59,656 5/10/2018 Altering Filters Distinct Student Counts, FTIC, academic year 08-09 Had a minimum of 6 units 59,656 Attempted credit courses earning some but fewer than 6 units 6,836 11% more

Altering Filters Had a minimum of 6 units 59,656 5/10/2018 Altering Filters Distinct Student Counts, FTIC, academic year 08-09 Had a minimum of 6 units 59,656 Attempted credit courses earning some but fewer than 6 units 6,836 11% more Attempted credit courses, but received 0 units 26,475 44% more Total amended cohort: 92,967

Student Equity Race/Ethnicity Age Gender Financial Aid DSPS Veterans 5/10/2018 Student Equity Race/Ethnicity Age Gender Financial Aid DSPS Veterans Foster Youth

5/10/2018 Demographic Changes After developing several alternate cohorts, we began review of various characteristics, looking for differences in demographics that may be hidden in the alternate cohorts Initially looked at race/ethnicity within the cohort of students who registered for credit courses but did not earn the 6 or more units. The following findings are from a subset of the districts we generated alternate cohorts for. Note that these characteristics are not present in all districts. Individual district demographics and other characteristics likely affect individual findings

Demographic Changes >= 6 units earned 9% Black 23% Hispanic 5/10/2018 Demographic Changes >= 6 units earned 9% Black 23% Hispanic

Demographic Changes .1 to 5.9 units earned 16% Black 26% Hispanic 5/10/2018 Demographic Changes .1 to 5.9 units earned 16% Black 26% Hispanic

5/10/2018 Demographic Changes 0 units earned 15% Black 26% Hispanic

Range of Change across districts 5/10/2018 Demographic Changes Percentage composition, select ethnicities Original Cohort Alternate cohort Change Range of Change across districts Asian 10% 4% -6% -11.1% to 2.4% Black 9% 15% 6% -0.9% to 6.2% Hispanic 23% 26% 3% -15.2% 1 to 16.2% White 42% 37% -5% -5.9% to 8.7% 2 Notes: Most districts showed increases in percentage of black and Hispanic students in alternate cohorts and decreases in percentage of white and Asian students. 1 This min was only found at one district. Other districts were in 0% to max range. 2 This max was only found at one district. Other districts were in -5.9 to 0% range.

Methods and Technology 5/10/2018 Methods and Technology

Developing alternate cohorts Identifying questions of interest early helps to shape the alternate cohort effort In this work, we used Microsoft SQL, Tableau Desktop and Excel. We also have utilized R and STATA and Dundas BI in other projects. Limitations to data available: Locally held and CCCCO referential files don’t provide the complete picture due to activity outside of local district

Developing alternate cohorts Method used Use known data set: referential files Initial exploration based on informed question or hunch Developed initial cohort, and alternate models Validate cohort results against other sources Review cohort results to address questions of interest Develop metrics for cohorts that best fit local needs Develop final output

Developing alternate cohorts Data sourced and loaded Cohort tables created Cohort option 1: credit threshold Cohort option 2: student goals Cohort option 3: Attempt Eng./math Cohort option 4: Other Evaluate results Are they more inclusive and show disparity? No? OK, let’s slice this a different way. Yes. Develop report Share results

5/10/2018 Follow-up

Things to think about With our equity hat on we recommend: Need to focus on correctly and accurately understanding this population of interest Investigate your district’s data using these alternative cohort definitions (or others) Discuss these finding with the Student Equity workgroup at your college

Questions and Feedback

IEBC Contact Information Cohort-generation algorithms, metric and output technical design John Watson Email: jwatson@iebcnow.org Phone: (530) 204-7129 Questions about CCCCO data, data quality, cohort-definitions Mary Kay Patton Email: mkpatton@iebcnow.org Phone: (916) 995-3183 General questions about student equity Brad Phillips Email: bphillips@iebcnow.org Phone:  (619) 252-8503 For more information about IEBC Analytics, visit www.iebcnow.org and click on Our Services > Analytics

End.