1. Thursday, March 17, 2016 10:00 AM

2. Background Cohort options and examples Methods and technology Questions 2 Agenda

3. Rules of the Road

4. Who we are Brad Phillips, PhD President/CEO Mary Kay Patton, MPPA Project and Data Quality Manager John Watson, PhD Sr. Director Analytics 4

5. Which of the following best describes you… a.District/campus administrator b.Educational researcher c.Student equity coordinator d.IT or database personnel e.Faculty member f.Other 5 Poll: Who is with us today?

6. What we do IEBC helps education stakeholders use data and information to make informed decisions, improve practice, and increase student success. IEBC has extensive experience with a collaborative, action research-based model that links data across educational segments. 6

7. Learn how some cohort definitions may exclude the very students you are trying to address Understand methods for building student cohorts for student equity analysis and why using one method over another really matters Review tools and techniques used for developing metrics for cohort analysis 7 Takeaways

8. 8 Background

9. We have our inclusive, equity hat on today The focus today is on the completion metrics Much of the source for this data, as recommended by the CCCCO is from the ARCC report (Student Success Scorecard) – Never intended for this use If we want to assume that all students (or almost all) come to college hoping to achieve, behavioral metrics are not appropriate A way to check this: “Are all students equally represented in the cohort?” If not, there is a problem with how the cohort is constructed 9 Background

10. 10 Examples of Alternate Cohorts

11. Data set used comes from CA community colleges who have participated in our CA Transitions project We start with FTIC students in the 08-09 academic year We are developing the cohort with an eye on only completion Our base cohort references the Student Success design: 11 Introduction

12. Had a minimum of 6 units59,656 12 Altering Filters Distinct Student Counts, FTIC, academic year 08-09

13. Had a minimum of 6 units59,656 Attempted credit courses earning some but fewer than 6 units 6,83611% more 13 Altering Filters Distinct Student Counts, FTIC, academic year 08-09

14. Had a minimum of 6 units59,656 Attempted credit courses earning some but fewer than 6 units 6,83611% more Attempted credit courses, but received 0 units26,47544% more Total amended cohort:92,967 14 Altering Filters Distinct Student Counts, FTIC, academic year 08-09

15. Race/Ethnicity Age Gender Financial Aid DSPS Veterans Foster Youth 15 Student Equity

16. 16 Demographic Changes After developing several alternate cohorts, we began review of various characteristics, looking for differences in demographics that may be hidden in the alternate cohorts Initially looked at race/ethnicity within the cohort of students who registered for credit courses but did not earn the 6 or more units. The following findings are from a subset of the districts we generated alternate cohorts for. Note that these characteristics are not present in all districts. Individual district demographics and other characteristics likely affect individual findings

17. 17 Demographic Changes >= 6 units earned 9% Black 23% Hispanic

18. 18 Demographic Changes.1 to 5.9 units earned 16% Black 26% Hispanic

19. 19 Demographic Changes 0 units earned 15% Black 26% Hispanic

20. Original Cohort Alternate cohortChange Range of Change across districts Asian10%4%-6% -11.1% to 2.4% Black9%15%6% -0.9% to 6.2% Hispanic23%26%3% -15.2% 1 to 16.2% White42%37%-5% -5.9% to 8.7% 2 20 Demographic Changes Percentage composition, select ethnicities Notes: Most districts showed increases in percentage of black and Hispanic students in alternate cohorts and decreases in percentage of white and Asian students. 1 This min was only found at one district. Other districts were in 0% to max range. 2 This max was only found at one district. Other districts were in -5.9 to 0% range.

21. 21 Methods and Technology

22. Developing alternate cohorts Identifying questions of interest early helps to shape the alternate cohort effort In this work, we used Microsoft SQL, Tableau Desktop and Excel. We also have utilized R and STATA and Dundas BI in other projects. Limitations to data available: Locally held and CCCCO referential files don’t provide the complete picture due to activity outside of local district

23. Developing alternate cohorts Method used Use known data set: referential files Initial exploration based on informed question or hunch Developed initial cohort, and alternate models Validate cohort results against other sources Review cohort results to address questions of interest Develop metrics for cohorts that best fit local needs Develop final output

24. 24 Cohort option 1: credit threshold Evaluate results Are they more inclusive and show disparity? No? OK, let’s slice this a different way. Data sourced and loaded Cohort tables created Yes. Develop report Developing alternate cohorts Cohort option 2: student goals Cohort option 3: Attempt Eng./math Share results Cohort option 4: Other

25. 25 Follow-up

26. Things to think about With our equity hat on we recommend: Need to focus on correctly and accurately understanding this population of interest Investigate your district’s data using these alternative cohort definitions (or others) Discuss these finding with the Student Equity workgroup at your college

27. Questions and Feedback

28. IEBC Contact Information Cohort-generation algorithms, metric and output technical design John WatsonEmail: jwatson@iebcnow.org Phone: (530) 204-7129 Questions about CCCCO data, data quality, cohort-definitions Mary Kay PattonEmail: mkpatton@iebcnow.org Phone: (916) 995-3183 General questions about student equityBrad PhillipsEmail: bphillips@iebcnow.org Phone: (619) 252-8503 For more information about IEBC Analytics, visit www.iebcnow.org and click on Our Services > Analytics www.iebcnow.org

29. End.