1. Doing What's Right Means Telling the Right Story:
Reporting Student Equity Completion Outcomes
Thursday, March 17, :00 AM

2. Agenda Questions Background Cohort options and examples
5/10/2018
Agenda
Background
Cohort options and examples
Methods and technology
Questions

3. Rules of the Road

4. Who we are Brad Phillips, PhD Mary Kay Patton, MPPA
5/10/2018
Who we are
Brad Phillips, PhD
President/CEO
Mary Kay Patton, MPPA
Project and Data Quality Manager
John Watson, PhD 
Sr. Director Analytics

5. Poll: Who is with us today?
5/10/2018
Poll: Who is with us today?
Which of the following best describes you…
District/campus administrator
Educational researcher
Student equity coordinator
IT or database personnel
Faculty member
Other

6. 5/10/2018
What we do
IEBC helps education stakeholders use data and information to make informed decisions, improve practice, and increase student success.
IEBC has extensive experience with a collaborative, action research-based model that links data across educational segments.

7. 5/10/2018
Takeaways
Learn how some cohort definitions may exclude the very students you are trying to address
Understand methods for building student cohorts for student equity analysis and why using one method over another really matters
Review tools and techniques used for developing metrics for cohort analysis

8. 5/10/2018
Background

9. Background We have our inclusive, equity hat on today
5/10/2018
Background
We have our inclusive, equity hat on today
The focus today is on the completion metrics
Much of the source for this data, as recommended by the CCCCO is from the ARCC report (Student Success Scorecard)
Never intended for this use
If we want to assume that all students (or almost all) come to college hoping to achieve, behavioral metrics are not appropriate
A way to check this: “Are all students equally represented in the cohort?”
If not, there is a problem with how the cohort is constructed

10. Examples of Alternate Cohorts
5/10/2018
Examples of Alternate Cohorts

11. 5/10/2018
Introduction
Data set used comes from CA community colleges who have participated in our CA Transitions project
We start with FTIC students in the academic year
We are developing the cohort with an eye on only completion
Our base cohort references the Student Success design:

12. Altering Filters Had a minimum of 6 units 59,656
5/10/2018
Altering Filters
Distinct Student Counts, FTIC, academic year 08-09
Had a minimum of 6 units
59,656

13. Altering Filters Had a minimum of 6 units 59,656
5/10/2018
Altering Filters
Distinct Student Counts, FTIC, academic year 08-09
Had a minimum of 6 units
59,656
Attempted credit courses earning some but fewer than 6 units
6,836
11% more

14. Altering Filters Had a minimum of 6 units 59,656
5/10/2018
Altering Filters
Distinct Student Counts, FTIC, academic year 08-09
Had a minimum of 6 units
59,656
Attempted credit courses earning some but fewer than 6 units
6,836
11% more
Attempted credit courses, but received 0 units
26,475
44% more
Total amended cohort:
92,967

15. Student Equity Race/Ethnicity Age Gender Financial Aid DSPS Veterans
5/10/2018
Student Equity
Race/Ethnicity
Age
Gender
Financial Aid
DSPS
Veterans
Foster Youth

16. 5/10/2018
Demographic Changes
After developing several alternate cohorts, we began review of various characteristics, looking for differences in demographics that may be hidden in the alternate cohorts
Initially looked at race/ethnicity within the cohort of students who registered for credit courses but did not earn the 6 or more units.
The following findings are from a subset of the districts we generated alternate cohorts for.
Note that these characteristics are not present in all districts. Individual district demographics and other characteristics likely affect individual findings

17. Demographic Changes >= 6 units earned 9% Black 23% Hispanic
5/10/2018
Demographic Changes
>= 6 units earned
9% Black
23% Hispanic

18. Demographic Changes .1 to 5.9 units earned 16% Black 26% Hispanic
5/10/2018
Demographic Changes
.1 to 5.9 units earned
16% Black
26% Hispanic

19. 5/10/2018
Demographic Changes
0 units earned
15% Black
26% Hispanic

20. Range of Change across districts
5/10/2018
Demographic Changes
Percentage composition, select ethnicities
Original Cohort
Alternate cohort
Change
Range of Change across districts
Asian
10% 4%
-6%
-11.1% to 2.4%
Black 9%
15% 6%
-0.9% to 6.2%
Hispanic
23%
26% 3%
-15.2% 1 to 16.2%
White
42%
37%
-5%
-5.9% to 8.7% 2
Notes: Most districts showed increases in percentage of black and Hispanic students in
alternate cohorts and decreases in percentage of white and Asian students.
1 This min was only found at one district. Other districts were in 0% to max range.
2 This max was only found at one district. Other districts were in -5.9 to 0% range.

21. Methods and Technology
5/10/2018
Methods and Technology

22. Developing alternate cohorts
Identifying questions of interest early helps to shape the alternate cohort effort
In this work, we used Microsoft SQL, Tableau Desktop and Excel. We also have utilized R and STATA and Dundas BI in other projects.
Limitations to data available:
Locally held and CCCCO referential files don’t provide the complete picture due to activity outside of local district

23. Developing alternate cohorts
Method used
Use known data set: referential files
Initial exploration based on informed question or hunch
Developed initial cohort, and alternate models
Validate cohort results against other sources
Review cohort results to address questions of interest
Develop metrics for cohorts that best fit local needs
Develop final output

24. Developing alternate cohorts
Data sourced and loaded
Cohort tables created
Cohort option 1:
credit threshold
Cohort option 2: student goals
Cohort option 3: Attempt Eng./math
Cohort option 4: Other
Evaluate results
Are they more inclusive and show disparity?
No? OK, let’s slice this a different way.
Yes.
Develop report
Share results

25. 5/10/2018
Follow-up

26. Things to think about With our equity hat on we recommend:
Need to focus on correctly and accurately understanding this population of interest
Investigate your district’s data using these alternative cohort definitions (or others)
Discuss these finding with the Student Equity workgroup at your college

27. Questions and Feedback

28. IEBC Contact Information
Cohort-generation algorithms, metric and output technical design
John Watson
Phone: (530)
Questions about CCCCO data, data quality, cohort-definitions
Mary Kay Patton
Phone: (916)
General questions about student equity
Brad Phillips
Phone:  (619)
For more information about IEBC Analytics, visit and click on Our Services > Analytics

29. End.