1. Moving from Aggregate to Subgroup Analysis to Assess Equity and Outcomes: Unpacking the Meaning of Transformative Change Matt S. Giani, Debra D. Bragg, & Heather L. Fox Office of Community College Research and Leadership University of Illinois at Urbana-Champaign TCI Research Symposium November 11, 2015

2. Equity in TCI “Transformative change refers to implementing, sustaining, and scaling change that produces unprecedented results without sacrificing the historic commitment of community colleges to access, opportunity, and equitable outcomes.”

3. Assessing Equity in TAACCCT Q: How do we know if TAACCCT reforms impacted institutional equity? DOL does not require that evaluations assess whether institutions became more equitable as a result of TAACCCT

4. Research Questions 1)What were the postsecondary outcomes of H2P students, and how did outcomes vary according to students’ demographic and socioeconomic background? 2)What impact did H2P make on the outcomes of healthcare students? 3)To what extent did H2P reduce gaps in outcomes stemming from student demographic and socioeconomic characteristics?

5. H2P Consortium

6. H2P’s Vision “Not only produce a highly skilled healthcare workforce but also galvanize a national movement to dramatically redesign and enhance health professional education and training though national curricular reform, industry engagement, innovative practices and programs, and intensive usage of data and accountability systems to ensure student success and program excellence” (H2P Proposal, 2009, pp. 1-2).

7. H2P Co-Grantees Anoka-Ramsey Community College - Coon Rapids, MN Ashland Community & Technical College -Ashland, KY Cincinnati State Technical & Community College - Cincinnati, OH, (H2P Lead) El Centro College - Dallas, TX Jefferson Community & Technical College - Louisville, KY Malcolm X College - Chicago, IL Owens Community College - Toledo, OH Pine Technical & Community College - Pine City, MN Texarkana College - Texarkana, TX

8. H2P’s Strategies 1)Online assessment and enhanced career guidance 2)Contextualized developmental education 3)Competency-based core curriculum 4)Industry-recognized stackable credentials 5)Enhanced retention support 6)Training programs for incumbent health professions workers 7)Enhance data and accountability systems 8)Galvanize a national movement

9. Participant 2. Contextualized Dev Ed (No. students enrolled, No. students increase to college level, No. students earning academic credit) 1. Online Assessment & Career Guidance (No. students, No. credits) 8. National movement (No. using labor market info, No. enrolled in core curriculum) 7. Data (No. students w/ education and employment records) 4. Retention Services (No. enroll fall to spring, No. enroll fall to fall, % completed of attempted) 3. Core Curriculum (No. students enrolled, No. students complete) 5. Stackable credentials (No. certificates per 100 FTE, No. certificates) Outcomes Employment Employment retention Average earnings Credit attainment Certificate attainment ( 1) Degree attainment 6. Incumbent training (No. enrolled, Min. semester credit)

10. H2P Participants – 6,569* Total

11. Methods

12. Research Questions 1)What were the postsecondary outcomes of H2P students, and how did outcomes vary according to students’ demographic and socioeconomic background? 2)What impact did H2P make on the outcomes of healthcare students? 3)To what extent did H2P reduce gaps in outcomes stemming from student demographic and socioeconomic characteristics?

13. Analytic Techniques 1)What were the postsecondary outcomes of H2P students, and how did outcomes vary according to students’ demographic and socioeconomic background? Analyzed mean retention and attainment rates by demographic group Used logistic regression to estimate the impact of demographic characteristics on retention and attainment

14. Analytic Techniques 2)What impact did H2P make on the outcomes of healthcare students? Drew on retrospective sample of healthcare students who enrolled in H2P colleges in fall 2009 Restricted sample to students in LVN and ADN programs Estimated impact of H2P using propensity score matching (PSM)

15. Analytic Techniques 3)To what extent did H2P reduce gaps in outcomes stemming from student demographic and socioeconomic characteristics? Selected particular student subgroups (students of color, Pell eligible, non-employed) Fit separate PSM models to each subgroup Analyses reveal how impact of H2P varied by student subgroup

16. Data Categories Demographic Course Data Credential Data UI Wage Data Strategy Data (credit for prior learning, retention services, etc.)

17. Outcome Variables 1)Any credential 2)Any credential or still enrolled by fall 2014 3)Long term certificate or associate’s degree 4)Associate’s degree

18. Control/Matching Variables Demographic (race/ethnicity, age, gender, Pell eligibility, whether first-time college student) Course data (dev ed participation, pre-cohort credits) UI Wage Data (whether employed quarter before enrollment, prior earnings) College

19. Results

20. Credential and Retention Rates, by College

21. Credential Rates, by Credential Type

22. No Credential, by Race/Ethnicity

23. Long Certificate/Associate’s Attainment, by Race/Ethnicity

24. Credential Attainment, by Pell Eligibility

25. Credential Attainment, by Pre-H2P Employment

26. Logistic Regression Credential or Retained Any Credential Long Certificate or Associate's Associate's Odds Ratio Sig. Odds Ratio Sig. Odds Ratio Sig. Odds Ratio Sig. Race/Ethnicity 1 (White) American Indian / Alaskan Native.964.9461.069.900.794.639.834.751 Asian1.364.123.978.908.878.508.921.709 Black.602***.000.576***.000.615***.000.642**.002 Latino1.029.814.666**.003.621***.001.410***.000 Multi-race.533.102.417*.035.446.150.000.998 Native Hawaiian / Pacific Islander 1.374.5971.189.751.183.104.255.194 Unknown.836.182.407***.000.161***.000.043***.000

27. Impact of H2P on Attainment (PSM)

28. Impact of H2P on Attainment – Students of Color

29. Impact of H2P on Attainment – Pell-Eligibility

30. Impact of H2P on Attainment – Pre-H2P Employment

31. Discussion

32. Limitations No data on transfer students Analyses focused on longer-term programs PSM can’t control for unmeasured differences Subgroup analyses limited by sample sizes Estimates show how big of an effect, but not why the effect occurred (requires mixed-methods evaluation)

33. Conclusions H2P students more likely to receive credential compared to Retro healthcare students H2P students of color had significantly lower attainment rates than white students *H2P students of color had significantly higher attainment rates than Retro students of color Results suggest H2P improved equity of outcomes for students of color Ongoing research is unpacking this effect

34. Questions? How can we help?

35. Contact Info: Matt S. Giani – mgiani@illinois.edumgiani@illinois.edu Debra D. Bragg – dbragg@illinois.edudbragg@illinois.edu Heather L. Fox – hlfox2@Illinois.eduhlfox2@Illinois.edu Office of Community College Research and Leadership (OCCRL) – http://occrl.Illinois.edu http://occrl.Illinois.edu © 2015 Board of Trustees, University of Illinois