1. DATA IN GEAR UP (DIG) Impact of GEAR UP Kentucky II On College Enrollment Judy H. Kim, Ph.D, Evaluation Consulting Group S eptember 10, 2013

2. Introduction Description of DIG Purpose Research Questions Methods Findings Conclusion * There will be a lot of information. Please feel free to ask questions throughout the presentation. AGENDA

3. US ED/RTI award For 2011 awardees only Purpose: Data acquisition and utilization DESCRIPTION OF DIG

4. WHY WE DID THIS STUDY: Assess impact of GEAR UP On college enrollment On persistence On which populations Examine effectiveness Of overall services Of isolated services PURPOSE

5. WHAT WE ASKED: 1.Are there notable differences in college enrollment and persistence between GEAR UP students and other students from low-income schools? 2.How does GEAR UP perform as a predictor of college enrollment and persistence in comparison to other predictors? 3.Which services, or combinations of services, have had the greatest impact in promoting college enrollment and persistence? RESEARCH QUESTIONS

6. HOW WE ANSWERED IT Descriptive analysis College enrollment by race/gender College persistence by race/gender Service data Logistic regression GEAR UP impact Linear regression Impact of instances and duration of services METHODS

7. GUK II  GUK II Cohort 1  HS graduating class of 2011  Complete EPAS data (7th-11 th )  Demographics  N=2,202  2,157 matched (45 deleted) Comparison  From non-GU KY schools with FRPL status of 50%+  HS graduating class of 2011  EPAS data (8 th, 10 th, 11 th )  Demographics COMPARISON GROUP METHODS

8. MATCHING COMPARISON GROUP Pool of 9,900 students Matching variables  Sex  Race  Caucasian  African American  Latino  8 th grade EXPLORE composite scores  Zero tolerance and exact matching  8 cases unable to match  Resulting in 2,149 GU and 2,149 non-GU METHODS

9. COLLEGE ENROLLMENT RQ 1: Are there notable differences in college enrollment and persistence between GEAR UP students and other students from low-income schools?

10. FINDINGS HS graduation and College Enrollment GUK II (n=2,149) Non-GU (n=2,149) #%#% Graduated from High School 1,16054.0%1,16754.3% Enrolled Full/Half Time in College 93543.5%76635.6% College enrollment 2 yr. and 4-yr institutions

11. GUK II Group  N=935  Sex  547 female (58.5%)  388 male (41.5%)  Race  850 Caucasian (90.9%)  73 African American (7.8%)  5 Latino (0.5%) Comparison Group  N=766  Sex  444 female (57.9%)  322 male (42.0%)  Race  693 Caucasian (90.5%)  55 African American (7.2%)  11 Latino (1.4%) COLLEGE ENROLLMENT BY GENDER/RACE BREAKDOWN FINDINGS

12. FINDINGS: CE BY GENDER AND RACE* College Enrollment by Gender and by Race* GUK IINon-GU #%#% College Enrollment Total 93543.5%76635.6% Female (n=1,079) Male (n=1,070) 547 388 50.7% 36.3% 444 322 41.1% 30.1% Black (n=162) Latino (n=32) White (n=1,939) 73 5 850 45.1% 15.6% 43.8% 55 11 693 34.0% 34.4% 35.7% More GUK II students enrolled in college than comparison group across categories, with the exception of Latino students More GUK II females enrolled in college by 9.6 percentage points More GUK II males enrolled in college by 6.2 percentage points More GUK II African American students enrolled in college by 11.1 percentage points More GUK II Caucasian students enrolled in college by 8.1 percentage points *Race categories are not comprehensive

13. FINDINGS: CE BY RACE* BY GENDER College Enrollment by Race* by Gender GUK IINon-GU #%#% College Enrollment Total 935 43.5% 766 35.6% Black (n=162) Female (n=82) Male (n=80) 73 39 34 45.1% 47.6% 42.5% 55 30 25 34.0% 36.6% 31.3% White (n=1,939) Female (n=981) Male (n=958) 850 500 350 43.8% 51.0% 36.5% 693 407 286 35.7% 41.5% 29.9% Disaggregated by race and gender, GUK II students within categories still enrolled in college at higher rates than that of the comparison group. More GUK II African American females enrolled by 11 percentage points More GUK II African American males enrolled by 11.2 percentage points More GUK II Caucasian females enrolled by 9.5 percentage points More GUK II male students by 6.6 percentage points

14. RACE/GENDER SUMMARY  Considerably more GUK II students enrolled in college than comparison group  Disaggregated by race, more GUK II Caucasian and African American students enrolled in college than comparison group  Disaggregated by race and gender, more GUK II African American male, African American female, Caucasian male, and Caucasian female students enrolled in college than comparison group FINDINGS

15. EPAS SUBGROUPS We wanted to know: If/how EPAS performance affected college enrollment If EPAS performance was a factor in the greater college enrollment of the GUK II students ------------------------ We discovered that was a LARGE query We narrowed the scope to math and reading (to parallel NCLB requirements at that time period)

16. FINDINGS: COLLEGE ENROLLMENT BY MATH SUBGROUP SubgroupEXPLOREPLANACTGUK IINon-GU #%#% Overall -- 93543.5%76635.6% M1 yes 103 (n=139) 74.1% 130 (n=182) 71.4% M2 yes no 23 (n=40) 57.5% 21 (n=43) 48.8% M3 yesnoyes 11 (n=20) 55.0% 20 (n=32) 62.5% M4 yesno 43 (n=80) 53.8% 25 (n=57) 43.9% M5 no 612 (n=1629) 37.6% 448 (n=1583) 28.3% M6 no yes 41 (n=70) 58.6% 26 (n=51) 51.0% M7 noyes 45 (n=63) 71.4% 53 (n=93) 57.0% M8 noyesno 57 (n=103) 55.3% 43 (n=114) 37.7%

17. FINDINGS: COLLEGE ENROLLMENT BY READING SUBGROUP subgroupEXPLOREPLANACTGUK IINon-GU #%#% overall -- 93543.5%76635.6% R1 yes 212 (n=327) 64.8% 227 (n=360) 63.1% R2 yes no 48 (n=78) 61.5% 55 (n=86) 64.0% R3 yesnoyes 42 (n=63) 66.7% 23 (n=43) 53.5% R4 yesno 53 (n=96) 55.2% 16 (n=63) 25.4% R5 no 363 (n=1154) 31.5% 237 (n=1094) 21.7% R6 no yes 37 (n=83) 44.6% 41 (n=88) 46.6% R7 noyes 77 (n=129)59.7%73 (n=154)47.4% R8 noyesno 103 (n=214) 48.1%94 (n=267)35.2%

18. FINDINGS: COLLEGE ENROLLMENT BY MATH AND READING SUBGROUP subgroupEXPLOREPLANACTGUK IINon-GU MRMRMR#%#% overall 93543.5%76635.6% M&R1 YYYYYY 74 (n=98) 75.5%101 (n=133)75.9% M&R2 YNYNYN 4 (n=6) 66.7% 5 (n=9) 55.6% M&R3 NNNNNN 322 (n=1061) 30.3% 204 (n=1007) 20.3% M&R4 NYNYNY 57 (n=107) 53.3% 55 (n=109) 50.5%

19. EPAS MATH AND READING SUBGROUP SUMMARY  More GUK II students enrolled in college across most of the EPAS categories  Notably, there were more GUK II students who enrolled in college than comparison group, even though they did not meet benchmarks  Suggests a GEAR UP effect that goes beyond academic performance FINDINGS

20. COLLEGE PERSISTENCE RQ 1: Are there notable differences in college enrollment and persistence between GEAR UP students and other students from low-income schools?

21. PERSISTENCE DEFINED High school graduating class of 2011 who enrolled in college in the fall of 2011 who then continued into their second year of college in the fall of 2012 Half-time and full-time 2-year and 4-year institutions DEFINITION

22. FINDINGS: COLLEGE PERSISTENCE BY 2- & 4-YR INSTITUTIONS GUK IINon-GU # %#% College Persistence Total 726 (n=935) 77.6% 609 (n=766) 79.5% Two-Year 258 (n=369) 69.9% 127 (n=179) 70.9% Four-Year 468 (n=566) 82.7% 482 (n=587) 82.1%

23. FINDINGS: COLLEGE PERSISTENCE BY GENDER & RACE GUK IINon-GU #%#% CP Total 726 (n=935) 77.6% 609 (n=766) 79.5% Female Male 446 (n=547) 280 (n=388) 81.5% 72.2% 372 (n=444) 237 (n=322) 83.8% 73.6% Black Latino White 46 (n=73) 2 (n=5) 673 (n=850) 63.0% 40.0% 79.2% 40 (n=55) 10 (n=11) 554 (n=693) 72.7% 90.9% 79.9% Overall, total N of GUK II who enrolled in the second year of college from high school graduation was still higher, but the actual CP rate was slightly lower than comparison group. Non-GU female students persisted more by 2.3 percentage points Non-GU male students persisted more by 1.4 percentage points Non-GU African American students persisted more by 9.7 percentage points Non-GU Caucasian students persisted more by 0.7 percentage points.

24. GUK IINon-GU #%#% College Persistence Total 726 (n=935) 77.6%609 (n=766) 79.5% Black Female Male 46 (n=73) 26 (n=39) 20 (n=34) 63.0% 66.7% 58.8% 40 (n=55) 24 (n=30) 16 (n=25) 72.7% 80.0% 64.0% White Female Male 673 (n=850) 415 (n=500) 258 (n=350) 79.2% 83.0% 73.7% 554 (n=693) 341 (n=407) 213 (n=286) 79.9% 83.8% 74.5% Disaggregated by race and gender:  More non-GU African American female students persisted by 13.3 percentage points  More non-GU African American male students persisted by 5.2 percentage points  More non-GU Caucasian female students persisted by 0.8 percentage points  More non-GU Caucasian male students persisted by 0.8 percentage points FINDINGS: COLLEGE PERSISTENCE BY GENDER & RACE

25. COLLEGE PERSISTENCE SUMMARY GUK II students persisted into the second year of college at a slightly lower rate than comparison group Total N from high school graduation to second year of college was still greater than comparison group The loss was with African American students FINDINGS

26. GUK II AS A PREDICTOR FOR COLLEGE ENROLLMENT RQ 2: How does GEAR UP perform as a predictor of college enrollment and persistence in comparison to other predictors?

27. PREDICTOR DEFINED “Other” predictors were ACT benchmarks in English, math, reading, science; gender; race Did not/could not factor in all the EPAS work that GUK II actually implemented DEFINITION

28. Predictor variables: GEAR UP student status, attainment of the ACT English, math, reading, and science benchmarks, Male, Black, and Latino 68.4% correctly classified FINDINGS: LOGISTIC REGRESSION

29. GUK is a modestly successful predictor for college enrollment, better than ACT science and reading benchmarks, being African American, Latino, and male FINDINGS: GUK AS A PREDICTOR

30. GUK II AND COLLEGE ENROLLMENT RQ 3: which services, or combinations of services, have had the greatest impact in promoting college enrollment and persistence?

31. GUK II SERVICES Awareness programs focused on providing counseling/information for students and parents about the value of college, preparation for postsecondary education, and college admission requirements Rigor focused on ensuring all students have access to rigorous coursework, and that teachers and schools are equipped to close the achievement gap. Engagement focused on helping parents set high expectations for their children including planning for college education Access focused on educating parents and students about federal and state financial aid resources that make college affordable Support services focused on preventing students from failing by providing targeted academic services, or supplemental enrichment and developmental instruction for selected students DEFINITION

32. GUK II SERVICES DEFINED Data collection Service data available for grades 8-12 Service data available by school Data analysis Eliminated grade 8 Focused on the five, large service categories Generalized school numbers to the student-level DEFINITION

33. FINDINGS: GUK II HS SERVICE PROFILES GUK II High School GUK II Students College Enrolled % College Enrolled Service Instances Service Duration Instances / Student Duration / Student 025040 905358.9%434,340 0.4848 061060 975152.6%71,680 0.0717 075030 663756.1%312,680 0.4741 105120 1286953.9%458,735 0.3568 115050 1006161.0%92,225 0.0922 125100 1246552.4%161,080 0.139 134019 733142.5%16016,573 2.19227 165039 13753.8%344,200 2.62323 165105 533667.9%515,005 0.9694 165170 874754.0%373,266 0.4338 171035 1147263.2%414,630 0.3641 215065 694565.2%273,710 0.3954 245045 743445.9%244,086 0.3255 272011 181266.7%121,290 0.6772 275031 20525.0%4311,594 2.15580

34. GUK II High School GUK II Students College Enrolled % College Enrolled Service Instances Service Duration Instances / Student Duration / Student 275057 562035.7%506,705 0.89120 275335 16425.0%637,347 3.94459 295265 1146657.9%455,150 0.3945 325350 774558.4%101,260 0.1316 351140 502040.0%242,765 0.4855 392060 422150.0%532,885 1.2669 425050 643656.3%166,800 0.25106 431450 1085550.9%275,311 0.2549 452070 582441.4%929,295 1.59160 485130 221150.0%71,110 0.3250 485250 1438962.2%202,950 0.1421 495040 1104540.9%376,370 0.3458 523030 371951.4%363,160 0.9785 585130 16850.0%3840 0.1953 591430 1638250.3%14480 0.093 FINDINGS: GUK II HS SERVICE PROFILES

35. FINDINGS: “TYPICAL” GUK II STUDENTS ServiceNo CollegeCollege EnrolledPersistent Awareness Instances12.6211.50 Duration2,2361,9431,942 Rigor Instances3.173.13 Duration1,0301,0231,018 Engagement Instances1.821.57 Duration252216 Access Instances4.824.30 Duration479397398 Support Instances13.7412.4112.42 Duration638623622 Total Instances36.1732.91 Duration4,6354,2024,196

36. REGRESSION MODELS # of service instances and duration of services by student, predicting CE had a multiple R 2 of 0.45 Inconclusive ELABORATE LINEAR REGRESSION # of service instances and duration of services by the five service areas, predicting CE had a multiple R 2 of 0.71 High degree of correlation Inconclusive LINEAR REGRESSION FINDINGS

37. ANALYSIS OF SERVICES SUMMARY Exploratory analyses resulted in inconclusive findings (as expected) Services targeted at-risk populations Awareness and Support had the highest instances per student average Model established for further analysis FINDINGS

38. More GUK students went to college than the comparison group More African American GUK II students went to college than the comparison group Extensive numbers and duration of services were provided to thousands of students with the goal of college-going in mind Further analysis is needed CONCLUSION GUK IS MAKING A DIFFERENCE!

39. QUESTIONS? COMMENTS? Judy Kim judy.kim@evalconsulting.org 240.449.6428