DATA IN GEAR UP (DIG) Impact of GEAR UP Kentucky II On College Enrollment Judy H. Kim, Ph.D, Evaluation Consulting Group S eptember 10, 2013

Introduction Description of DIG Purpose Research Questions Methods Findings Conclusion * There will be a lot of information. Please feel free to ask questions throughout the presentation. AGENDA

US ED/RTI award For 2011 awardees only Purpose: Data acquisition and utilization DESCRIPTION OF DIG

WHY WE DID THIS STUDY: Assess impact of GEAR UP On college enrollment On persistence On which populations Examine effectiveness Of overall services Of isolated services PURPOSE

WHAT WE ASKED: 1.Are there notable differences in college enrollment and persistence between GEAR UP students and other students from low-income schools? 2.How does GEAR UP perform as a predictor of college enrollment and persistence in comparison to other predictors? 3.Which services, or combinations of services, have had the greatest impact in promoting college enrollment and persistence? RESEARCH QUESTIONS

HOW WE ANSWERED IT Descriptive analysis College enrollment by race/gender College persistence by race/gender Service data Logistic regression GEAR UP impact Linear regression Impact of instances and duration of services METHODS

GUK II  GUK II Cohort 1  HS graduating class of 2011  Complete EPAS data (7th-11 th )  Demographics  N=2,202  2,157 matched (45 deleted) Comparison  From non-GU KY schools with FRPL status of 50%+  HS graduating class of 2011  EPAS data (8 th, 10 th, 11 th )  Demographics COMPARISON GROUP METHODS

MATCHING COMPARISON GROUP Pool of 9,900 students Matching variables  Sex  Race  Caucasian  African American  Latino  8 th grade EXPLORE composite scores  Zero tolerance and exact matching  8 cases unable to match  Resulting in 2,149 GU and 2,149 non-GU METHODS

COLLEGE ENROLLMENT RQ 1: Are there notable differences in college enrollment and persistence between GEAR UP students and other students from low-income schools?

FINDINGS HS graduation and College Enrollment GUK II (n=2,149) Non-GU (n=2,149) #%#% Graduated from High School 1, %1, % Enrolled Full/Half Time in College % % College enrollment 2 yr. and 4-yr institutions

GUK II Group  N=935  Sex  547 female (58.5%)  388 male (41.5%)  Race  850 Caucasian (90.9%)  73 African American (7.8%)  5 Latino (0.5%) Comparison Group  N=766  Sex  444 female (57.9%)  322 male (42.0%)  Race  693 Caucasian (90.5%)  55 African American (7.2%)  11 Latino (1.4%) COLLEGE ENROLLMENT BY GENDER/RACE BREAKDOWN FINDINGS

FINDINGS: CE BY GENDER AND RACE* College Enrollment by Gender and by Race* GUK IINon-GU #%#% College Enrollment Total % % Female (n=1,079) Male (n=1,070) % 36.3% % 30.1% Black (n=162) Latino (n=32) White (n=1,939) % 15.6% 43.8% % 34.4% 35.7% More GUK II students enrolled in college than comparison group across categories, with the exception of Latino students More GUK II females enrolled in college by 9.6 percentage points More GUK II males enrolled in college by 6.2 percentage points More GUK II African American students enrolled in college by 11.1 percentage points More GUK II Caucasian students enrolled in college by 8.1 percentage points *Race categories are not comprehensive

FINDINGS: CE BY RACE* BY GENDER College Enrollment by Race* by Gender GUK IINon-GU #%#% College Enrollment Total % % Black (n=162) Female (n=82) Male (n=80) % 47.6% 42.5% % 36.6% 31.3% White (n=1,939) Female (n=981) Male (n=958) % 51.0% 36.5% % 41.5% 29.9% Disaggregated by race and gender, GUK II students within categories still enrolled in college at higher rates than that of the comparison group. More GUK II African American females enrolled by 11 percentage points More GUK II African American males enrolled by 11.2 percentage points More GUK II Caucasian females enrolled by 9.5 percentage points More GUK II male students by 6.6 percentage points

RACE/GENDER SUMMARY  Considerably more GUK II students enrolled in college than comparison group  Disaggregated by race, more GUK II Caucasian and African American students enrolled in college than comparison group  Disaggregated by race and gender, more GUK II African American male, African American female, Caucasian male, and Caucasian female students enrolled in college than comparison group FINDINGS

EPAS SUBGROUPS We wanted to know: If/how EPAS performance affected college enrollment If EPAS performance was a factor in the greater college enrollment of the GUK II students We discovered that was a LARGE query We narrowed the scope to math and reading (to parallel NCLB requirements at that time period)

FINDINGS: COLLEGE ENROLLMENT BY MATH SUBGROUP SubgroupEXPLOREPLANACTGUK IINon-GU #%#% Overall % % M1 yes 103 (n=139) 74.1% 130 (n=182) 71.4% M2 yes no 23 (n=40) 57.5% 21 (n=43) 48.8% M3 yesnoyes 11 (n=20) 55.0% 20 (n=32) 62.5% M4 yesno 43 (n=80) 53.8% 25 (n=57) 43.9% M5 no 612 (n=1629) 37.6% 448 (n=1583) 28.3% M6 no yes 41 (n=70) 58.6% 26 (n=51) 51.0% M7 noyes 45 (n=63) 71.4% 53 (n=93) 57.0% M8 noyesno 57 (n=103) 55.3% 43 (n=114) 37.7%

FINDINGS: COLLEGE ENROLLMENT BY READING SUBGROUP subgroupEXPLOREPLANACTGUK IINon-GU #%#% overall % % R1 yes 212 (n=327) 64.8% 227 (n=360) 63.1% R2 yes no 48 (n=78) 61.5% 55 (n=86) 64.0% R3 yesnoyes 42 (n=63) 66.7% 23 (n=43) 53.5% R4 yesno 53 (n=96) 55.2% 16 (n=63) 25.4% R5 no 363 (n=1154) 31.5% 237 (n=1094) 21.7% R6 no yes 37 (n=83) 44.6% 41 (n=88) 46.6% R7 noyes 77 (n=129)59.7%73 (n=154)47.4% R8 noyesno 103 (n=214) 48.1%94 (n=267)35.2%

FINDINGS: COLLEGE ENROLLMENT BY MATH AND READING SUBGROUP subgroupEXPLOREPLANACTGUK IINon-GU MRMRMR#%#% overall % % M&R1 YYYYYY 74 (n=98) 75.5%101 (n=133)75.9% M&R2 YNYNYN 4 (n=6) 66.7% 5 (n=9) 55.6% M&R3 NNNNNN 322 (n=1061) 30.3% 204 (n=1007) 20.3% M&R4 NYNYNY 57 (n=107) 53.3% 55 (n=109) 50.5%

EPAS MATH AND READING SUBGROUP SUMMARY  More GUK II students enrolled in college across most of the EPAS categories  Notably, there were more GUK II students who enrolled in college than comparison group, even though they did not meet benchmarks  Suggests a GEAR UP effect that goes beyond academic performance FINDINGS

COLLEGE PERSISTENCE RQ 1: Are there notable differences in college enrollment and persistence between GEAR UP students and other students from low-income schools?

PERSISTENCE DEFINED High school graduating class of 2011 who enrolled in college in the fall of 2011 who then continued into their second year of college in the fall of 2012 Half-time and full-time 2-year and 4-year institutions DEFINITION

FINDINGS: COLLEGE PERSISTENCE BY 2- & 4-YR INSTITUTIONS GUK IINon-GU # %#% College Persistence Total 726 (n=935) 77.6% 609 (n=766) 79.5% Two-Year 258 (n=369) 69.9% 127 (n=179) 70.9% Four-Year 468 (n=566) 82.7% 482 (n=587) 82.1%

FINDINGS: COLLEGE PERSISTENCE BY GENDER & RACE GUK IINon-GU #%#% CP Total 726 (n=935) 77.6% 609 (n=766) 79.5% Female Male 446 (n=547) 280 (n=388) 81.5% 72.2% 372 (n=444) 237 (n=322) 83.8% 73.6% Black Latino White 46 (n=73) 2 (n=5) 673 (n=850) 63.0% 40.0% 79.2% 40 (n=55) 10 (n=11) 554 (n=693) 72.7% 90.9% 79.9% Overall, total N of GUK II who enrolled in the second year of college from high school graduation was still higher, but the actual CP rate was slightly lower than comparison group. Non-GU female students persisted more by 2.3 percentage points Non-GU male students persisted more by 1.4 percentage points Non-GU African American students persisted more by 9.7 percentage points Non-GU Caucasian students persisted more by 0.7 percentage points.

GUK IINon-GU #%#% College Persistence Total 726 (n=935) 77.6%609 (n=766) 79.5% Black Female Male 46 (n=73) 26 (n=39) 20 (n=34) 63.0% 66.7% 58.8% 40 (n=55) 24 (n=30) 16 (n=25) 72.7% 80.0% 64.0% White Female Male 673 (n=850) 415 (n=500) 258 (n=350) 79.2% 83.0% 73.7% 554 (n=693) 341 (n=407) 213 (n=286) 79.9% 83.8% 74.5% Disaggregated by race and gender:  More non-GU African American female students persisted by 13.3 percentage points  More non-GU African American male students persisted by 5.2 percentage points  More non-GU Caucasian female students persisted by 0.8 percentage points  More non-GU Caucasian male students persisted by 0.8 percentage points FINDINGS: COLLEGE PERSISTENCE BY GENDER & RACE

COLLEGE PERSISTENCE SUMMARY GUK II students persisted into the second year of college at a slightly lower rate than comparison group Total N from high school graduation to second year of college was still greater than comparison group The loss was with African American students FINDINGS

GUK II AS A PREDICTOR FOR COLLEGE ENROLLMENT RQ 2: How does GEAR UP perform as a predictor of college enrollment and persistence in comparison to other predictors?

PREDICTOR DEFINED “Other” predictors were ACT benchmarks in English, math, reading, science; gender; race Did not/could not factor in all the EPAS work that GUK II actually implemented DEFINITION

Predictor variables: GEAR UP student status, attainment of the ACT English, math, reading, and science benchmarks, Male, Black, and Latino 68.4% correctly classified FINDINGS: LOGISTIC REGRESSION

GUK is a modestly successful predictor for college enrollment, better than ACT science and reading benchmarks, being African American, Latino, and male FINDINGS: GUK AS A PREDICTOR

GUK II AND COLLEGE ENROLLMENT RQ 3: which services, or combinations of services, have had the greatest impact in promoting college enrollment and persistence?

GUK II SERVICES Awareness programs focused on providing counseling/information for students and parents about the value of college, preparation for postsecondary education, and college admission requirements Rigor focused on ensuring all students have access to rigorous coursework, and that teachers and schools are equipped to close the achievement gap. Engagement focused on helping parents set high expectations for their children including planning for college education Access focused on educating parents and students about federal and state financial aid resources that make college affordable Support services focused on preventing students from failing by providing targeted academic services, or supplemental enrichment and developmental instruction for selected students DEFINITION

GUK II SERVICES DEFINED Data collection Service data available for grades 8-12 Service data available by school Data analysis Eliminated grade 8 Focused on the five, large service categories Generalized school numbers to the student-level DEFINITION

FINDINGS: GUK II HS SERVICE PROFILES GUK II High School GUK II Students College Enrolled % College Enrolled Service Instances Service Duration Instances / Student Duration / Student %434, %71, %312, %458, %92, %161, %16016, %344, %515, %373, %414, %273, %244, %121, %4311,

GUK II High School GUK II Students College Enrolled % College Enrolled Service Instances Service Duration Instances / Student Duration / Student %506, %637, %455, %101, %242, %532, %166, %275, %929, %71, %202, %376, %363, % % FINDINGS: GUK II HS SERVICE PROFILES

FINDINGS: “TYPICAL” GUK II STUDENTS ServiceNo CollegeCollege EnrolledPersistent Awareness Instances Duration2,2361,9431,942 Rigor Instances Duration1,0301,0231,018 Engagement Instances Duration Access Instances Duration Support Instances Duration Total Instances Duration4,6354,2024,196

REGRESSION MODELS # of service instances and duration of services by student, predicting CE had a multiple R 2 of 0.45 Inconclusive ELABORATE LINEAR REGRESSION # of service instances and duration of services by the five service areas, predicting CE had a multiple R 2 of 0.71 High degree of correlation Inconclusive LINEAR REGRESSION FINDINGS

ANALYSIS OF SERVICES SUMMARY Exploratory analyses resulted in inconclusive findings (as expected) Services targeted at-risk populations Awareness and Support had the highest instances per student average Model established for further analysis FINDINGS

More GUK students went to college than the comparison group More African American GUK II students went to college than the comparison group Extensive numbers and duration of services were provided to thousands of students with the goal of college-going in mind Further analysis is needed CONCLUSION GUK IS MAKING A DIFFERENCE!

QUESTIONS? COMMENTS? Judy Kim