Interpreting the Evidence around College Access Strategies Neal D. Finkelstein Society for Research on Educational Effectiveness March 4,

Background IES Practice Guide “Helping Students Navigate the Path to College: What High Schools Can Do” September

Background What Works Clearinghouse standards were used to determine levels of evidence – strong, moderate and low. 3

Panel Recommendations 4 1) Offer courses and curricula that prepare students for college- level work, and ensure that students understand what constitutes a college-ready curriculum by 9 th grade. Level of Evidence: Low

Panel Recommendations 5 2) Utilize assessment measures throughout high school so that students are aware of how prepared they are for college, and assist them in overcoming deficiencies as they are identified. Level of Evidence: Low

Panel Recommendations 6 3)Surround students with adults and peers who build and support their college going aspirations. Level of Evidence: Low

Panel Recommendations 7 4) Engage and assist students in completing critical steps for college entry. Level of Evidence: Moderate

Panel Recommendations 8 5) Increase families’ financial awareness, and help students apply for financial aid. Level of Evidence: Moderate

Research Scan Research scan over the period revealed more than 500 studies 99 studies had causal designs 16 studies of 10 programs met WWC standards, with or without reservations 9

Research Scan The summary of evidence is methodical, but not without complexity. For example, 13 studies of programs potentially met WWC standards, but provided insufficient information to establish baseline equivalence. 10

Research Scan No easy wins on the strict mapping of evidence to recommendations; the benefit to the practitioner lies in panelists working through the subtleties. 11

Research Scan “College access programs consist of multiple components that address a variety of steps students must take to prepare for and enter college. The bundling of multiple practices within an access program makes it difficult to assess the effectiveness of a single practice.” (Page 49) 12

Research Scan Positive effects on one or more relevant outcomes: 5 programs. No detectable effects on relevant outcomes: 5 programs. 13

Research Scan 14 Positive EffectsNo detectable Effects FAFSA Experiment (2009)EXCEL (2007) Career Beginnings(1990)Middle College High Schools (1998) Talent Search (2006)Career Academies ( ) Sponsor-a-Scholar (1998)Quantum Opportunity Program (QOP) Talent Development High School (2005) Upward Bound ( )

Observations The glass is ¼ full – some evidence with some caveats. The institutional context of program implementation sites challenges generalizability. The research standards by the WWC set the bar for new studies – essential for the long term. 15

Observations Assertion: College access programs implement as bundles of interlocking strategies, and schools operate as settings for interlocking strategies. 16

Challenge Design a study, which meets rigorous standards, that can differentiate the effectiveness of individual strategies, within the context of the real-time school setting. 17

Interpreting the Evidence around College Access Strategies Neal D. Finkelstein Society for Research on Educational Effectiveness March 4,