PromiseNet 2017 – Kalamazoo, MI Taking Stock, Moving Forward: Learning from Promise Research Dr. Michelle Miller-Adams W.E. Upjohn Institute Grand Valley State University Dr. Jennifer Iriti Evaluation for Learning Group University of Pittsburgh - LRDC PromiseNet 2017 – Kalamazoo, MI

Presentation outline Taking Stock – The College Promise Landscape Promise Program Database Research Overview Locally based programs – “5 in 5” Lessons from three Communities Higher ed-based programs Statewide programs – Tennessee Promise Update Moving Forward Breakout Session

The College Promise Landscape Statewide programs Local Promise programs Higher ed-based Tennessee Promise Oregon Promise Excelsior Scholarship (New York) Kalamazoo Promise Pittsburgh Promise El Dorado Promise Say Yes to Education Ventura College Promise Santa Barbara City College Promise Milwaukee Area Technical College Promise

About 90 programs nationwide included within Upjohn Institute definition.

Locally based programs: conceptual framework Three domains for outcomes – pre-K through 12 impacts, post-secondary impacts, community-level impacts Pipeline –progress along the way is linked, success in later stages builds on success in earlier stages Underscores need for student support and community alignment along the way

High-level research summary The next four slides summarize the findings of rigorous research studies that have investigated the various outcomes of Promise programs. Only those studies that use a methodology that can assign causation are included here. Other information about program outcomes may come from administrative data, descriptive studies, or dashboards compiled by Promise programs or outside evaluators; theseare not included here. Green cells indicate that a topic has been studied. Light gray cells indicate that no causal research has taken place in this area. Dark gray cells signify quasi- experimental methodologies with some limitations. Check marks signify that positive and statistically significant effects have been found. Asterisks indicate mixed results. Relevant studies are listed below the table. These can be accessed at this link. If you have new or updated studies that should be reviewed and included in this summary, please contact the authors.

Locally based programs: K-12 impact summary Impact Area Kalamazoo El Dorado Pittsburgh New Haven Say Yes Multi-site Aspirations & college culture ✔ School district enrollment Achievement K-12 impacts Outcomes down left hand side– aspirations/college culture, enrollment in district, achievement Across the top, specific Promise programs that were part of the Lumina-funded research collaboration and a cross-site analysis done by LeGower and Walsh Gray not explored via rigorous quasi-experimental or systematic studies Say Yes– only for city-wide Green….positive finding, check mark significant; Gray - not explored Lots of nuance– studies explore race and class with interesting findings. See: Ash and Ritter (2014) Bartik, Hershbein, and Lachowska (2015) Iriti, Bickel, Meredith, Walker, and Nelson (2009) Bartik and Lachowska (2012) LeGower and Walsh (2014) Bifulco, Rubenstein, Sohn, and Murchie (2016) Miron, Jones, and Kelaher-Young (2011) Gonzalez et. al (2014) Ritter and Ash (2016)

Locally based programs: Post-secondary impact summary Impact Area Kalamazoo El Dorado Pittsburgh New Haven Say Yes Post-secondary enrollment ✔ * Matching Persistence Degree completion See: Bartik, Hershbein, and Lachowska (2015) Page & Iriti (2016) Daugherty and Gonzalez (2016) Page, Iriti, Lowry and Anthony (forthcoming) Bifulco (2016) Ritter and Ash (2016)

Locally based programs: Community/economic development impacts Impact Area Kalamazoo El Dorado Say Yes Multi-site Housing values ✔ Migration Impact Rate of return School district enrollment Very under-studied in terms of impacts. Odd, too, considering these are the goals that help set place-based programs apart! Decline in out-migration from areas around Promise programs Hard to study b/c of lack of ready data sets and multiple causes affecting outcomes Explain that enrollment effect is both a K-12 impact and a community-level impact See: Bartik, Hershbein, and Lachowska (2013) Bifulco, Rubenstein, Sohn, and Murchie (2016) Bartik & Sotherland (2015) LeGower & Walsh (2014)

State of local Promise research Outcome Area Overall Findings K-12 Positive where data availability and program criteria amenable Post-secondary Community & economic development Positive but significantly under-researched Variability in design hypothesized to influence impacts but not carefully empirically studied yet.

Take a snapshot Maintain contact Expect the unexpected Create a college going culture Partner up!

LOVE – LISTEN – LOOK – LAUGH – LEAD – be bold, respond, improve every child, every contribution LISTEN – to the scholars, the schools, the community LOOK – at the data bravely LAUGH – don’t take yourself too seriously - enjoy LEAD – be bold, respond, improve

5 in 5: Focus squarely on kids Partner with IHE from beginning Coordinate pre-K – 12 supports Promise what you can deliver Keep it simple

Higher ed-based programs: possible outcomes Outcome Area Possible Outcomes K-12 Higher ed geographic sending area response to increased access; Pathways to that specific institution created Post-secondary Increased access and persistence; Some based on message more so than actual new dollars added Community & economic development Increase in adults with a credential living in the local community

Statewide programs: possible outcomes Outcome Area Possible Outcomes K-12 Spillover effects on college-going culture and readiness, high-school graduation rates Post-secondary Increased enrollment, persistence and completion; transfer rate from 2-year to 4-year schools Community & economic development Better alignment with state workforce needs; attraction of new businesses, creation of new jobs

Keeping the Promise: Early Outcomes of Tennessee’s Tuition-Free College Initiative Emily House Chief Research Officer PromiseNet conference Kalamazoo, Michigan November 6, 2017

Drive to 55 Governor Bill Haslam’s attainment goal By 2025, 55 percent of working age adults in Tennessee with have a postsecondary credential. Goal based on findings of A Decade Behind report (Carnevale and Smith, 2012) Currently: ~40 percent attainment 34.7 percent have AA or higher (ACS) ~5 percent have postsecondary certificates

Economic development Pitched as a workforce development as well an educational attainment initiative. Resonates with (R majority) state legislature Business expansion, recruitment to TN Increase in-state employment, workforce participation?

Signature program: Tennessee Promise Last dollar scholarship for recent high school grads to attend a community or technical college in Tennessee 13 community colleges, 27 TCATs, some universities Students paired with a mentor Adult in community Low-touch mentoring (texting, “task master”) Eight hours of community service each term File/renew FAFSA each year

Eligibility Requirements (2017-18) Jan 16 FAFSA deadline Jan-Feb First mentor meeting Mar-April Second July 1 Community Service Fall semester Third mentor meeting (optional) Dates have shifted over the past few years, but this is the general idea. Nov 1 Application Closes

Eligibility Requirements Once enrolled: Cannot stop out or drop out Enroll full time Refile FAFSA each year Complete 8 hours of community service each term “One and done” – no regain provision.

Enrollment Where TN Promise students enroll: 85 percent community colleges 13 percent technical colleges (TCATs) 2 percent Promise-eligible 4-years Each year, ~6,500 students fulfill all requirements but do not enroll: Out of state/privates Public universities Do not enroll. WHY?

Characteristics of program participants   FTF out of HS at CCs Fall 2014 TN Promise CC Students Fall 2015 Female 56% Average ACT 18.7 19.1 Average HS GPA 3.04 3.05 Race/Ethnicity White 70% 74% Black 19% 14% Hispanic 5% 4% Financial Aid TELS 48% 58% Any Pell 53% Pell covers T&F 45% Full Pell 40% 34% Consistent for all cohorts of TNP students. Work to do in terms of EQUITY

Enrollment FTF Fall 2014 Fall 2015 Fall 2016 % Change 2014-15 2014-16   FTF Fall 2014 Fall 2015 Fall 2016 % Change 2014-15 2014-16 TCATs 8,691 10,432 11,500 +20.1% +32.2% Community Colleges 17,379 22,190 20,770 +27.7% +19.5% Locally Governed Institutions 11,983 11,309 12,171 -5.6% +1.9% University of Tennessee 7,977 7,541 7,804 -5.5% +2.1% Total 46,030 51,472 52,245 +11.8% +13.5% These numbers suggest that many first-time freshmen may have substituted a community college or TCAT for a four-year institution. Overall, there was an 11.8% increase in first-time freshmen enrollment at Tennessee public institutions from Fall 2014 to Fall 2015. Changes in FTF enrollment in Fall 2016 and Fall 2017 have leveled off: many students continue to enroll in less-than-four-year institutions at high rates, but the proportion of students enrolled in each institution type remains consistent since Fall 2015. These numbers likely represent some new entrants to higher education. The decrease at four-years is not equal to the increase in CCs and TCATs, so we must have some new people in there too.

TN Promise Retention – Cohort 1 Fall 2015-Spring 2016 retention: 80.6 percent 78.5 percent at CCs; 94.7 percent at TCATs Comparable to previous years Fall 2015-Fall 2016 retention: 63 percent 58 percent at CCs; 83 percent at TCATs Retention of non-Promise students: 42 percent As of Fall 2017: 56.2 percent “success” 14.5 percent graduated 39 percent still enrolled 2.7 percent transferred

College-going rate

FAFSA filing & loan take up Tennessee is #1 state for FAFSA filing since inception of program. 73.5 percent of senior class (2017) filed a FAFSA. 13 percentage point increase since 2014. In Fall 2015, 17 percent fewer students originated federal loans At some community colleges, decrease as much as 25 percent. Decline has continued into 2016. Loan AMOUNT decreased by 12 percent

Questions? Thank you! emily.house@tn.gov

“Moving Forward” breakout session Using worksheet on your table, jot down the 1 or 2 research questions that you think most need to be answered What research or evaluation insight would help you most in your work with a Promise program? For example, what do you need to know about the implementation of your Promise program or the impact it is having? Share these questions with others at your table and see if there are areas of overlap Choose 1 question to share with the full group in a brief report-out Organizers will collect your comments/questions and share them with Promise Research Consortium members.

For more on Promise research… Breakout Session #1 Promise Program Design Lab Breakout Session #2 Show me the Evidence: Policy and Practice from Research Promise Program Evaluation Lab

For more information… Michelle Miller-Adams – miller-adams@upjohn.org Jennifer Iriti – iriti@pitt.edu Promise Programs Database – www.upjohn.org/promise/database Research Bibliography – http://www.upjohn.org/promise-research- bibliography-–-outcome-area