1. PromiseNet 2017 – Kalamazoo, MI
Taking Stock, Moving Forward:
Learning from Promise Research
Dr. Michelle Miller-Adams
W.E. Upjohn Institute
Grand Valley State University
Dr. Jennifer Iriti
Evaluation for Learning Group
University of Pittsburgh - LRDC
PromiseNet 2017 – Kalamazoo, MI

2. Presentation outline Taking Stock – The College Promise Landscape
Promise Program Database
Research Overview
Locally based programs – “5 in 5” Lessons from three Communities
Higher ed-based programs
Statewide programs – Tennessee Promise Update
Moving Forward Breakout Session

3. The College Promise Landscape
Statewide programs
Local Promise programs
Higher ed-based
Tennessee Promise
Oregon Promise
Excelsior Scholarship (New York)
Kalamazoo Promise
Pittsburgh Promise
El Dorado Promise
Say Yes to Education
Ventura College Promise
Santa Barbara City College Promise
Milwaukee Area Technical College Promise

4. About 90 programs nationwide included within Upjohn Institute definition.

6. Locally based programs: conceptual framework
Three domains for outcomes – pre-K through 12 impacts, post-secondary impacts, community-level impacts
Pipeline –progress along the way is linked, success in later stages builds on success in earlier stages
Underscores need for student support and community alignment along the way

7. High-level research summary
The next four slides summarize the findings of rigorous research studies that have investigated the various outcomes of Promise programs.
Only those studies that use a methodology that can assign causation are included here. Other information about program outcomes may come from administrative data, descriptive studies, or dashboards compiled by Promise programs or outside evaluators; theseare not included here.
Green cells indicate that a topic has been studied. Light gray cells indicate that no causal research has taken place in this area. Dark gray cells signify quasi- experimental methodologies with some limitations. Check marks signify that positive and statistically significant effects have been found. Asterisks indicate mixed results.
Relevant studies are listed below the table. These can be accessed at this link.
If you have new or updated studies that should be reviewed and included in this summary, please contact the authors.

8. Locally based programs: K-12 impact summary
Impact Area
Kalamazoo
El Dorado
Pittsburgh
New Haven
Say Yes
Multi-site
Aspirations & college culture ✔
School district enrollment
Achievement
K-12 impacts
Outcomes down left hand side– aspirations/college culture, enrollment in district, achievement
Across the top, specific Promise programs that were part of the Lumina-funded research collaboration and a cross-site analysis done by LeGower and Walsh
Gray not explored via rigorous quasi-experimental or systematic studies
Say Yes– only for city-wide
Green….positive finding, check mark significant; Gray - not explored
Lots of nuance– studies explore race and class with interesting findings.
See:
Ash and Ritter (2014)
Bartik, Hershbein, and Lachowska (2015)
Iriti, Bickel, Meredith, Walker, and Nelson (2009)
Bartik and Lachowska (2012)
LeGower and Walsh (2014)
Bifulco, Rubenstein, Sohn, and Murchie (2016)
Miron, Jones, and Kelaher-Young (2011)
Gonzalez et. al (2014)
Ritter and Ash (2016)

9. Locally based programs: Post-secondary impact summary
Impact Area
Kalamazoo
El Dorado
Pittsburgh
New Haven
Say Yes
Post-secondary enrollment ✔ *
Matching
Persistence
Degree completion
See:
Bartik, Hershbein, and Lachowska (2015)
Page & Iriti (2016)
Daugherty and Gonzalez (2016)
Page, Iriti, Lowry and Anthony (forthcoming)
Bifulco (2016)
Ritter and Ash (2016)

10. Locally based programs: Community/economic development impacts
Impact Area
Kalamazoo
El Dorado
Say Yes
Multi-site
Housing values ✔
Migration
Impact
Rate of return
School district enrollment
Very under-studied in terms of impacts. Odd, too, considering these are the goals that help set place-based programs apart!
Decline in out-migration from areas around Promise programs
Hard to study b/c of lack of ready data sets and multiple causes affecting outcomes
Explain that enrollment effect is both a K-12 impact and a community-level impact
See:
Bartik, Hershbein, and Lachowska (2013)
Bifulco, Rubenstein, Sohn, and Murchie (2016)
Bartik & Sotherland (2015)
LeGower & Walsh (2014)

11. State of local Promise research
Outcome Area
Overall Findings
K-12
Positive where data availability and program criteria amenable
Post-secondary
Community & economic development
Positive but significantly under-researched
Variability in design hypothesized to influence impacts but not carefully empirically studied yet.

12. Take a snapshot
Maintain contact
Expect the unexpected
Create a college going culture
Partner up!

13. LOVE – LISTEN – LOOK – LAUGH – LEAD – be bold, respond, improve
every child, every contribution
LISTEN –
to the scholars, the schools, the community
LOOK –
at the data bravely
LAUGH –
don’t take yourself too seriously - enjoy
LEAD –
be bold, respond, improve

15. 5 in 5: Focus squarely on kids Partner with IHE from beginning
Coordinate pre-K – 12 supports
Promise what you can deliver
Keep it simple

16. Higher ed-based programs: possible outcomes
Outcome Area
Possible Outcomes
K-12
Higher ed geographic sending area response to increased access; Pathways to that specific institution created
Post-secondary
Increased access and persistence; Some based on message more so than actual new dollars added
Community & economic development
Increase in adults with a credential living in the local community

17. Statewide programs: possible outcomes
Outcome Area
Possible Outcomes
K-12
Spillover effects on college-going culture and readiness, high-school graduation rates
Post-secondary
Increased enrollment, persistence and completion; transfer rate from 2-year to 4-year schools
Community & economic development
Better alignment with state workforce needs; attraction of new businesses, creation of new jobs

18. Keeping the Promise: Early Outcomes of Tennessee’s Tuition-Free College Initiative
Emily House
Chief Research Officer
PromiseNet conference
Kalamazoo, Michigan
November 6, 2017

19. Drive to 55 Governor Bill Haslam’s attainment goal
By 2025, 55 percent of working age adults in Tennessee with have a postsecondary credential.
Goal based on findings of A Decade Behind report (Carnevale and Smith, 2012)
Currently: ~40 percent attainment
34.7 percent have AA or higher (ACS)
~5 percent have postsecondary certificates

20. Economic development
Pitched as a workforce development as well an educational attainment initiative.
Resonates with (R majority) state legislature
Business expansion, recruitment to TN
Increase in-state employment, workforce participation?

21. Signature program: Tennessee Promise
Last dollar scholarship for recent high school grads to attend a community or technical college in Tennessee
13 community colleges, 27 TCATs, some universities
Students paired with a mentor
Adult in community
Low-touch mentoring (texting, “task master”)
Eight hours of community service each term
File/renew FAFSA each year

22. Eligibility Requirements (2017-18)
Jan 16
FAFSA deadline
Jan-Feb
First
mentor
meeting
Mar-April
Second
July 1
Community
Service
Fall semester Third mentor meeting (optional)
Dates have shifted over the past few years, but this is the general idea.
Nov 1
Application
Closes

23. Eligibility Requirements
Once enrolled:
Cannot stop out or drop out
Enroll full time
Refile FAFSA each year
Complete 8 hours of community service each term
“One and done” – no regain provision.

24. Enrollment Where TN Promise students enroll:
85 percent community colleges
13 percent technical colleges (TCATs)
2 percent Promise-eligible 4-years
Each year, ~6,500 students fulfill all requirements but do not enroll:
Out of state/privates
Public universities
Do not enroll. WHY?

25. Characteristics of program participants
FTF out of HS at CCs
Fall 2014
TN Promise CC Students
Fall 2015
Female
56%
Average ACT
18.7
19.1
Average HS GPA
3.04
3.05
Race/Ethnicity
White
70%
74%
Black
19%
14%
Hispanic 5% 4%
Financial Aid
TELS
48%
58%
Any Pell
53%
Pell covers T&F
45%
Full Pell
40%
34%
Consistent for all cohorts of TNP students. Work to do in terms of EQUITY

26. Enrollment FTF Fall 2014 Fall 2015 Fall 2016 % Change 2014-15 2014-16
FTF
Fall 2014
Fall 2015
Fall 2016
% Change
TCATs
8,691
10,432
11,500
+20.1%
+32.2%
Community Colleges
17,379
22,190
20,770
+27.7%
+19.5%
Locally Governed Institutions
11,983
11,309
12,171
-5.6%
+1.9%
University of Tennessee
7,977
7,541
7,804
-5.5%
+2.1%
Total
46,030
51,472
52,245
+11.8%
+13.5%
These numbers suggest that many first-time freshmen may have substituted a community college or TCAT for a four-year institution.
Overall, there was an 11.8% increase in first-time freshmen enrollment at Tennessee public institutions from Fall 2014 to Fall 2015.
Changes in FTF enrollment in Fall 2016 and Fall 2017 have leveled off: many students continue to enroll in less-than-four-year institutions at high rates, but the proportion of students enrolled in each institution type remains consistent since Fall 2015.
These numbers likely represent some new entrants to higher education. The decrease at four-years is not equal to the increase in CCs and TCATs, so we must have some new people in there too.

27. TN Promise Retention – Cohort 1
Fall 2015-Spring 2016 retention: 80.6 percent
78.5 percent at CCs; 94.7 percent at TCATs
Comparable to previous years
Fall 2015-Fall 2016 retention: 63 percent
58 percent at CCs; 83 percent at TCATs
Retention of non-Promise students: 42 percent
As of Fall 2017: 56.2 percent “success”
14.5 percent graduated
39 percent still enrolled
2.7 percent transferred

28. College-going rate

29. FAFSA filing & loan take up
Tennessee is #1 state for FAFSA filing since inception of program.
73.5 percent of senior class (2017) filed a FAFSA.
13 percentage point increase since 2014.
In Fall 2015, 17 percent fewer students originated federal loans
At some community colleges, decrease as much as 25 percent.
Decline has continued into 2016.
Loan AMOUNT decreased by 12 percent

30. Questions? Thank you!

31. “Moving Forward” breakout session
Using worksheet on your table, jot down the 1 or 2 research questions that you think most need to be answered
What research or evaluation insight would help you most in your work with a Promise program? For example, what do you need to know about the implementation of your Promise program or the impact it is having?
Share these questions with others at your table and see if there are areas of overlap
Choose 1 question to share with the full group in a brief report-out
Organizers will collect your comments/questions and share them with Promise Research Consortium members.

32. For more on Promise research…
Breakout Session #1
Promise Program Design Lab
Breakout Session #2
Show me the Evidence: Policy and Practice from Research
Promise Program Evaluation Lab

33. For more information… Michelle Miller-Adams – miller-adams@upjohn.org
Jennifer Iriti –
Promise Programs Database –
Research Bibliography – bibliography-–-outcome-area