1. Early Findings SREE Spring 2018 Michael J. Weiss
Detroit Promise Path
Early Findings SREE Spring 2018 Michael J. Weiss
This presentation is based on technical assistance and research conducted by my MDRC colleagues, primarily Alyssa Ratledge and Colleen Sommo. They couldn’t be here today, so you’re stuck with me. I apologize in advance for my more limited knowledge of this work.

2. What are Promise Programs?
Offer the “Promise” of something (typically tuition/fee free college for local students)
Over 200 nationwide
Vary in many ways (how aid is disbursed, eligibility criteria, type of award)
Ways in which they vary:
How aid is disbursed (e.g., first dollar vs. last dollar)
Eligibility (city HS, county HS, single HS, etc.)
Type of award (financial, admission guarantee, college savings account, etc.)
Source:

3. What is Detroit Promise?
Last-dollar scholarships to Detroit resident recent high school graduates attending one of five Detroit area community colleges
Operates out of the Detroit Regional Chamber
“Requires” full-time enrollment
5 Detroit Colleges:
Henry Ford College
Macomb Community College
Oakland Community College
Schoolcraft College
Wayne County Community College
Goal: Improve retention and graduation rates for Detroit Promise students

4. How this project came about
Detroit Regional Chamber disappointed with dropout rates among Detroit Promise students
Reached out to MDRC to work together to enhance their program from one focused on access to one achieving success
Together, MDRC and the Chamber developed the Detroit Promise Path, building off of 15+ years of experience in postsecondary research and TA

5. Last dollar scholarship + Full-time “requirement” are what the Detroit Promise program brought to the table and what the control group in the experiment are offered.
MDRC worked with the Detroit Chamber to add on the following additional components, to make the Detroit Promise Path. For each component, I’ll discuss some research that led to the inclusion of this component and describe DPP’s instantiation of it.
Required academic coaching. The inclusion of academic coaching as an important component to student success has a long history in community college research, but I’ll focusing on the multiple experimental evaluations of its effectiveness.
In 2002, MDRC launched the Opening Doors demonstration, which included a RA evaluation of a program in two Ohio colleges that’s linchpin was an advisor with whom students were expected to meet at least once a month, for two semesters. Since that evaluation, numerous RCTs have demonstrated the potential effectiveness of coaching or advising, including, but not limited to RCTs of InsideTrak, STAR, CUNY’s ASAP, Stay the Course, and BottomLine.
Based on this, each college enrolling Detroit Promise students now has a part- or full-time campus coach dedicated just to Detroit Promise students. The coach serves a small caseload of students, approximately 80 students during the first year operating the program and growing to students since. Coaches assist students with academic and personal support topics and undertake an intrusive advising model, in which they actively reach out to students through multiple modes of contact – in-person meetings, phone calls, s, and texts. The coaches have office space on each college campus and can refer students to other on-campus services, such as tutoring. The coaches use a Management Information System (MIS) to collect data about students’ participation and monitor students’ success, identifying areas for improvement and working with students to ensure their retention and success. All coaches meet weekly with the Detroit Promise Coordinator at the Chamber to review data and address student issues as they occur.
Monthly incentives. The use of incentives to induce desired behavior has a long history in economics. MDRC first used financial incentives in a community college setting as part of the enhanced advising program in Ohio described earlier – students received $150 per semester as “a way of ensuring that they stay in contact with their advisor.” Simultaneously, financial incentives were a critical part of our work on performance-based scholarships, CUNY’s ASAP, and the RCTs of many other economists.
In DPP, enrolled students who meet with their coach as directed receive a monthly financial incentive of $50 to offset expenses not covered by financial aid – an incentive most similar to CUNY’s ASAP, although of less than half the value.
Summer Engagement. While “regular” Detroit Promise offers students financial support to take summer courses, take-up rates are VERY low. Multiple MDRC studies, starting with our first RCT of Kingsborough Community College’s LC’s, have shown that interventions can induce summer enrollment and help students stay on track to timely graduation by accumulating more credits (we’ve also seen effects on summer enrollment as part of PBS, CUNY’s ASAP, and EASE). DPP’s coaches encourage summer enrollment and remind students of the possibility of Detroit Promise financial coverage for these courses.
Behavioral messaging. Drawing from behavioral economics work being undertaken by MDRC and others in the field, we have included behavioral messaging in the and text communications sent to students. These messages include how to participate in program components and maximize financial support. Coaches often create their own customized messages that they personalize to subpopulations. For example – a la the work of Lindsay Page and Ben Castleman, when FAFSA opens they’ll text everyone to fill it out, later they’ll message only those who have not yet completed their FAFSA.
Program management. The program is managed by the Detroit Promise Coordinator, who oversees the coaches and monitor students’ participation and outcomes through regular data reports. These data reports provide a mechanism for continuous program improvement through central oversight. MDRC created a management information system (MIS) to monitor student progress, track appointments, send reminders via and text about program requirements, track financial incentives, and automatically create summary dashboards about students’ activities/participation for staff.
Finally, one lesson we’ve learned from our time conducting RCTs is that short-term interventions tend to have in program effects only. Consequently, as we saw when evaluating CUNY’s ASAP, offering services for a long duration can be critical – DPP is a three-year program.

6. RCT of Detroit Promise Path
Colleges: 5 community colleges
Cohorts: 3 cohorts (Fall 2016, Winter and Fall 2017)
Student eligibility criteria:
Detroit resident
Recent high school graduates
Sample size: About 1200 students
Early results based on first cohort (n=625)
Contrast: Promise vs. Promise Path
Promise: scholarship plus full-time “requirement”
Promise Path: Promise plus the add-ons I described on the last slide.

7. Sample Characteristics
Detroit Promise Path Evaluation Sample
Average age 18
Neither parent has a college degree 75
Identify as African American 80
Planned to attend college after graduating from HS 95

8. Participation Rates
> 95 percent of students responded to outreach from coaches
About 70 percent of enrolled students meet with their coaches twice per month

9. Promise Path Boosts Enrollment
Note significant “summer melt” – something that we’re hoping will be better addressed for the next fall cohort.
Note “soft” full-time requirement.
Note large impact on Semester 2 enrollment – partially a result of impact on getting 1st semester enrollees to return and partially a result of getting those who didn’t enroll in semester 1 to matriculate in semester 2
Large effects on FT enrollment in semester 2.
Statistical significance levels are indicated as *** = 1 percent; ** = 5 percent; * = 10 percent.

10. Next Steps Track full sample
Continue to track students’ academic outcomes for three years

11. MDRC’s College Promise Success Initiative
Provides technical assistance to Promise and free college programs interested in promoting success in college by implementing evidence-based student support practices.
The initiative will disseminate best practices and actionable tools to the over 200 College Promise and free college programs that exist across the country and provide in-depth technical assistance to a select number of programs.