TECHHIRE GRANTS MANAGEMENT PLENARY SESSION: EVALUATION OVERVIEW & UPDATE

TECHHIRE GRANTS MANAGEMENT PLENARY SESSION: EVALUATION UPDATE Speakers Molly Irwin, Chief Evaluation Officer, U.S. Department of Labor Christina Yancey, Evaluation Team Manager, U.S. Department of Labor

Why the Focus on Evidence Increased emphasis on evidence-based policymaking at all levels of government and in the private sector, for example: The President’s FY18 budget, “An effective and efficient Federal government requires evidence—evidence about where needs are greatest, what works and what does not work, where and how programs could be improved...” The Commission on Evidence-Based Policymaking Evidence reviews and tiered-evidence funding initiatives (including TechHire!) Private philanthropic organizations calling for evaluation and the use of evidence Increased emphasis on learning and program improvement on the ground

Why the Focus on Evidence Plan Use Evaluation to Improve Implement Evaluate

What we know now Grantees based some programs on promising practices, such as direct employer relationships, sector-based strategies, and career pathways. Grantees are also exploring new and innovative practices, such as IT bootcamps and other rapid, rigorous training strategies. TechHire is building new evidence across all of these strategies, partnership approaches, and target population engagement.

How the TechHire evaluation will build evidence It will answer these questions: How are the programs implemented? What factors influence implementation? What is the impact on employment, earnings, and other outcomes? How do effects vary by program and participant characteristics? By implementing the following design: Implementation study involving all TechHire programs Impact evaluation Random assignment with a subset of grantees Quasi-experiment with all TechHire programs

Where you fit in As partners in building evidence about workforce programs for youth so we can continue to improve the programs and outcomes for the youth we serve.

Overview of Evaluation Activities Type of Information Target Date All Grantees DOL WIPS data, QPRs Quarterly Grantee web survey Sept 2018 Partner Information Form July 2018 Partner web survey Telephone interviews (Grantees and Partners) Feb 2019 Only RCT-Grantees Random assignment Beginning Jan 2018 Grantee site visits April 2018, Oct 2019 Participant surveys (6 & 18 months follow-up) July 2018 thru June 2020

Grantee Responsibilities Ongoing: Regularly update WIOA WIPS In 2018: Complete a grantee survey on the web Program context, partnerships, training strategies, supportive services, implementation successes and challenges, etc. In 2019: Participate in a telephone discussion to follow-up on survey responses In 2019: Provide contact information for key partners so the evaluation team can contact them for a survey

Evaluation Team Responsibilities Keep you informed and provide updates on the research progress. Provide technical support to ensure research is minimally burdensome. Share research findings in a wide array of formats (brief, key takeaways + technical reports) as soon as we can. Be available.

Upcoming Next Steps Complete site visits to short list of grantees for RCT (late July 2017) RCT sites announced (late July/August 2017)

Contacts Molly Irwin, Ph.D., DOL Chief Evaluation Officer Irwin.Molly.E.@dol.gov Christina Yancey, Ph.D., DOL Evaluation Lead Yancey.Christina.L@dol.gov Joseph Gasper, Ph.D., Westat, Project Director, JosephGasper@westat.com

Thank you!