1. Youth CareerConnect Evaluation:
Early implementation and what’s to come
Presentation at the YCC Annual Grantee Meeting
Denver, Colorado
July 19-21, 2016
Jeanne Bellotti • Nan Maxwell • Peter Schochet

2. Thank you!!!

3. Outline of presentation
Evaluation overview
Early implementation findings
Future evaluation activities

4. Evaluation overview

5. YCC Evaluation: Three components
Implementation study
Lottery-based impact study
Comparison group impact study

6. Participation in impact study
Data collection
Grantee/school district
Lottery-based impact study
Follow-up survey
Kentucky Educational Development Corporation Manufacturing Renaissance Metropolitan School District of Pike Township
Administrative data (school records)
Kentucky Educational Development Corporation Los Angeles Unified School District
Manufacturing Renaissance Metropolitan School District of Pike Township
Comparison group design
Anson County Schools Brockton School District Board of Education, Buffalo, New York Chicago Public Schools Galveston Independent School District Laurens County School District 55 Laurens County School District 56 Los Angeles Unified School District Marlborough School District Metropolitan School District of Pike Township New York City Department of Education Prince Georges County School District Pulaski County School District Santa Cruz Valley Unified School District Springfield School District Toledo Public Schools Westside Community Schools

7. Participation in implementation study
Data collection
Grantee
QPR narratives-PTS-Grantee survey
ALL*
Site visits
Board of Education, Buffalo, New York Jobs for the Future, Inc. Kentucky Educational Development Corporation Laurens County School District 56 Los Angeles Unified School District Manufacturing Renaissance Metropolitan School District of Pike Township New York City Department of Education Pima County Toledo Public Schools
Baseline information forms
Kentucky Educational Development Corporation Manufacturing Renaissance Metropolitan School District of Pike Township Laurens County School District 56 Los Angeles Unified School District Toledo Public School Pima County

8. Implementation study: research questions
What types of students does YCC serve?
What program components are being implemented?
What distinguishes YCC from other programs?
What challenges do grantees face in implementing YCC, and how do they overcome those challenges?
How do grantees plan to sustain the program beyond the grant period?

9. Early implementation study: data sources

10. Early implementation findings

11. How do YCC programs differ from others?

12. Are students participating?
YCC students have attendance rates slightly higher than national averages
Relatively few YCC students left the program

13. What does professional development look like?
Most frequent activities
Project-based learning
Incorporating an industry focus into the curriculum
Building industry-specific skills and competencies
Collaborating or establishing communities of practice
Challenges
Inability to compel teachers to participate
Difficult to align with other professional development activities

14. What are early YCC program successes?
Recruiting and enrolling a diverse group of students
Establishing community partnerships
Offering activities and services in all key program components
Expanding access to activities, services and supports

15. What challenges did grantees face?
Services that required coordination were implemented slowly
Limited staff capacity impeded some activities and services
The rigor of YCC courses left some students struggling

16. Future evaluation activities

17. Additional research questions Additional data collection
Implementation study
Additional research questions
Additional data collection
How do grantees plan to sustain the program after the grant ends?
What implementation practices appear promising for scaling and replication?
2017 grantee survey with all 24 grantees
Final round of site visits to 10 grantees (winter/spring 2018)
PTS data through end of 2018

18. Lottery-based impact study
Research questions
Additional data collection
What appears to be the impact of YCC on….
school engagement and satisfaction
behavior at school
postsecondary credits earned during high school
educational expectations and knowledge
work readiness skills
paid work experiences
employment expectations, and
reduced involvement with criminal justice and substance abuse?
Follow-up data of students in three selected sites (spring/summer 2018)
School records in four selected school districts (summer/fall 2018)

19. Comparison group impact study
Research questions
Additional data collection
What is the impact of YCC on…
school attendance
credit accumulation, and
math and English proficiency?
Does the impact of YCC vary by student or program characteristics?
School records in sixteen school districts (summer/fall 2018)

20. For more information U.S. Department of Labor Evaluation team
Jessica Lohman:
Gloria Salas-Kos:
Evaluation team
Jeanne Bellotti:
Nan Maxwell:
Peter Schochet: