1. CHEO Evaluation Rutgers University, School of Management and Labor Relations Heather McKay Director, Innovative Training and Workforce Development Research

2. Rutgers CWW and Evaluation Conducted numerous evaluations and research projects in education, training and workforce throughout the US and abroad Rutgers CWW has worked in Colorado since 2008 Conducted three research projects in Colorado to date: online learning project, sectors project evaluation, TAACCCT Round 1

3. Rutgers Team Heather McKay Suzanne Michael Debbie Borie-Holtz Sara Haviland Laura Barrett Renee Edwards Joseph Rua

4. Why Evaluate? Federal grant and an evaluation is required Understand the type and degree of impact of TAA on program development, academic success and employment Identify best strategies/practices Collect observations, insights and lessons learned to inform, refine and/or develop more effective programs/services to meet existent and emergent needs of students, colleges and industries Identify issues/needs that can inform academic practice and public policy Provide feedback to colleges throughout the lifetime of the grant so that changes can be made mid-course Share information between the colleges Tell Rutgers what you need from this evaluation

5. Process Evaluation Qualitative Quantitative Feedback ongoing

6. Focus of the Qualitative Evaluation Assess the implementation of CHEO across all colleges Examine how each college is implementing their individual project goals. Document and understand the partnerships in the project. Assess how these groups are working together and the influence these partnerships are having on the project. Look at the process of recruiting TAA and TAA eligible/like students to programming Examine the experience of students. Study the development, delivery, and use of the online and hybrid curriculum

7. Focus of Qualitative Evaluation Cont. Examine the relationship and partnership with local workforce systems and employers Track career coach contact Track the development of stackable credentials in the region. Examine faculty training Examine the use of NANSLO

8. Focus of the Quantitative Evaluation Group level comparison cohorts for redesigned programming and coursework Individual matching of students in redesigned programming to others at the colleges Analysis of individual level data with workforce data including UI, TAA, and where possible other workforce data sources. Analyze the path and assess the population of students pursuing the stackable credentials identified in the grant. Follow the path of students using the services of the career coach Document increased capacity of institutions for serving students.

9. Attending group Sessions/calls Career Coach Sessions College Site Visits NANSLO Meetings College Administrators Grant Personnel Partner Employers State-level Stakeholders Faculty Students NANSLO partners Historical and Current Academic & Student Profile Data from College Data Sources UI Data Career Coach data Pre & Post Course Surveys Interviews Focus Groups Individual Student Data Comparative Cohorts Observations Interviews & Focus Groups \ COETC EVALUATION QUALITATIVE AND QUANTITATIVE ANALYSES

10. DATA PROCESS: INDIVIDUAL QUANTITATIVE DATA Student Variables Banner/or state equivalent Career Coach reporting Wage Data Possible workforce data Data Sources Cross- walked Data Reported Individual & Group Level (Individual Level Data: Identifiers Removed)

11. Important Issues To Date Institutional Review Boards Cross walking data Access to data and confidentiality Comparative cohorts Reporting data Communication flow between Rutgers, Pueblo CC and the colleges

12. Timeline Data conversations have begun IRB conversations have begun College interviews will be scheduled over the coming months Following these conversations the evaluation plan will be refined and revised

13. Contact Information and Questions Heather McKay hmckay@work.rutgers.edu 848-932-1275