1. Early Lessons from TAACCCT Evaluation by Donna Tonini, Research Specialist, OCCRL National Council for Workforce Education Conference Milwaukee, WI 10/16/2013

2. Who is OCCRL? Our Mission: A research office which studies policies, programs, and practices designed to enhance outcomes for diverse youth and adults who seek to transition to and through college to employment. Examples of our evaluation projects include: TAACCCT Grant Evaluations Shifting Gears Adult Education Bridge Program Evaluations College and Career Readiness Adult Career Pathways

3. What is TAACCCT? Department of Labor (DOL) Grant: Trade Adjustment Assistance Community College and Career Training, 10/2011 – 9/2014 Purpose: to provide funding to community colleges to scale up and improve short-term (< 2 years) educational & career-training programs to prepare students for employment in high-skilled, high-wage jobs. Target Population: While the grant supports TAA (Trade Adjustment Assistance) workers (who were retrenched /downsized due to overseas job outsourcing), the DOL also expects the programs will benefit a wide range of individuals. Goal: to enable the higher education community to develop, institute and improve education and training programming which will assist the target population in acquiring the skills, degrees, and credentials to meet the high-skilled and high-wage employment needs mandated by employers.

4. Understanding Priorities of the Sponsor and the Goals of the Colleges The DOL’s Round I Priorities for TAACCCT: Accelerate progress for low-skilled workers Improve retention and achievement rates and/or reduce time to completion Build programs that meet industry Strengthen online and technology-enabled learning

5. Introducing the Colleges: H2P: Health Professions Pathways Consortium Cincinnati State – Lead (OH) Anoka-Ramsey Community College (MN) Ashland Community and Technical College (KY) City Colleges of Chicago (IL) El Centro College (TX) Jefferson Community and Technical College (KY) Owens Community College (OH) Pine Technical College (MN) Texarkana College (TX)

6. TAACCCT Strategies – H2P Online assessment and enhanced career guidance Contextualized developmental education Competency-based core curriculum Industry-recognized stackable credentials Enhanced retention support Training programs for incumbent health professions workers Enhanced data and accountability systems Galvanize a national movement

7. Develop Evaluation Plan Based on Priorities & Goals - Three Linked Studies Implementation Evaluation of the model, including Programs of Study and core strategies, as articulated through consortium theory of change. Impact Evaluation of short- and longer-term outcome measures for participants, including a rigorous evaluation (quasi or true experimental design). Continuous Improvement through regular review of the measures and outcomes, using consortium and college data to support sustainability and scale up. Implementation Outcomes Continuous Improvement

8. Develop Questions that Address Priorities & Goals What is the baseline and progress on implementation related to TAACCCT-impacted Programs of Study and strategies? Who are your partners and what roles do they play in supporting the implementation of TAACCCT-impacted Programs of Study and strategies? How are student outcomes affected? What strategies and activities are the consortium and co-grantee colleges implementing, and what is the expected impact on the Programs of Study? How are student outcomes affected? What modifications and improvements are being made to funded Programs of Study? How are student outcomes affected? What barriers and supports exist, and how do they impact colleges’ efforts in creating and enhancing Programs of Study and strategies? How are student outcomes affected?

9. Lessons Differing interpretations of DOL requirements Clarity of expectations Information sharing Defining a common set of measurables Creating value for the client

10. Open, Clear, and Constant Communication! Identify key stakeholders:  Program leadership  Institutional Review (IR)  External partners (Employers, Workforce Agencies) Involve stakeholders in process:  Engage colleges in shared vision  Develop logic models Leverage promising practices:  Communities of Practice (COP)  Monthly teleconferences  Basecamp  Blogs

11. Sample Logic Model: Implementation of H2P Grant Strategies (Graphic developed by D. Bragg, 2013)

12. Develop Data Collection and Analysis Plans to Address Priorities & Goals

13. Tools for Collecting Data on Strategies, POS

14. Highlighting Promising Practices Curricular Innovations:  Core curriculum  Contextualized developmental education  Galvanizing a national movement Improved Student Services:  Dedicated Retention Specialist/Career Coach  Online prior learning assessments (PLA) Strengthened Career Pathways:  Stackable credentials  Incumbent worker training programs Enhanced Partnerships:  Employers  Workforce agencies

15. Locating the Stories to Tell What is the baseline and progress on implementation related to TAACCCT-impacted Programs of Study (POS) and strategies? POS: The largest number of POS are Associate’s degree programs. Top 3 (of 8) Strategies at Implementation Level: * Enhanced Retention Support (8 of 9 colleges) * Online Assessment and Enhanced Career Guidance (7 of 9 colleges) * Enhanced Data and Accountability Systems (7 of 9 colleges) Commonalities: Barrier: Lack of TAA participants (1) Strength: Value added by external partners (employers, etc.) Area for Improvement: Communication, info-sharing (inter-college)

16. Finding Patterns in the Data How are the consortia, students and employers impacted? Outcomes: Enrollments - H2P: 3,835Credentials - H2P:430 Sample Outcomes from student demographic course-level data: Completers – Retention – Credit Hours - Employment Upcoming Analysis: Multi-level analysis of student level data - degree attainment, wage increases. Pathways to Results – Continuous Improvement

17. Annual Performance Report – Table 1

18. Obstacles and Opportunities For Learning Please share your experiences from the field for discussion. Office of Community College Research and Leadership University of Illinois at Urbana-Champaign 51 Gerty Drive, 129 CRC | Champaign, IL 61820 | Phone: (217) 244-9390 | Email: occrl@illinois.edu