A Training Case Study Community Literacy Summit Washington, D.C. March 19, 2007 Douglas Marriott, Outreach Director “Challenges in Measuring and Collecting Data for Evaluation”

Patti Patti  The Literacy Network of Greater Los Angeles (LNGLA) Guiding Principles and Current Programming  Project Background  General project challenges of data measurement with MTA collaboration training  The unique and successful case of the MTA training, community literacy in action, measuring and collecting data for evaluation Presentation Overview

Patti Patti Guiding Principles of LNGLA We envision a Greater Los Angeles where all people are empowered by literacy to achieve their full potential in our community, a community which is socially just, culturally dynamic and economically productive. Our goal is make literacy resources easily accessible to learners, literacy service providers, and employers in our community through partnerships, advocacy, and direct program services.

Patti Patti Current Programming Resource and Referral  That Literacy Phone Book   READ Special Events  Literacy In Media Awards  Literacy Summit Volunteer Program – Recruits, trains, and places volunteers at partner provider site. Spun off from the – Develops and implements workplace and workforce literacy trainings for workers within Los Angeles

Patti Patti Project Background  Employment Policy Foundation study gave rise to The L.A. Workforce Literacy Project  is a public/private partnership of organizations dedicated to the advancement of workforce literacy  Project has developed and implemented a 25 hour Trainer Certification program and a Quality Standards Handbook  Implementation of planning began in Spring of 2006 and has included workplace trainings at local businesses and workforce trainings at social service agencies

Patti Patti The MTA Bus Driver Project We Make the Road by Walking (or riding)…  Between 2002 and 2006, the Los Angeles County Metropolitan Transportation Agency (Metro) began providing more bus service in Los Angeles County.  Over the past two years, Metro has been unable to fill from 200 to 300 bus operator vacancies, costing millions in overtime labor  Metro performed an analysis of the high failure rate in their training program, determining that the bus operator success rate could be significantly improved by offering interested applicants a pre-training program tailored to help them succeed in bus operator training.

Patti Patti  Recruitment, data management, support services, case management  Lead on project coordination, job training and customer service instruction  Employment! HR orientations, program support  instructional services workshops), trained instructors, customized curriculum development, volunteer support services, resource and referral information Collaboration in Action

Patti Patti

Patti Patti Ongoing Challenges for MTA Project Skills Assessment Implement a standard intake and assessment system and develop customized training for workers and employer. Performance Measurement Create MTA training components and requirements for program completion. Data Management Design and implement a shared, data management tool to support needs of all partnering agencies.

Patti Patti Skills Assessment Need or Expectation: A uniform tool to place participants in program Challenge: Meeting the needs of all partnering agencies and adult learners

Patti Patti Skills Assessment Solution: A standardized assessment tool (CASAS Workplace Exam) is implemented to determine placement in two week, five week, or sixteen week trainings An MTA customer service exam is used as a pre and post assessment tool

Patti Patti Performance Measurement Need or Expectation: Customized training course to prepare participants for success at Metro Challenge: Creating a customized curriculum to reach parallel missions of collaborators and individual needs of participants

Patti Patti Performance Measurement Solution: Participants are provided with job readiness training, customer service training, English as a Second Language (ESL) when needed, basic skills training in reading and writing, study skills and critical thinking skills all contextualized to the needs of successfully completing the Metro training. The training is constantly adapting to lessons learned.

Patti Patti Data Management Need or Expectation: Each participant needs to be followed to evaluate needs of the program and provide accurate accounting for all partners. Challenge Identifying resources and best methods for capturing data

Patti Patti Data Management Solution WorkSource center dedicates staff member to data management and support services with direct links to MTA HR departments and partner agencies.

Patti Patti Results to Date  Between Aug and Jan there were 7 trainings that reached 233 participants  219 completed trainings and earned certificates  198 passed MTA interview and began new employment orientation  MTA statistics show the agency is ½ way to meeting their goal of 300 new operators. The vacancy rate is down 161 from a high of 315  Part-time attrition decreased from 100% to 33.49% as many are succeeding in MTA training program

Patti Patti Conclusions  Continual process improvement impossible without detailed data collection for evaluation  All stakeholders must share a vision for what program success will look like with mutually agreed upon outcomes to be evaluated  Parallel missions, mutual respect for roles and resources, creativity and flexibility are key components to project success.