Burt S. Barnow George Washington University

Slides:



Advertisements
Similar presentations
WIA Closure, Exit and Follow-up
Advertisements

COMMON MEASURES Presented by: Phil Degon Workforce Administration September 2006.
COMMON MEASURES. Training Objectives Review Common Measures data sources Discuss Reports and Desk Aids.
Perkins IV National Definitions and State Reporting: The Impact on Data Collection in Texas Gabriela Borcoman Texas Higher Education Coordinating Board.
Judy Mortrude DEED Program Administrator Workforce Innovation and Opportunities Act.
PERFORMANCE ACCOUNTABILITY: WIA YOUTH PERFORMANCE MEASURES John J. Heldrich Center for Workforce Development Rutgers, the State University of New Jersey.
Promoting a flexible, innovative, and effective workforce system within the State of Michigan. WIOA Overview Michigan Works! Association Conference October.
Employment and Training Administration DEPARTMENT OF LABOR ETA Simple Ways to Improve Your Reporting Greg Wilson Office of Performance and Technology Employment.
BIDDER’S WORKSHOP CAREER AND YOUTH SERVICES. The Workforce Development Board (WDB) 7/1/2015 The Golden Sierra Workforce Board provides (WIOA) oversight.
Data Integration Project. MoSTEMWINS Data Projects Strategy 1 -- Develop and Implement a statewide data system in support of tracking student performance.
WIA REAUTHORIZATION Nancy W. Hawkins October 7, 2003 Florida Workforce Summit Meeting Lake Buena Vista, Florida.
New Options and Opportunities Under the Workforce Innovation and Opportunity Act.
1 Literacy and Numeracy Gains Webinar February 3, :00 am - 11:00 am.
WIA PERFORMANCE MEASURES AND STANDARDS: The WIASRD, Common Measures and Standards Negotiation Challenges Christopher T. King Ray Marshall Center for the.
Measurement Standardization in Perkins The Perspective from the Integrated Performance Information (IPI) Project Data Quality Institute June 14, 2005 Bryan.
Presented by Lois ScottAugust 21, Why We Are Here Financial and Participant Data Overview Program Year 2014 – 2015 – Program Performance – Performance.
FY07 COMMON MEASURES CHANGES FOR REPORTING AND MOSES TRACKING.
Understanding the NRS Rosemary Matt NYS Director of Accountability.
U.S. Department of Labor Employment and Training Administration Keith Rowe ETA – Dallas Region Office Presenter ETA – PROTECH WISPR Quarterly Reports and.
December 2010 Performance Reporting and Analysis, Manager One-Stop and Program Support Florida Agency for Workforce Innovation 107 East Madison Street.
Integrated Performance Information (IPI) Project Mike Switzer Workforce Florida, Inc. Jay Pfeiffer Florida Department of.
Employment and Training Administration DEPARTMENT OF LABOR ETA 1 Change in Reporting Requirements for the Workforce Investment Act Standardized Record.
INDIAN AND NATIVE AMERICAN (INA) EMPLOYMENT AND TRAINING PROGRAM UNDER SECTION 166 OF THE WORKFORCE INVESTMENT ACT (WIA) An Orientation for Grantees Introduction.
Changing Perspectives on Workforce System Performance- Adjustment Models Workforce Innovations Conference July 21, 2004.
1 The Role of Performance Management in Workforce Investment Programs Burt S. Barnow Institute for Policy Studies Johns Hopkins University Prepared for.
Brette Kaplan WurzburgSteven Spillan Brustein & Manasevit, PLLC Spring Forum 2015 An Overview of the New AEFLA.
Strategies, Funding, Performance From TEGL 2-07 “Features of Registered Apprenticeship, including its customized format, the extensive industry knowledge.
1 Implementing WIA Performance Measures for the European Social Fund Cynthia Fagnoni, Managing Director Education, Workforce, and Income Security Issues.
Trade Act Participant Report (TAPR) 2005 Revisions for Implementing Common Measures.
Recent Advances in Performance Measurement of Federal Workforce Development Programs Evaluation and Performance Management of Job Training Programs Organized.
Collaboration and Partnerships CareerSource Central Florida
A Call to Action for 2016 Student Success Anson Green Director Texas Workforce Commission November 17, 2016 WIOA UPDATE NOVEMBER 17,
PERKINS IV AND THE WORKFORCE INNOVATION AND OPPORTUNITY ACT (WIOA): INTERSECTIONS AND OPPORTUNITIES.
Durham Workforce Development Board Request for Proposals Bidders’ Conference February 16, 2016.
Common Performance Measures for Employment and Training Programs SC Workforce Development Partnership Conference October 26-29, 2003 Brad Sickles
1 Understanding the Updated Red and Green Report for WIA Programs Presented by: BETTYE MCGLOCKTON, DEHRYL MCCALL, MERSHAL NOBLE AGENCY FOR WORKFORCE INNOVATION.
WIOA’s Goal Make Participants’ Skills Everyone’s Business
Department of Labor, Licensing and Regulation
A Brief Look at Career and Technical Education NCCCS - Perkins Update
ABE Policy & Accountability
TAACCCT Performance Reporting Q&A: .
WIOA Section 166 – Indian and Native American Program
Alachua/Bradford FloridaWorks Regional Workforce Board Overview
WIOA Performance Workgroup
Introduction to Performance and Accountability in Adult Education
Chapter 14 Workforce and Career Development History
Aligning Program Practice and WIOA Performance Reporting
North Carolina Workforce Development System
The Application of Common Measures
Demonstration Project: Career Pathways and Co-Enrollment of WIOA Title II Students into Title I Programs Competitive Grant Proposal: Application Webinar.
CareerSource Chipola Performance Overview
THE WORKFORCE INNOVATION AND OPPORTUNITY ACT (WIOA)
Cleveland/Cuyahoga County Workforce Development Board – Area
WIOA Annual Performance Report
Understanding and Achieving Performance Measures
WIOA Partner Program Briefing: Adult and Dislocated Worker Programs
Promising Practices for Increasing Certificate and Credentialing Outcomes H-1B Ready to Work.
New York State Report Cards
Poll Question Have you read the Supplemental Wage Information Guidance issued under TEGL 26-16, PM 17-6, and TAC 17-04?
Performance Accountability
WIOA Accountability Ben Konruff
YCC Career Pathways Discussion
Connecting TANF to Career Pathways with HPOG
CareerSource Chipola Performance Overview
Making Common Sense of the Common Measures
Technical Assistance Webinar
WIOA Youth Performance
WIOA: OPPORTUNITIES AND CHALLENGES
Workforce Innovation and Opportunity Act
From Theory to Practice
Presentation transcript:

The U.S. Experience with Performance Monitoring and Measurement: Strengths and Weaknesses Burt S. Barnow George Washington University Prepared for the Conference on More Effective Workforce Programs through Comparative Performance Monitoring Austrian Embassy, Washington, DC November 13, 2018

Brief History of Performance Measurement in U.S. E&T Programs Performance measurement started in 1970s under the Comprehensive Employment and Training Act (CETA) CETA and subsequent E&T programs had federal money distributed to states and local areas where the services were delivered Over time, states have been given a larger role in establishing policies, and now federal government holds states accountable through performance measurement system and states oversee local programs Performance measurement was part of law under Job Training Partnership Act of 1982 and subsequent programs (WIA and WIOA)

History of Performance Measurement (continued) Motivations for instituting performance measurement in E&T programs Improve knowledge of effectiveness Reduce/eliminate principal agent problems Eradicate perverse incentives (e.g., cream skimming) Identify good and bad programs Reward good programs and penalize poor performers Original performance measurement effort for E&T programs led by economists & evaluators, so approach was different than most later work under Government Performance and Results Act (GPRA) of 1993 for other departments and programs

History of Performance Measurement (continued) Regression analysis was used to create level playing field, where performance expectations were set lower if area served more people with disadvantages in labor market and worse economic conditions Adjustments for hard to serve groups also intended to reduce incentive for programs to engage in “cream skimming” where programs served people likely to look good instead of people who needed help or more likely to benefit from program Performance standards were initially set using regression models so that roughly 75% of local programs would “pass,” but this approach no longer used Until current program (WIOA), programs included additional funding for bonuses for high performance and sanctions for poor performance, with technical assistance for low performance in one year and possibility of loss of right to operate program for poor performance two consecutive years

Performance Measures under Job Training Partnership Act Adult follow-up employment rate, defined as the proportion of adult respondents who were employed at least 20 hours per week during the 13th week after termination Adult follow-up weekly earnings, defined as average weekly earnings for all adults who were employed for at least 20 hours per week during the 13th week after termination Welfare adult follow-up employment rate, defined in the same manner as for the adult follow-up employment rate but for adult welfare recipients only Welfare follow-up weekly earnings, defined in the same manner as adult follow-up weekly earnings Youth entered employment rate defined as the proportion of youth terminees (other than potential dropouts who remained in school) who entered employment with at least 20 hours per week Youth employability enhancement rate, defined as the proportion of youth who obtained one of the employability enhancements at termination

Performance measures under Workforce Investment Act of 1998 after Institution of “Common Measures” Entered Employment 1st Qtr. after exit (Adult programs) Employment Retention 2nd and 3rd Qtr. after exit (Adult programs) Six Months Average Earnings 2nd and 3rd Qtr. after exit (Adult programs) Placement in Employment/Education 1st Qtr. after exit (Youth programs) Attainment of a Degree or Certificate by 3rd Qtr. after exit (Youth programs) Literacy Numeracy Gains (Youth programs) Statute called for “customer satisfaction measures” that were dropped after common measures adopted Before common measures were adopted, the Adult programs had a measure for attainment of a recognized credential related to educational achievement or occupational skills

Performance Measures under Workforce Innovation and Opportunity Act of 2014 Percent Employed 2nd Qtr. after exit (Adult programs) Placement in Employment/Education 2nd Qtr. after exit (Youth programs) Percent Employed 4th Qtr. after exit (Adult programs) Placement in Employment/Education 4th Qtr. after exit (Youth programs) Median Earnings 2nd Qtr. after exit (All programs) Credential Attainment (up to 1 year after exit)(All programs except Wagner-Peyser) Measurable Skill Gains (All programs except Wagner-Peyser) Effectiveness in Serving Employers (All programs)

Trends in Performance Measurement Trend toward longer-term outcomes, with no measures at time of exit and one measure now 4th quarter after exit Credential attainment added under WIA and measureable skill gains added under WIOA For the first time, WIOA only includes sanctions but no rewards for good performance Employer and participant satisfaction measures were dropped when common measures were added

Trends in Performance Measurement WIOA adds services to employer measures, and DOL is piloting 3 measures Employer penetration rate: % of establishments in state receiving services Repeat business customers: % of establishments who are receiving a service and received a service in previous 3 years Retention with same employer: % of participants who exit and are employed with the same employer in the 2nd and 4th quarters after exit States must select two of the measures and may pilot their own measure Ongoing evaluation of employer services measures by Urban Institute and GW Efficiency (cost measures) were used for a while under JTPA, but were dropped when research showed they created poor incentives

Some Suggestions for Consideration Prior research (Barnow, Heckman et al.) indicated little relationship between performance measures and program impact for JTPA Use WIA evaluation data to see if still the case Consider ways to link evaluation and performance WIOA includes sanctions but no rewards: This is unpopular with the programs, and with right structure, rewards can help develop program insights Bonus funds could be used for pilots exempt from performance measures if subject to rigorous evaluation Negotiation for standards were very unpopular under WIA (Barnow and King, D’Amico et al.): How is record under WIOA?

Some Suggestions for Consideration Given limitations of statistical models, are performance sanctions set at appropriate level? Goals for job search much different than for training: Should performance standard depend on activity rather than be same for all? Wagner-Peyser services inexpensive and intended to have quick employment effects Training under WIOA more costly and intended to boost earnings Performance measures have moved from termination date to 4 quarters after termination: Is that appropriate? Longer-term measures better capture long-term results Shorter-term measures better for management purposes

Burt S. Barnow Barnow@GWU.edu (202) 994-6379 For questions, comments, or additional information contact Burt S. Barnow Barnow@GWU.edu (202) 994-6379