Statistical Targeting to Support Reemployment Services December 17, 2008 2PM ET

Presentation Slide Area Attendee List Chat Room Presentation Slide Area Moderator: There are 4 areas that I would like to call to your attention: The first is the presentation slide area, it’s where the main content for the webinar will appear and it will be the main focus of your attention throughout the webinar. The second is the attendee list, it’s located on the top left of your screen and it displays a list of all attendees to this webinar. The third is the chat room, also at the left of the screen, directly below the attendee list, it allows you to ask questions, or make comments during the webinar. We are using a ONE WAY conference call for the audio portion of this webinar – you can hear the presenters, but they cannot hear you. Due to the large number of attendees, and to control background noise and interruptions, we’re going to rely on this CHAT feature to solicit your questions throughout the session. Those attending with a group will want to designate a “scribe” to represent your group by entering questions as they occur ANYTIME throughout the session. Your questions are automatically transmitted to the presenters and will be answered during dedicated question and answer periods. The fourth is the “Meeting” menu located at the top left of your monitor. Once you click the “Meeting” menu, you will be presented with a few options to choose from. The only option I want to point out to you is the “Full Screen” option. If you feel the size of the room is too small or if it is difficult for you to read the text, I suggest you switch to Full Screen mode. Depending on the size of your monitor, this will increase the size of the webinar room so that images will appear larger.

Submitting Questions Chat Room Enter questions into the Chat Room (located in lower left corner of the virtual classroom). To submit a question or comment, type the question in the text box and click the arrow button. Your name, the text “(Submitted Question)” and your question will appear in red on your screen, indicating successful submission. Questions are directly transmitted to presenters – other participants will not see your questions. CLOSE CHAT SLIDE Moderator: To submit a question using the chat feaure, type the question in the text box and click the arrow button to submit. When you submit a question, your name, the text “Submitted Question” and your question will appear in red on your screen, indicating successful submission. Note that Questions are directly transmitted to presenters – other participants will not see your questions. Again, please be sure to enter your questions at ANYTIME throughout our session. We’ll make time to answer as many of them as possible during the session. (ADVANCE SLIDE) Text Box Arrow Button Statistical Targeting to Support Reemployment Services

Practice In the chat room, please type the name of your organization, your location, and how many people are attending with you today. In order to get an idea of who is attending this webinar with us today, and to give you a chance to practice using the Chat Feature, please type the name of your organization, your location, and how many people are attending with you today in the Chat Room, and then click the Arrow button to submit your entry. (NOTE: Wait for responses to begin appearing in presenter chat, acknowledge some of those responses, and transition to the next slide.) Statistical Targeting to Support Reemployment Services

Access to Webinar Materials Webinar Resources:   Recordings, transcripts, PPTs & handouts are posted to “My Events” page All resources are posted under the… Moderator: (Public Webinar) Please note that all webinar resources, such as the PPT, recording, transcript and any handouts used during the webinar will be available for download from your “My Events” page. When you initially log into your W31 account, your “My Events” page will be the first page you’re taken to. The recording and transcript will be available within 2 business days after the session concludes while the PPT and any handouts will be posted before the session begins. To access the recording or any resource associated with the recording, like the PPT or handouts, simply mouse over and left click the “View Resources” link to the right of the session title to download or view those files. (ADVANCE SLIDE) Statistical Targeting to Support Reemployment Services

Presenters Presenters: Scott Gibbons, Office of Workforce Security, Employment and Training Administration Chris O’Leary, Senior Economist, Upjohn Institute Moderator: Cheryl Atkinson, Administrator, Office of Workforce Security, Employment and Training Administration Statistical Targeting to Support Reemployment Services

Presentation Context Reemployment TAT project http://www.workforce3one.org/reemploymentworks/ WPRS and FDSS are reemployment vehicles Use best available data to support informed decision making Stress early intervention Stress individual attention Statistical Targeting to Support Reemployment Services

Target claimants through data How does WPRS work Worker Profiling: Identify characteristics of claimants who need services Target claimants through data Reemployment Services: Refer claimants to services based on claimant needs and capacity Monitor claimants following intervention Statistical Targeting to Support Reemployment Services

Resource allocation methodology Funds cannot serve everyone Rationale for WPRS Resource allocation methodology Funds cannot serve everyone Models prioritize service delivery Early identification and action Potential for a win / win situation Statistical Targeting to Support Reemployment Services

Why Serve WPRS Claimants? Legal Basis: Public Law (PL) 103-152 Practical Basis: they are generally not… Waiting to be recalled by previous employer Waiting to be called by union hiring hall Repeat filers Seasonal employees Statistical Targeting to Support Reemployment Services

Why Serve WPRS Claimants? They generally have: Long-term attachment to workforce Successful work history Longer job tenures Higher education Similar industry and occupational profiles Similar base period earnings Financial incentives to return to work Statistical Targeting to Support Reemployment Services

LMI Data, State Data, ES Data, Characteristics… Profiling Model UI Claimant First Pay Records Historical UI Data, LMI Data, State Data, ES Data, Characteristics… Profiling Model Statistical Targeting to Support Reemployment Services

Claimant Characteristics Labor Market Information WPRS Model Variables Claimant Characteristics Education, tenure, industry, occupation, etc Labor Market Information Ind. & Occ. growth/decline rate, base period wages relative to industry/occupation average, LAUS Proxies for Job Search Intensity Wage replacement rate, prior exhaustion history, no. of employers, severance pay Statistical Targeting to Support Reemployment Services

LMI Data, State Data, ES Data, Characteristics… Profiling Model UI Claimant First Pay Records Historical UI Data, LMI Data, State Data, ES Data, Characteristics… Profiling Model State or Local Criteria for Referral Claimants Not in Profiling Pool Claimants in Profiling Pool Statistical Targeting to Support Reemployment Services

LMI Data, State Data, ES Data, Characteristics… Profiling Model UI Claimant First Pay Records Historical UI Data, LMI Data, State Data, ES Data, Characteristics… Profiling Model State or Local Criteria for Referral Claimants Not in Profiling Pool Claimants in Profiling Pool Service Capacity WPRS referrals To Service Statistical Targeting to Support Reemployment Services

Total UI Claimants Profiled: 7,384,622 FY08 WPRS Claimant Flows Total UI Claimants Profiled: 7,384,622 Claimants Eligible for Referral: 3,390,927 Total Referred to Services: 1,187,254 Completing at least one service: 868,271 Statistical Targeting to Support Reemployment Services

Service Delivery UI Claimant First Pay Records Historical UI Data, LMI Data, State Data, ES Data, Characteristics… Profiling Model State or Local Criteria for Referral Claimants Not in Profiling Pool Claimants in Profiling Pool Service Capacity WPRS referrals To Service Exemptions Service Delivery Orientation & Assessment Placement Services Counseling Self- Employment Job Search Workshops Education and Training Statistical Targeting to Support Reemployment Services

FY08 WPRS Services Delivered Orientation: 623,392 Individual Assessment: 442,976 Counseling: 132,238 Placement Services: 410,212 Job Search Workshops: 373,152 Education and Training: 114,132 Statistical Targeting to Support Reemployment Services

Service Delivery UI Claimant First Pay Records Historical UI Data, LMI Data, State Data, ES Data, Characteristics… Profiling Model State or Local Criteria for Referral Claimants Not in Profiling Pool Claimants in Profiling Pool Service Capacity WPRS referrals To Service Wage Records Exemptions Service Delivery Reporting On Service delivery 9048 Outcomes 9049 Orientation & Assessment Placement Services Counseling Self- Employment Job Search Workshops Education and Training Statistical Targeting to Support Reemployment Services

Evaluation of Worker Profiling Models Recent Evaluations Evaluation of Worker Profiling Models ETA Occasional Paper: 2007-15 GAO evaluation of the WPRS system GAO 07-680 Statistical Targeting to Support Reemployment Services

TAT project provides perfect opportunity to refocus these tools Focus all elements of WPRS on the reemployment goal Requires a cohesive approach to reemployment through WPRS System only works if all elements work together Statistical Targeting to Support Reemployment Services

Based on recent patterns and trends Updating may be sufficient Referrals to service For referrals to be timely, objective, and accurate, model must be effective If referrals are bad, it doesn’t matter if services and service delivery are perfect, results will be suboptimal Based on recent patterns and trends Updating may be sufficient Statistical Targeting to Support Reemployment Services

Claimants must follow through If states have a good model, determine that a claimant needs help, and claimant refuses… …they must hold claimants accountable Many use pending issues for ERP no-shows Most states will not do this with WPRS Statistical Targeting to Support Reemployment Services

Provide appropriate services Needs vary from person to person States should seek to balance action in light of their overall goal Serve more claimants less intensively Serve less claimants more intensively All actions should be based on claimants’ needs Statistical Targeting to Support Reemployment Services

Establish Feedback Loops All components must communicate with and inform other components WPRS modelers should know what service providers are experiencing Service providers should know what sort of profile to expect UI notified if claimant does not attend Communication is key to making this thing work Statistical Targeting to Support Reemployment Services

What are we doing to improve reemployment outcomes? Building new and updated models Fostering discussion within states about program goals, methods and tools Looking for ways to fund services Improving program reporting/assessment Statistical Targeting to Support Reemployment Services

Innovative Ideas in Targeting Reemployment Service Delivery Statistical Targeting to Support Reemployment Services

Expanding uses of WPRS scores Using profiling scores to target ERPs Using profiling scores to support service delivery decision making Statistical Targeting to Support Reemployment Services

Weeks of Potential Duration On UI Probability Of Exhaustion High (26 weeks) Low (8 weeks) WPRS Statistical Targeting to Support Reemployment Services

Weeks of Potential Duration On UI Probability Of Exhaustion High (26 weeks) Low (8 weeks) REA WPRS Statistical Targeting to Support Reemployment Services

Weeks of Potential Duration On UI Probability Of Exhaustion High (26 weeks) Low (8 weeks) REA WPRS ERP Statistical Targeting to Support Reemployment Services

Expanding uses of WPRS data Using demographic and characteristic data Utilizing LMI, BLS information to help with decision making Employer driven job referrals Fault-overpayment models Statistical Targeting to Support Reemployment Services

Targeting and Assessment From Existing Data State Program or Service Provider Preferences UI Claimant First Pay Records Historical UI Data, LMI Data, State Data, ES Data, Characteristics… Employer Needs Targeting and Assessment From Existing Data Statistical Targeting to Support Reemployment Services

Targeting and Assessment From Existing Data State Program or Service Provider Preferences UI Claimant First Pay Records Historical UI Data, LMI Data, State Data, ES Data, Characteristics… Employer Needs Targeting and Assessment From Existing Data WPRS for Likely Exhaustees Reemployment Bonuses Self Employment Assistance REAs Fault Overpayment Detection Reemployment Services Demand Driven Job Referrals Statistical Targeting to Support Reemployment Services

www.upjohn.org/targeting.html Christopher J. O’Leary Profiling and Targeting Public Employment Programs Christopher J. O’Leary W. E. Upjohn Institute for Employment Research oleary@upjohn.org www.upjohn.org/targeting.html Statistical Targeting to Support Reemployment Services

Australian Experience A Simpler FDSS Overview Definitions U.S. Experience Programs implemented Demonstrations and pilots - FDSS European Experience Australian Experience A Simpler FDSS Statistical Targeting to Support Reemployment Services

Definitions Profiling – Sorting for referral to services based on presumed needs given observable characteristics of job seekers. Targeting – Referral to services with the aim of promoting increased effectiveness based on sub-group impact estimates. Statistical Targeting to Support Reemployment Services

U.S. Experience Implemented Demonstrations and Pilots (TES Book 2002) Job Search Assistance (1993 WPRS) Self Employment Assistance (1993, 1998) Demonstrations and Pilots (TES Book 2002) Welfare to Work Reemployment Bonus Personal Reemployment Accounts Frontline Decision Support System (FDSS) Statistical Targeting to Support Reemployment Services

Frontline Decision Support System (FDSS) Systematic Job Search Module Chance of returning to same industry Estimate likely reemployment earnings Scan job prospects Identify related occupations Provide targeted information about specific jobs Service Referral Module Rank services by effectiveness for re-employment Indicate recent rates of usage Statistical Targeting to Support Reemployment Services

Data Sources UI wage records Employment/earnings history Program records: ES, UI, JTPA/WIA WtW Worker/client characteristics Program participation Employment history Labor Market Information Local unemployment rates Occupation/industry projections Related/transferable occupations Massive layoffs Statistical Targeting to Support Reemployment Services

Service Referral Module FDSS Tools Intake Core Services Systematic Job Search Module Review intake info & work history Estimate chance of returning To industry/occupation Earnings history & estimate Search job orders Choose related occupations Referrals Service Referral Module Training Services Intensive Services Statistical Targeting to Support Reemployment Services

Service Referral Module Foundations Estimate models of the probability of reemployment (employability score) Rank services by rate of successful outcomes for individuals grouped by employability score Statistical Targeting to Support Reemployment Services

Service Referral Module Provides a customized list of services most effective for participants in each employability group (5 in each region) Core and intensive list Training list List is based on service usage and outcome patterns for recent participants in each region grouped by employability score Statistical Targeting to Support Reemployment Services

Statistical Targeting to Support Reemployment Services

Statistical Targeting to Support Reemployment Services

Statistical Targeting to Support Reemployment Services

Statistical Targeting to Support Reemployment Services

Statistical Targeting to Support Reemployment Services

Statistical Targeting to Support Reemployment Services

European Experience Denmark Flex-security policy success Predicting long term unemployment Statistical model Caseworker assessments Early referral to reemployment services Implemented and withdrawn (job barometer) Statistical Targeting to Support Reemployment Services

European Experience Switzerland Pilot test of statistical assignment to most effective services for employment success Effectiveness models based on job seeker characteristics, service usage, and outcomes 50-50 random assignment of availability of effectiveness model results for use by frontline staff in counseling job seekers Extremely low usage by frontline staff Statistical Targeting to Support Reemployment Services

European Experience The Netherlands Frontline staff subjectively assign unemployed into one of 4 employability categories: (1) difficult, (2) moderately difficult, (3) favorable, and (4) extremely good Services focused on the middle two groups Targets for performance are based groups Outcomes measure the success rates of individual reintegration plans Statistical Targeting to Support Reemployment Services

European Experience: Grouping Austria 3 zones 3 bundles Problems (objective) and needs (subjective) France 7 groups Questionnaire and interview (subjective) Germany 4 groups Characteristics and questionnaire (mixed) Hungary 3 groups Statistical model (objective) Statistical Targeting to Support Reemployment Services

Australian Experience The Job Network (1998 unemployment 7.7%) Competitive placement enterprises (some for profit) Centrelink (PES) is system entry point Job Seeker Classification Instrument (JSCI) is a statistical score estimate of long term unemployment Target intensive services to long-term unemployed Staff judgment supplements statistical targeting Try to service highest long-term unemployment risk Participation in assigned services is required for continued UI eligibility Statistical Targeting to Support Reemployment Services

Australian Experience Employment Pathway Fund (2009) (Unemployment rate 4.2%) Centrelink applies statistical JSCI and sets Employment Pathway Plan (EPP) Customer groups: work ready, employability development, LTU get a dole and PSE Customized services for job seekers Meet labor demands by employers in geographic areas with skills shortages Statistical Targeting to Support Reemployment Services

A Simpler FDSS Don’t estimate return to prior industry Don’t estimate reemployment earnings Rely on existing OES and LMI data Don’t estimate employability scores Rely on WPRS to group customers Still requires linking data across programs including: UI, ES, WIA, and LMI Statistical Targeting to Support Reemployment Services

A Simpler FDSS Systematic Job Search Provide background information view Customized info on prior occupations OES wage info for prior occupations LMI info on employment growth Customized info on declining industries UI claims and mass layoff data Related occupations (O*Net) OES info on related occupations Statistical Targeting to Support Reemployment Services

A Simpler FDSS Service Referral Module Rely on WPRS models Set up regions in the state Separate metro regions Balance of the state Set up 5 ranges in WPRS score distribution Sort services by WPRS score groups Statistical Targeting to Support Reemployment Services

Skills Development Referral Module A Simpler FDSS Skills Development Referral Module Conduct skills inventory Identify skills gaps for available jobs List available training institutions Provide consumer reports on training institution effectiveness Statistical Targeting to Support Reemployment Services

A Simpler FDSS WPRS models must be up to date Identify proper OES and LMI data Summarize recent UI claims for high layoff occupation/industry groups Access mass layoff survey data to identify declining industries Pilot test to learn if the WPRS score can be used to replace employability score in FDSS Share tested FDSS procedures with all states www.upjohn.org/targeting.html Statistical Targeting to Support Reemployment Services

Profiling and Targeting Public Employment Programs Christopher J. O’Leary W. E. Upjohn Institute for Employment Research oleary@upjohn.org www.upjohn.org/targeting.html Statistical Targeting to Support Reemployment Services

Please enter your questions into the Chat Room! Question and Answer Period (ADVANCE SLIDE) Please enter your questions into the Chat Room! Statistical Targeting to Support Reemployment Services

Share Your Ideas with Your Peers! Simply log on to Workforce3 One and look for the “Share Content” located on the Homepage Share your demand-driven strategic plans, models, innovations, resources, and ideas! Workforce3 One is a is a tool built for you and BY you, encouraging peer-to-peer learning among the various communities. The success of Workforce3 One relies on contributions from people like you! Share your ideas, innovations, and more with others! We welcome suggestions for documents to share, programs to feature, and any relevant news or information that you would like to exchange with your colleagues. To submit your content, visit the Suggest Content page on the Workforce3One webspace, which is located at the URL featured on this slide. (ADVANCE SLIDE) Submit your content to Workforce3 One at: http://workforce3one.org/members/mywf3/suggest.cfm Statistical Targeting to Support Reemployment Services

Access to Webinar Materials Webinar Resources:   Recordings, transcripts, PPTs & handouts are posted to “My Events” page All resources are posted under the… Moderator: (Public Webinar) Please note that all webinar resources, such as the PPT, recording, transcript and any handouts used during the webinar will be available for download from your “My Events” page. When you initially log into your W31 account, your “My Events” page will be the first page you’re taken to. The recording and transcript will be available within 2 business days after the session concludes while the PPT and any handouts will be posted before the session begins. To access the recording or any resource associated with the recording, like the PPT or handouts, simply mouse over and left click the “View Resources” link to the right of the session title to download or view those files. (ADVANCE SLIDE)

Stay Informed, Get Connected! Workforce3 One: Communities of practice Live web-based events Register for updates! For more information about the workforce investment system: Visit www.careeronestop.org Call 1-877-US2-JOBS You can learn more and stay connected with trends and innovations by logging on to Workforce3 One, where you’ll find: Engaging communities of practice where you can share ideas, questions and innovations, and connect with peers; Learning through live Web-conferencing events that feature leaders and experts from industry and from government; and, A means of registering to be informed of news and events as they occur. We encourage you to make note of Workforce3 One – it is a powerful new tool, funded by ETA and “powered” by YOU! You can also learn more about the workforce investment system by visiting www.careeronestop.org and by calling 1-877-US2-JOBS. (ADVANCE SLIDE) Statistical Targeting to Support Reemployment Services

THANKS! www.workforce3one.org Moderator: On behalf of Workforce3 One and today’s webinar presenters, I’d like to conclude today’s session by thanking you all for your time, and we look forward to seeing you on future webinars! Thank you. Statistical Targeting to Support Reemployment Services