1. WIA Net Impacts: Preliminary Estimates ADARE Project Briefing #3 W. E. Upjohn Institute Kevin Hollenbeck, W. E. Upjohn Institute Christopher T. King & Daniel Schroeder, Ray Marshall Center, University of Texas-Austin May 7, 2003 Welcome to the Marshall Center PowerPoint Template. To add text, click on the visible text, delete the template titles, and enter your data. To add additional slides, use Ctrl + M. You can modify the slide layout (i.e. select a pre-set layout for tables instead of a layout for text) by right-clicking on the slide background and selecting Slide Layout. Commonly used sample layouts are included. To delete slides you don’t want to use, choose Delete Slide from the Edit menu. You can also Duplicate Slides through the Insert menu. Pre-set slide transitions are included. They can be changed through the Slide Show menu. Finally, to hide comment fields (like this box), select Comments from the View menu. Please email Sarah Looney at looneys@mail.utexas.edu if you have any questions. looneys@mail.utexas.edu Welcome to the Marshall Center PowerPoint Template. To add text, click on the visible text, delete the template titles, and enter your data. To add additional slides, use Ctrl + M. You can modify the slide layout (i.e. select a pre-set layout for tables instead of a layout for text) by right-clicking on the slide background and selecting Slide Layout. Commonly used sample layouts are included. To delete slides you don’t want to use, choose Delete Slide from the Edit menu. You can also Duplicate Slides through the Insert menu. Pre-set slide transitions are included. They can be changed through the Slide Show menu. Finally, to hide comment fields (like this box), select Comments from the View menu. Please email Sarah Looney at looneys@mail.utexas.edu if you have any questions. looneys@mail.utexas.edu W.E.Upjohn Institute for Employment Research

2. W.E.Upjohn Institute for Employment Research Overview Background & WIA Context Approach Preliminary Impact Estimates Discussion Next Steps Use this page to create numbered lists.

3. W.E.Upjohn Institute for Employment Research Background & Context Key to assessing program impacts: Key to assessing program impacts: answer two questions: ‘treatment group’ 1.What happened to those who participated in the program (the ‘treatment group’)? 2.What would have happened to them if they had not participated? First question is straightforward, but the second is more complicated… This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style. This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style.

4. W.E.Upjohn Institute for Employment Research Background & Context The second question is answered by creating a group of people who are as similar as possible to the treatment group, except that they do not receive the treatment. ‘comparison group’Outcomes for the treatment group and this ‘comparison group’ are tracked over time. Program impactProgram impact is the difference between the outcomes for the two groups. This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style. This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style.

5. W.E.Upjohn Institute for Employment Research Background & Context Ideal Approach:Ideal Approach: Experimental design, which uses random assignment of participants to groups to ensure the similarity of the treatment and control groups. This is the “gold standard,” but it can be expensive to implement, and is not always feasible. This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style. This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style.

6. W.E.Upjohn Institute for Employment Research Background & Context experiment is not possibleAn experiment is not possible for WIA impact estimation because eligible clients cannot be denied service. Next-Best Approach?Next-Best Approach? Quasi-experimental design, with a comparison group selected to ensure similarity to the treatment group. But, must be carefully done to get at true program impacts, and not just differences that already existed between the groups. This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style. This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style.

7. W.E.Upjohn Institute for Employment Research Background... History of quasi-experiments: Multiple, complicated answers to the impact question are not useful to policymakers. Examples: CETA & CLMS, 1st Job Corps Evaluation —> 1985 JTLS Advisory Committee —> National JTPA Study, 2nd Job Corps Evaluation, & state evaluations (e.g., TX, UT, WA) This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style. This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style.

8. W.E.Upjohn Institute for Employment Research Background... WIA Context— Service Sequencing:Service Sequencing: Core, Intensive & Training Services Young, highly devolved programYoung, highly devolved program This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style. This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style.

9. W.E.Upjohn Institute for Employment Research Approach Data: Data: WIASRD files for FL, GA, IL, MD, MO, TX, & WA; ES registrant data for two states only; and linked UI wage records for all states. Impacts estimated Impacts estimated for WIA Adults & Dislocated Workers, not Youth. This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style. This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style.

10. W.E.Upjohn Institute for Employment Research Approach... Treatment group = Treatment group = those who received intensive or training services and exited WIA in PY2000. Comparison group = Comparison group = those who received core services only, and exited in PY 2000. Thus, this design estimates the incremental impact of intensive/training, above and beyond the receipt of core services. This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style. This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style.

11. W.E.Upjohn Institute for Employment Research Approach... Labor Market Outcomes— Impact estimates will be presented for these Labor Market Outcomes— Employment in 4th post-exit quarter Earnings in 4th post-exit quarter Note: States de-identified until state clearance is granted. Criterion for employment is a minimum of $100 quarterly earnings. This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style. This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style.

12. W.E.Upjohn Institute for Employment Research Preliminary Impact Estimates Outcomes for 6 states: employment & (conditional) earnings in 4th post-exit quarter. Employment, 4th Post-Exit Quarter— Adults: Adults: participation —> positive impacts in 3 states, negative impacts in 3 states. Dislocated Workers: Dislocated Workers: participation —> positive impacts in 4 states, negative in 2 states. This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style. This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style.

13. W.E.Upjohn Institute for Employment Research Impact Estimates... This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style. This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style.

14. W.E.Upjohn Institute for Employment Research Impact Estimates... Earnings, 4th Post-Exit Quarter— Adults: Adults: participation —> positive impacts in 4 states, zero impact in 1 state, negative impact in 1 state. Dislocated Workers: Dislocated Workers: participation —> positive impacts in 2 states, negative in 4 states. This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style. This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style.

15. W.E.Upjohn Institute for Employment Research Impact Estimates... This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style. This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style.

16. W.E.Upjohn Institute for Employment Research Discussion [Still editing from here on…. -DGS] Considerable impact variation across subgroups (i.e., adults, dislocated workers) and across states Small sample sizes, especially for WIA core services Alternative matching techniques, e.g., caliper matching and others Local stratification This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style. This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style.

17. W.E.Upjohn Institute for Employment Research Next Steps Use ES registrant data to construct comparison groups for all 7 current ADARE states (and add new states) Further refine and test alternative statistical matching techniques Estimate impacts using PY2001 WIA participant data & longer-term UI wage records Vet methods and results more widely This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style. This page creates bulleted lists but it also includes animation (click F5 to view). To add animation to other slides, choose Slide Show from the menu bar. You can then choose from Preset Animation features or design a custom animation style.

18. W.E.Upjohn Institute for Employment Research For More Information Kevin Hollenbeck269/385-0431 hollenbeck@upjohninstitute.org Chris King512/471-2186 ctking@uts.cc.utexas.edu Daniel Schroeder512/471-2196 schroed@uts.cc.utexas.edu Use this page as the final slide in your presentation.