1. Program Reporting Changes and Requirements for the WIA Quarterly and Annual Reports ETA 9090 / 9091

2. 2 Agenda Overview Highlights –What’s New in ETA 9090 / 9091 Common Measures Key Concepts & Considerations 2

3. 3 Agenda (con’t) ETA 9090 Reporting Instructions ETA 9091 Reporting Instructions Quiz - time permitting

4. 4 Overview

5. 5 Common Performance Measures Policy for ETA’s Performance Accountability System - TEGL 17-05 Common Measures Policy - TEGL 28-04, April 15, 2004 Revised Reporting Instructions for PY05 & PY06 1 st (2 nd ?) Quarter PY05 results Background

6. 6 Recent Revisions to Common Measures Policy TEGL 17-05 Rescinds and replaces: –TEGL 28-04 –TEGL 7-99 core performance measures –Customer Satisfaction measures in TEGL 6-00 and TEGL 6-00 Ch. 1

7. 7 Current Reporting Requirements TEGL 28-04 –Common Measures –Participation & Exit –Implementation began 7/1/05 Revised Reporting Instructions –WI/A Quarterly Report (ETA 9090) –WIA Annual Report (ETA 9091)

8. 8 Illustration: Participation and Exit ParticipationNo Services Exit Date Participation Date Eligible and Receives Service End of 90- Day Period Last Service

9. 9 Key changes in TEGL 17-05 Program Participation Adult Earnings Measure Literacy and Numeracy Gain –Testing Interval CM definition of Certificate replaces Credential beginning July 1, 2006

10. 10 Program Participation Participant includes: –Those who receive services remotely or through electronic technologies –Individuals that use self-directed tools for job search – Does not apply to WIA Youth program No longer exclude self-directed job search that does not result in referral to a job

11. 11 (PY 2006) AVERAGE EARNINGS Definition: For those employed in the 1 st, 2 nd and 3 rd quarter after exit, average six month earnings in the 2 nd and 3 rd quarter after exit Calculation: Of those employed in the 1 st 2 nd and 3 rd quarters after exit… Earnings from the 2 nd and 3 rd quarter after exit Number who exit during the quarter

12. 12 Literacy Numeracy Based on date of first youth service –No longer tied to date of participation Testing Interval –Youth participants may be included for up to 3 years if they remain Basic Skills Deficient

13. 13 1 st (2 nd ?) Quarter PY05 results –Failure to report –Inaccurate data –More work to be done

14. 14 1 st Quarter PY05 observations Total Adults ( self services only ) > Total Adult Participants –Should be smaller number Total Youth (14-21) Participants and Exiters –Should equal sum of Younger Youth and Older Youth –Should equal sum of In-School and Out-of-School Current Quarter (most recent) roughly equal to ¼ Cumulative 4-Quarter

15. 15 Highlights

16. 16 What's New in the WIA Quarterly Report - ETA 9090

17. 17 Uses Common Measures concepts –Participant –Exit Common Measures Definitions –Adult, DW, NEG Entered Employment Employment Retention Earnings Increase (PY05) Average Earnings (PY06) –Youth (14-21) Placement in Employment or Education Attainment of Degree or Certificate Literacy and Numeracy Gains –(opt. PY05, req. PY06) Performance Measures

18. 18 More changes Total Participants / Total Exiters breakouts –WIA Adults (self services only) –NEG –Received Training Services Breakout for Total Participants only –In-School / Out-of-School Rolling four quarters Customer Satisfaction not reported

19. 19 What's New in the WIA Annual Report ETA 9091

20. 20 Performance Measures Uses Common Measures concepts –Participant –Exit Common Measures Definitions –Adult, DW Entered Employment Employment Retention Earnings Increase (PY05) Average Earnings (PY06)

21. 21 Tables Additional table H 1 for Youth –Common Measures Placement in Employment or Education Attainment of Degree or Certificate Literacy or Numeracy Gains Table M –Total Participants and Total Exiters breakouts WIA Adult (self services only) In-school/out-of-school Table O –Additional lines for 3 Youth Common Measures

22. Common Measures Key Concepts & Considerations 22

23. 23 Who’s reported in participant counts? All individuals who have been determined eligible and receive a service, including self-service and informational activities, in either a physical location (One-Stop career center or affiliate site) or remotely through electronic technologies

24. 24 Who’s included in WIA performance measures calculations? All participants who receive a core, intensive, or training service who exit the program are to be included in the performance measures calculations, except that WIA Section 136 excludes WIA adult and dislocated worker program participants who only receive self- service or informational activities

25. 25 What are Self-Service and Informational Activities? Core services that are made available and accessible to the general public; that are designed to inform and educate individuals about the labor market, their employment strengths and weaknesses, and the range of services appropriate to their situation; and that do not require significant staff involvement with the individual in terms of resources or time. Ref. 20 CFR 666.140(a)(2)

26. 26 ETA’s Interpretation of Self-Service and Informational Activities Self-Service = participants serve themselves in accessing workforce investment system information in either a physical location or remotely through electronic technologies Informational activities = may include self-services and staff assisted core services that are designed to inform and educate a participant about the labor market and to enable a participant to identify his or her individual employment strengths, weaknesses, and the range of services appropriate for the individual.

27. 27 Significant Staff Involvement Any assistance provided by staff beyond the informational activities regardless of the length of time involved in providing such assistance Includes a staff member’s assessment of a participant’s skills, education, or career objectives in order to assist participants in any of the following: –Decide on appropriate next steps in search of employment, training, and related services –Assess their personal barriers to employment –Access other related services necessary to enhance their employability and individual employment related needs.

28. 28 Informational Activities When a staff member provides a participant with readily available information that does not require an assessment of the participant’s skills, education, or career objectives, the participant is a recipient of informational activities. This includes: –Labor market trends –Unemployment rate information –Businesses that are hiring or reducing workforce –Info on high growth industries –Info on demand occupations

29. 29 Participant vs. WIA Performance ServiceParticipantWIA Perf CORECORE Self ServiceYesNo Informational Activities Self ServiceYesNo Staff Assisted Readily Available InformationYesNo Requires Staff AssessmentYes IntensiveYes TrainingYes Follow-upNo

30. 30 Participants CoreIntensive Training Self-Service Info. Activities

31. 31 Self-Service Info. Activities Self-Service Significant Staff Involve Core Training Intensive WIA PERF CALC Core

32. 32 2 Groups of Participants/Exiters Participants –New participants after 7/1/05 –Carryover participants from PY04 Exiters –New Exiters after 7/1/2005 –Reach Back period Don’t recalculate prior WIA exit dates

33. 33 Other Considerations What is level of integration? –Required and partner programs with-in One-Stops? –State MIS –How are services tracked among partner programs? Converting Data from WIASRD records Determining participation and “soft” exit dates Services sorted by –Date –Program –Routine? –Follow-up? –Commence Participation? –Extend Exit?

34. 34 Scenarios The following scenarios could occur in the delivery of services. While these scenarios do not cover every situation states and local workforce investment areas may encounter, they can be used to determine the level of staff involvement that will trigger a period of participation and a participant’s inclusion in WIA performance measures calculations.

35. 35 An individual comes into the One-Stop Career Center believing that he can access all government services. He asks for directions to the Motor Vehicle Administration. The receptionist gives the information and the individual leaves. This person is not a participant and will not appear in any counts for performance measures. The information that he sought did not directly relate to the services provided by the One-Stop Career Center or the workforce investment system Scenario 1

36. 36 An individual comes into the One-Stop Career Center and accesses the computers in the self- service area, using a system user account identification. The self-service area is jointly funded by WIA and Wagner-Peyser Act funds. She takes some notes, scribbles the notes on a sheet of paper near the computer, and leaves the facility. Scenario 2

37. 37 This person may be a participant but it is difficult to determine the nature of the information that the individual has accessed or written down, as there had been no staff interaction with this individual. In this case the individual is a participant because she used her system user account identification to access a saved job search on the state’s Internet job bank system and copied employer contact information. This participant’s self- referral activity was captured by the state’s datasystem and a service transaction was recorded. Because the state job bank is funded by both WIA and the Wagner-Peyser Act, this individual is included in the WIA and Wagner-Peyser Act participant counts, but not included in the WIA performance measures calculation because the participant received only self-services. This person is included in the Wagner-Peyser Act performance calculations.

38. 38 Scenario 3 An individual accesses job listings from his home computer and uses the resume writer software that is on the local One-Stop Web site. The Web site is jointly funded by the Wagner-Peyser Act and WIA. The individual sends his resume in response to a job listing. This person is a participant as he has used workforce investment system information and services to assist his job search. This individual is included in Wagner- Peyser Act and WIA participant counts, but not included in the WIA performance measures calculation because the participant received only self-services. This person is counted in the Wagner-Peyser Act performance calculations.

39. 39 Scenario 4 An individual accesses job listings from a computer at the local library and uses her account identification that she established with the workforce investment system to log on to the computer. The individual sends her resume in response to a job listing for which she believes she will be a suitable candidate. This person is a participant as she has used workforce investment system services to assist her job search. This individual is included as a Wagner-Peyser Act and WIA participant, but not included in the WIA performance measures calculation because the participant received only self-services. This person is counted in the Wagner-Peyser Act performance calculations.

40. 40 Scenario 5 An individual has figured out how to access workforce investment system information from his cell phone that includes a personal data assistant. He thinks that he has found the job of his dreams on the state job bank and posts his resume with the employer. This person is a participant as he has used workforce investment system services to assist his job search. This individual is included as a Wagner-Peyser Act and WIA participant, but not included in the WIA performance calculations because the individual received only self-services. This person is counted in the Wagner-Peyser Act performance calculations

41. 41 Scenario 6 An individual arrives at the One-Stop Career Center and, familiar with computers, she decides to go to the self-service section and search the job listings. Later, she wants to review occupational trends information. She pulls up some information, but is not quite sure how the trends were calculated. She questions the staff about the statistics and asks where she might find additional information on the need for workers in this field. The staff member answers the question and directs the individual to the Career Voyages, another self-service Web site where more information and other helpful links are available.

42. 42 Scenario 6 con’t. This person is a participant as she has used workforce investment system information and self-service electronic tools jointly funded by WIA and Wagner- Peyser Act funds to assist her job search. This individual is included as a Wagner-Peyser Act and WIA participant, but not included in the WIA performance calculations because the participant received only self- service and informational activities. This individual did not receive services requiring significant staff involvement because the staff person provided readily available information and instructions about tools and services in the research room without having to assess the individual’s skills, education, or career objectives. This person is counted in the Wagner-Peyser Act performance calculations.

43. 43 Scenario 7 An individual arrives at a One-Stop Career Center looking for information on the availability of jobs in a specific field. The staff member asks the individual several questions to determine the individual’s previous work experience, education, and training experience in the specific industry and occupation. The staff member assists the individual to search for employment on a resource room computer and provides him with the local occupational and economic trend information.

44. 44 Scenario 7 con’t. This person is a participant as he has used workforce investment system staff-assisted service to assist his job search. This individual is included as a Wagner-Peyser Act and WIA participant, and must be included in the WIA performance measures calculation because the participant received more than self-service and informational activities. The staff member performed an assessment of the participant’s work readiness for employment in the local area and assisted in the participant’s job search. This person is also counted in the Wagner-Peyser Act performance calculations.

45. 45 Scenario 8 An individual arrives at the One-Stop Career Center and wants to know the various services that are available at the One-Stop Career Center. The One-Stop Career Center specialist gives the individual an orientation to the One-Stop Career Center and advises the individual of what services are available. The staff member sets the individual up at a computer and shows him how to use the equipment and to access the job search tools. Later, the individual participates in an assessment interview and begins to develop an employment plan.

46. 46 Scenario 8 con’t. This individual is a participant as she has used workforce investment system services to assist her job search. This individual is included as a Wagner-Peyser Act and WIA participant. She is included in the WIA performance measures calculations because her assessment interview and the development of an employment plan required significant staff involvement. This individual should also be included in the Wagner-Peyser Act performance calculations.

47. 47 Workforce Investment Act Quarterly Report: General Reporting Instructions and ETA Form 9090

48. 48 ETA 9090 Reporting Instructions General Instructions Aggregate Performance data –Determine levels of participation and accomplishments Time Periods –Most recent quarter –Most recent four quarter period

49. 49 WIA Adult WIA Dislocated Worker Program WIA Youth Program National Emergency Grant Covered Programs

50. 50 Separate Breakout on ETA 9090 Total Participant Total Exiters Entered Employment Rate Employment Retention Rate Six Months Earnings Increase National Emergency Grants

51. 51 Statewide activities –Include activities that involve enrollment of individuals –Exclude activities that do not involve enrollment of Individuals –Indirect services (e.g. research or evaluation) –Highly specialized services (incumbent worker training) Covered Programs (con’t)

52. 52 Due Date Report QuarterDue Dates January – MarchMay 15 th April – JuneAugust 14 th July-SeptemberNovember 14 th October – DecemberFebruary 14th Note: When due date falls on a Saturday or Sunday the quarterly report is due the Friday before.

53. 53 WIA Quarterly Report Format ETA 9090 Four Columns –Performance Items –Program Group Performance Items broken into subpopulations –Current Quarter (most recent) –Cumulative 4-Quarter Reporting Period

54. 54 WIA Quarterly Report Format (con’t) Performance Items –Total Participants –Total Exiters –Performance Measures

55. 55 Total Participants Total Adult Participants –Total Adult Participants (self services only) –WIA Adults –WIA Dislocated Workers –National Emergency Grant Received Training Services –WIA Adults –WIA Dislocated Workers Total Youth (14-21) Participants –Younger Youth (14-18) –Older Youth (19-21) –Out-of-School –In-School

56. 56 Total Exiters Total Adult Participants –Total Adult Participants (self services only) –WIA Adults –WIA Dislocated Workers –National Emergency Grant Total Youth (14-21) Participants –Younger Youth (14-18) –Older Youth (19-21) –Out-of-School –In-School

57. 57 Performance Measures 3 Youth Common Measures –Placement in Employment or Education –Attainment of Degree or Certificate –Literacy or Numeracy Gains 2 WIA Younger Youth Measures –Diploma or Equivalent –Skill Attainment

58. 58 Performance Measures (con’t) Entered Employment Rate (Ad, DW, OY,NEG) Employment & Credential (Ad, DW) or Credential Rates (OY) Retention Rate (Ad, DW, OY, YY, NEG) Six-Months Earnings (Ad, DW, OY,NEG)

59. 59 Rationale for Rolling Quarter Methodology Issue: We have program years, state and federal fiscal years, and calendar years, making communication of "annual results" to customers and stakeholders seem convoluted This methodology allows us to speak in terms of four quarters of data (i.e., annual data) for any workforce program, whether the program is based on a program year or fiscal year

60. 60 The Rolling Four-Quarter Methodology Explained Take a look at the following, which could be considered a “rolling four-letter alphabet” Notice that the first letter “drops off” so another letter can be added, creating the perception that the letters are “rolling” ABCDEFGHI ABCDEFGHI

61. 61 Rolling Four-Quarter Methodology (cont’d) Rolling quarters operate much the same way: –We must have four quarters at any given time so we can communicate in terms of “annual” results –The earliest quarter drops off so the next (most recent) quarter can be added –As this process continues, it creates a “rolling quarter” perception Jan – Mar Apr – June July – Sept Oct- Dec Jan – Mar Apr – June July – Sept Oct – Dec Jan - Mar Jan – Mar Apr – June July – Sept Oct- Dec Jan – Mar Apr – June July – Sept Oct – Dec Jan - Mar Jan – Mar Apr – June July – Sept Oct- Dec Jan – Mar Apr – June July – Sept Oct – Dec Jan - Mar

62. 62 Rolling 4 Quarters Total Participants PY05 1 st Qtr PY06 shift last row one column to right 200420052006 Jan-MarApr-JuneJuly-SeptOct-DecJan-MarApr-JuneJuly-SeptOct-DecJan-MarApr-JuneJuly-Sept Total Participants14-Nov Total Participants14-Feb Total Participants15-May Total Participants14-Aug

63. 63 Rolling 4 Quarters Total Exiters PY05 1 st Qtr PY06 shift last row one column to right 14-Aug Total Exiters 15-May Total Exiters 14-Feb Total Exiters 14-Nov Total Exiters Oct-DecJuly-SeptApr-JuneJan-MarOct-DecJuly-SeptApr-JuneJan-MarOct-DecJuly-SeptApr-JuneJan-Mar 200620052004

64. 64 Placement in Employment or Education Time Periods For the Quarter ending Sept 2005 Each following quarter shift 1 column to the right 200420052006 Jan-MarApr-JuneJuly-SeptOct-DecJan-MarApr-JuneJuly-SeptOct-DecJan-MarApr-JuneJuly-SeptOct-Dec Exit Cohort Employment or Education Wage Delay for Employment outcome 14-Nov

65. 65 For the Quarter ending Sept 2005 Each following quarter - shift one quarter to right 2003200420052006 July-SeptOct-DecJan-MarApr-JuneJuly-SeptOct-DecJan-MarApr-JuneJuly-SeptOct-DecJan-MarApr-June Exit Cohort Employment Qtr Retention & Earnings 2 Qtr Wage Record Delay 14-Nov5 Retention & Earning Increase Time periods

66. 66 ETA 9091 Reporting Instructions

67. 67 Workforce Investment Act Annual Report: General Reporting Instructions and ETA Form 9091 General Instructions Emulate Private Sector’s Report to Stockholders –Meets federal reporting requirements WIA 136(d)(1) and WIA 185(d) –Allows flexibility to showcase unique programs, strategies, and accomplishments.

68. 68 General Instructions (con’t) Three required elements (WIA Sec. 136) –State & Local performance data Perf data must be based on WIASRD data –Status of state evaluation activities –Cost of WIA activities relative to the effect of those activities on the performance of participants

69. 69 Covered Programs WIA Adult Program WIA Dislocated Worker Program WIA Youth Program

70. 70 Covered Programs (con’t) Performance information on NEG participants will not be included – Reported on Quarterly Report Statewide activities –Include activities that involve enrollment of individuals –Exclude activities that do not involve enrollment of Individuals –Indirect services (e.g. research or evaluation) –Highly specialized services (e.g. incumbent worker training)

71. 71 Annual Report Specifications Two Required Sections Narrative Section Tables A - O

72. 72 Narrative Section Narrative Section - must include: –Cost of WIA activities relative to the effect on the performance of participants –Description of state evaluations of WIA activities Questions the evaluation will/did address Description of methodology Info on timing of feedback and deliverables

73. 73 Narrative (con’t) Narrative Section may include: –success stories, –messages from Governor, –info on State WIB members, –market analysis, strategies for improvement, program’s effects on major industries, etc.

74. 74 Table Section State Wide Performance data Tables A-N Local Area Performance date Table 0, one for each local area Don’t forget Customer Satisfaction!! Results are reported in Table A

75. 75 American Customer Satisfaction Index (ASCI) Required Methodology for Statewide Results Who Will be surveyed? Participants – WIA Title IB exiters Employers who received a substantial service that involved personal contact by WIA staff Time frame for surveys Participants – within 60 days of expected last date of service Employers – ASAP after completion of service, but no later than 60 days after completion of service

76. 76 Required Methodology for Statewide Results (con’t) Telephone Surveys Random Samples –500 completed surveys –70% response rate 3 required questions –10 point scale 1= Very Dissatisfied, 10 = Very Satisfied –Weighted Average results

77. 77 Parameters for Collecting Customer Satisfaction information to Measure Local WIB Performance Purchase license and use ASCI methodology Develop alternative measurement instruments and methodology –objective, quantifiable, and measurable results –Consistently applied among all local WIBs –Collected in uniform and Equitable manner –Conform to widely accepted methodological and statistical criteria for measuring customer satisfaction Additional Info on Customer Satisfaction requirements http://www.doleta.gov/Performance/

78. 78 Due Date Program YearDue Dates PY 2005October 1, 2006 PY 2006October 1, 2007 PY 2007October 1, 2008 Note: When due date falls on a Saturday or Sunday the quarterly report is due the Friday before.

79. 79 Submission Procedures Tables A-O submitted through Enterprise Business Support System (EBSS) 3 copies of narrative mailed to ETA and/or Additional electronic copy for posting on ETA’s website (WIA.AR@dol.gov) –WordPerfect, Microsoft Word, Acrobat or other commonly used format. Must complete Report Validation prior to submitting the WIA Annual Report