1. FY2007 Federal Performance for Managers1June 2006 FY2007 Federal Performance for Managers Overview and Discussion of Performance Measures for FY07

2. FY2007 Federal Performance for Managers2June 2006 FY2007 Performance Measures Technical Assistance Session for Managers Agenda  Performance Measures At-A-Glance  Performance Reporting Cohorts  Key Definitions You Need to Know  Performance Measures Methodologies  Who’s In/Out of the Measure  How the Measures are Calculated  Resources

3. FY2007 Federal Performance for Managers3June 2006 FY2007 Performance Measures At-A-Glance Performance Measures Adult MeasuresYouth Adults Dislocated Workers TradeNEGVeterans Labor Exchange Older Youth Younger Youth All Youth Entered Employment Rate Retention Rate Average Earnings Employment & Certificate Rate Earnings Gain Skill Attainment Rate Youth Diploma Rate Attainment Degree/Certificate Placed in Employment/ Education Rate Literacy/Numeracy Gain

4. FY2007 Federal Performance for Managers4June 2006 FY2007 LABOR EXCHANGE, VETERAN AND TITLE I PERFORMANCE MEASURES COHORTS Performance MeasuresReport Cohort Entered Employment Rate (Adult, DW, OY, LX) Exit Cohort 10/01/2005 – 09/30/2006 Retention Rate (Adult, DW, OY, YY, LX) Exit Cohort 04/01/2005 – 03/31/2006 Average Earnings (Adult, DW, LX) Exit Cohort 04/01/2005 – 03/31/2006 Employment/Certificate Rate (Adult, DW, OY) Exit Cohort 10/01/2005 – 09/30/2006 Earnings Gain (OY) Exit Cohort 04/01/2005 – 03/31/2006 Skill Attainment Rate (YY) Participation/Goals Cohort 04/01/2006 – 03/31/2007 Youth Diploma Rate (YY) Exit Cohort 04/01/2006 – 03/31/2007 Attainment of Degree/ Certificate Rate (All Youth) Exit Cohort 10/01/2005 – 09/30/2006 Placed in Employment/ Education Rate (All Youth) Exit Cohort 10/01/2005 – 09/30/2006 Literacy and Numeracy Gain (All Youth) Participation/Test Cohort 07/01/2006 – 06/30/2007

5. FY2007 Federal Performance for Managers5June 2006 FY2007 Performance Measures A Few Key Definitions  Who is a Participant?  What about Self-Service?  What is a Program Exit?  What is the Exit Quarter?  What is an Exclusionary Exit?  What is Supplemental Employment Data?

6. FY2007 Federal Performance for Managers6June 2006 A Few Key Definitions… Participant - A participant is an individual who is determined eligible to participate in the program and receives a service funded by the program in either a physical location (One-Stop Career Center or affiliate site) or remotely through electronic technologies. Self Service Participant For Labor Exchange, we will begin counting as self-service participants, those who receive only self-service or informational services whether through the virtual career center (i.e. MJQ) or at a physical career center. Labor Exchange self-service participants will be counted in performance calculations.

7. FY2007 Federal Performance for Managers7June 2006 A Few Key Definitions… Program Exit - The term “program exit” means a participant does not receive a service funded by the program or funded by a partner program for 90 consecutive calendar days and is not scheduled for future services. Operational Parameters: Title I participants will be reported as exiters according to the exit date entered by local staff on the Basic tab of MOSES. Local operators are responsible for assuring that participants that receive no service in 90 days are exited. Labor Exchange participants who receive no service for 90 days will be reported as exiters. To the extent possible, it is the responsibility of local staff to assure that a program exit is not entered until all services are completed (e.g. Labor Exchange, NEG, etc.). This will assure that the participant enters the exit cohort for all programs on a common exit date.

8. FY2007 Federal Performance for Managers8June 2006 A Few Key Definitions… Exit Quarter - Represents the calendar quarter in which the date of exit is recorded for the individual. Quarter 1Quarter 2Quarter 3Quarter 4 JANFEBMARAPRMAYJUNJULAUGSEPOCTNOVDEC

9. FY2007 Federal Performance for Managers9June 2006 Example-Quarters A Few Key Definitions… 1 st Quarter after exit Quarter 1Quarter 2Quarter 3Quarter 4 JANFEBMARAPRMAYJUNJULAUGSEPOCTNOVDEC 1 st, 2 nd and 3 rd Quarters After Exit 2 nd Quarter after exit 3 rd Quarter after exit Exit Quarter

10. FY2007 Federal Performance for Managers10June 2006 A Few Key Definitions… Exclusionary Exit: A participant in any of the following categories either at the time of exit or during the three quarter measurement period following the exit quarter, will be excluded from all performance measures: Institutionalized Health/Family Medical Care Deceased Reservist Called to Active Duty Youth Relocated to a Mandated Program Invalid SSN

11. FY2007 Federal Performance for Managers11June 2006 A Few Key Definitions… Supplemental Employment Data: Career Centers may use supplemental sources of data to document a participant’s entry and retention in employment for those participants not covered by wage records; e.g. self-employment, federal jobs, etc. Operational Parameters: Supplemental employment data is entered on MOSES via the Services/Employment Information tab. All supplemental data and methods must be documented and are subject to audit.

12. FY2007 Federal Performance for Managers12June 2006 FY07 Adult Performance Measures (Labor Exchange, Veterans, Trade, Title I Adults, DW, NEG) ■Entered Employment Rate ■Retention Rate ■Average Earnings ■Employment & Certificate Rate ( Title I Adults and Dislocated Workers )

13. FY2007 Federal Performance for Managers13June 2006 Entered Employment Rate Of those who are not employed at the date of participation: # of adults who are employed in the first quarter after the exit quarter divided by # of adults who exit during the quarter FY07 Adult Performance Measures…

14. FY2007 Federal Performance for Managers14June 2006 = 73% 180 adults employed 1 st qtr. after exit – 15* = 165 240 total exiters – 15* = 225 * These 15 exiters were already employed at participation and are excluded from the entire calculation. FY07 Adult Performance Measures… [ADULT ENTERED EMPLOYMENT RATE]

15. FY2007 Federal Performance for Managers15June 2006 FY07 Adult Performance Measures… Of those who are employed in the first quarter after the exit quarter: # of adults who are employed in both the second and third quarters after the exit quarter divided by # of adults who exit during the quarter Employment Retention Rate

16. FY2007 Federal Performance for Managers16June 2006 = 81% 145 adults employed in both 2nd and 3rd qtr after exit 180 adults exited during the quarter Of those who are employed in the 1st quarter after exit: Exiters who were employed at time of participation are included. FY07 Adult Performance Measures… [ADULT EMPLOYMENT RETENTION RATE]

17. FY2007 Federal Performance for Managers17June 2006 FY07 Adult Performance Measures… Of those adults who are employed in the first, second and third quarters* after the exit quarter: Total earnings in the second plus the total earnings in the third quarters after the exit quarter divided by # of adult participants who exit during the quarter Adult Average Earnings * via wage match only

18. FY2007 Federal Performance for Managers18June 2006 = $15,909 Summed wages from quarters 2 and 3 after exit = $1,225,000 77 exiters FY07 Adult Performance Measures… [ADULT AVERAGE EARNINGS] Of those adults who are employed in the 1 st, 2 nd AND 3rd qtr after the exit quarter:

19. FY2007 Federal Performance for Managers19June 2006 Employment & Credential Measure (Adults and Dislocated Workers) Of those who receive training services: # of participants who were employed in the first quarter after the exit quarter and received a credential/certificate by the end of the third quarter after the exit quarter divided by # of participants who exit during the quarter FY07 Adult Performance Measures…

20. FY2007 Federal Performance for Managers20June 2006 = 66% 105 were both employed 1 st qtr after exit AND received a credential/certificate by end of the 3 rd quarter after exit 160 exiters during the quarter Of the Adults and Dislocated Workers who receive training services, FY07 Adult Performance Measures… [ADULT EMPLOYMENT & CERTIFICATE RATE]

21. FY2007 Federal Performance for Managers21June 2006 Carmen enrolled as a dislocated worker in September 2005. She was still working when she registered, but had received notice of layoff. She exited in June 2006. She starts a new job in September 2006. What is her performance outcome for Entered Employment? Carmen's performance outcome for Entered Employment is positive because she was employed during the first quarter after exit, July-September. Remember Notice of Layoff or Military Separation means “unemployed” at participation. FY07 Adult Performance Measures…Entered Employment Rate Quiz Positive Excluded Negative

22. FY2007 Federal Performance for Managers22June 2006 Ben completed an A+ Certification training and earned his certificate while enrolled in WIA. He exits with a job in April, but loses the job in June and does not find another until October. What is Ben’s performance outcome for the Employment and Credential measure? Ben's performance outcome for the Employment and Credential measure is negative because he was not employed during his first quarter after exit, July-September. FY07 Adult Performance Measures…Empl/Credential Rate Quiz Positive Excluded Negative

23. FY2007 Federal Performance for Managers23June 2006 Julie is an adult. She exits in December and does not find work until April. She keeps the job through September. What is her performance outcome for Employment Retention? Julie is excluded from the Employment Retention Measure because she was not employed during her first quarter after exit, January-March. FY07 Adult Performance Measures…Employment Retention Quiz Positive Excluded Negative

24. FY2007 Federal Performance for Managers24June 2006 FY07 Adult Performance Measures…Average Earnings Quiz John is a dislocated worker. He exits in March and finds a new job in July making about $38,000/year and keeps on working. What is John’s performance outcome for Average Earnings measure? John is excluded from the Average Earnings measure because he was not employed during his first quarter after exit. (April – June) Excluded Positive Negative

25. FY2007 Federal Performance for Managers25June 2006 FY07 Older Youth Performance Measures ■Entered Employment Rate ■Retention Rate ■Earnings Gain ■Employment & Certificate Rate

26. FY2007 Federal Performance for Managers26June 2006 FY07 Older Youth Performance Measures… Of those Youth who are not employed at the date of participation and who are either not enrolled in post-secondary education or advanced training in the first quarter after the exit quarter or are employed in the first quarter after the exit quarter: # of youth who are employed in the first quarter after the exit quarter divided by # of youth who exit during the quarter Entered Employment Rate

27. FY2007 Federal Performance for Managers27June 2006 FY07 Older Youth Performance Measures… = 68% 105 youth employed 1st qtr. after exit – 4* - 7** = 94 150 exiters – 4* - 7** = 139 *These 4 youth were excluded from the entire calculation because they were employed at registration. **These 7 youth are not employed and are enrolled in advanced training or post secondary education in the quarter after exit, they are excluded. [OLDER YOUTH ENTERED EMPLOYMENT RATE]

28. FY2007 Federal Performance for Managers28June 2006 Of those youth who are employed in the first quarter after the exit quarter and who are either not enrolled in post-secondary education or advanced training in the third quarter after the exit quarter or are employed in the third quarter after the exit quarter: # of youth who are employed in the third quarter after the exit quarter divided by # of participants who exit during the qtr FY07 Older Youth Performance Measures… Employment Retention Rate

29. FY2007 Federal Performance for Managers29June 2006 FY07 Older Youth Performance Measures… = 82% 75 youth employed in 3 rd qtr. after exit 99 youth employed in 1 st qtr. after exit – 7* = 92 *These 7 youth are excluded because they are enrolled in post secondary education or advanced training and not employed in 3 rd quarter after exit. Of those youth who are employed in the first quarter after the exit quarter and who are either not enrolled in post-secondary education or advanced training in the third quarter after the exit quarter or are employed in the third quarter after the exit quarter: [OLDER YOUTH EMPLOYMENT RETENTION RATE]

30. FY2007 Federal Performance for Managers30June 2006 Earnings Gain Measure Of those youth who are employed in the first qtr after the exit quarter and who are either not enrolled in post-secondary education or advanced training in the third quarter after the exit quarter or are employed in the third quarter after the exit quarter: Post-Earnings in the [second + third qtrs] after the exit qtr minus Pre-Earnings in the [second + third qtrs] prior to the participation qtr divided by # of participants who exit during the quarter FY07 Older Youth Performance Measures…

31. FY2007 Federal Performance for Managers31June 2006 FY07 Older Youth Performance Measures… = $2381 $950,000 - $550,000 = $400,000 240 exiters - 60 * - 12** = 168 *These 60 customers were not employed in the 1 st quarter after exit, so they are excluded. **Supplemental data was used to verify employment for these 12 customers, so they are excluded. Of those youth who are employed in the first qtr after the exit quarter and who are either not enrolled in post-secondary education or advanced training in the third quarter after the exit quarter or are employed in the third quarter after the exit quarter: [OLDER YOUTH EARNINGS GAIN MEASURE]

32. FY2007 Federal Performance for Managers32June 2006 Employment/Education & Credential Measure # of older youth participants who were either employed, in post-secondary education or in advanced training in the first qtr after the exit qtr and received a credential by the end of the third qtr after the exit qtr divided by # of older youth participants who exit during the qtr FY07 Older Youth Performance Measures…

33. FY2007 Federal Performance for Managers33June 2006 FY07 Older Youth Performance Measures… = 63% 120 w/credential by the 3 rd quarter after exit - 25* = 95 150 exiters *These 25 were not employed or enrolled in advanced training or post secondary education in the quarter after exit. [OLDER YOUTH EMPL/CREDENTIAL RATE]

34. FY2007 Federal Performance for Managers34June 2006 FY07 Older Youth Performance Measures… Entered Employment Quiz Excluded Negative Adam, an Older Youth, entered the WIA youth program last year without a job. He exited January 5th with a job. April 3rd he lost his job. What is Adam's outcome for the Entered Employment measure? Adam's outcome for the Entered Employment measure is positive because he was employed during his first quarter after exit, April- June. Positive

35. FY2007 Federal Performance for Managers35June 2006 Miriam, an Older Youth, exits with a job on October 15. She leaves that job December 15, finds another one April 7, and keeps it through the following September. What is her outcome for Employment Retention? FY07 Older Youth Performance Measures… Employment Retention Quiz Miriam's outcome for Employment Retention is excluded because she was not employed during her first quarter after exit, January-March. Excluded Positive Negative

36. FY2007 Federal Performance for Managers36June 2006 Tanya, an Older Youth, entered WIA and completed a GED. She exits in June and travels (jobless) through the summer before beginning classes for her Associate's Degree in October. What is her outcome for the Credential measure? FY07 Older Youth Performance Measures…Empl/Educ & Credential Quiz Excluded Positive Negative Tanya's outcome for Credential is negative because she was not employed during her first quarter after exit, July-September.

37. FY2007 Federal Performance for Managers37June 2006 FY07 Younger Youth Performance Measures ■Skill Attainment Rate ■Diploma or Equivalent Attainment ■Retention Rate

38. FY2007 Federal Performance for Managers38June 2006 Skill Attainment Rate Of all in-school youth and any out-of-school youth assessed to be in need of basic skills, work readiness skills, and/or occupational skills: Total # of goals attained divided by Total # goals set FY07 Younger Youth Performance Measures…

39. FY2007 Federal Performance for Managers39June 2006 FY07 Younger Youth Performance Measures… = 83% 125* Youth Goals with a Status of “Attained” 150 Youth Goals set *These goals are required to have the status changed to “Attained” within one (1) year of being set to be include in the numerator. [YOUNGER YOUTH SKILL ATTAINMENT RATE]

40. FY2007 Federal Performance for Managers40June 2006 Diploma or Equivalent Attainment Rate Of those who participate in the program without a diploma or equivalent: # of participants who attained a secondary school diploma or equivalent by the end of the first quarter after the exit qtr divided by # of participants who exit during the qtr (except those still in secondary school at exit) FY07 Younger Youth Performance Measures…

41. FY2007 Federal Performance for Managers41June 2006 FY07 Younger Youth Performance Measures… = 71% 60 Youth attained Diploma or Equivalent = 60 105 Youth exiters – 20* = 85 *These 20 Younger Youth were still in secondary school at exit, and they are excluded. Of those who participate in the program without a diploma or equivalent and did not exit as Still in secondary school at Exit : [YOUNGER YOUTH DIPLOMA/EQUIVALENCY RATE]

42. FY2007 Federal Performance for Managers42June 2006 Retention Rate # of participants who were found in either post-secondary education, advanced training, employment, military service or a qualified apprenticeship in the third qtr after the exit qtr divided by # of participants who exit during the qtr (except those still in secondary school at exit) FY07 Younger Youth Performance Measures…

43. FY2007 Federal Performance for Managers43June 2006 FY07 Younger Youth Performance Measures… = 71% 85 were retained in 3 rd qtr after exit 120 Youth Exiters [YOUNGER YOUTH RETENTION RATE] Of those younger youth who did not exit as Still in Secondary School at Exit:

44. FY2007 Federal Performance for Managers44June 2006 Tamika exits from WIA in December. She is enrolled in high school and expects to attain her diploma in June. What is her outcome on the Diploma/Equivalent Attainment measure? FY07 Younger Youth Performance Measures… Diploma or Equiv Rate Quiz Tamika's outcome on the Diploma/Equivalent Attainment measure is excluded because she is in secondary school at exit. Excluded Positive Negative

45. FY2007 Federal Performance for Managers45June 2006 Javier exits WIA on September 23. He gets a job right away and works through April 4, when he suddenly decides to quit. What is his outcome on the Retention measure? FY07 Younger Youth Performance Measures… Retention Quiz Javier's outcome on the Retention measure is positive because he was in a positive outcome (employment) during his third quarter after exit, April-June. Excluded Negative Positive

46. FY2007 Federal Performance for Managers46June 2006 FY07 All Youth Performance Measures ■Placement in Employment or Education ■Attainment of a Degree or Certificate ■Literacy and Numeracy Gains

47. FY2007 Federal Performance for Managers47June 2006 Attainment of a Degree or Certificate Of those enrolled in education (at the date of participation or at any point during the program): # of youth participants who attain a diploma, GED or certificate by the end of the third quarter after the exit quarter divided by # of youth participants who exit during the quarter FY07 All Youth Performance Measures…

48. FY2007 Federal Performance for Managers48June 2006 FY07 All Youth Performance Measures… = 77% 120 Youth attained a diploma or certificate by 3 rd qtr after exit 156 Youth exiters in Education Of those enrolled in education (at the date of participation or at any point during the program): [YOUTH ATTAINMENT OF DEGREE OR CERTIFCATE]

49. FY2007 Federal Performance for Managers49June 2006 Placement in Employment or Education Of those who are not in post-secondary education or employment (including the military) at the date of participation: # of youth participants who are in employment (including military) or enrolled in post-secondary education and/or advanced training/ occupational skills training in the first quarter after the exit quarter divided by # of youth participants who exit during the quarter FY07 All Youth Performance Measures…

50. FY2007 Federal Performance for Managers50June 2006 FY07 All Youth Performance Measures… = 77% 142 Youth in qualifying outcomes in 1st qtr. after exit 185 Youth Exiters Excluded from this measure are those that were employed, in the military, or in post-secondary education at date of participation. [YOUTH PLACEMENT IN EMPLOYMENT OR EDUCATION]

51. FY2007 Federal Performance for Managers51June 2006 Literacy and Numeracy Gains Of those out-of-school youth who are basic skills deficient (and enroll on or after July 1, 2006): # of youth participants who increase one or more educational functioning levels divided by # of youth participants who have completed a year in the program (i.e., one year from the date of first youth program service) plus the # of youth participants who exit before completing a year in the program FY07 All Youth Performance Measures…

52. FY2007 Federal Performance for Managers52June 2006 High Adult Secondary Education Low Adult Secondary Education High Intermediate Basic Education Low Intermediate Basic Education Beginning Basic Education Beginning ABE Literacy Tests such as TABE, CASAS and ABLE provide numerical scores that correspond with one of six Adult Basic Education educational functioning levels. Adult Basic Education (ABE) Educational Functioning Levels FY07 All Youth Performance Measures…

53. FY2007 Federal Performance for Managers53June 2006 High Advanced ESL Low Advanced ESL High Intermediate ESL Low Intermediate ESL Beginning ESL Beginning ESL Literacy Tests such as CASAS, Oral BEST, BEST Plus and Literacy BEST provide numerical scores that correspond with one of six ESL educational functioning levels. English As A Second Language (ESL) Ed Functioning Levels FY07 All Youth Performance Measures…

54. FY2007 Federal Performance for Managers54June 2006 FY07 All Youth Performance Measures… = 56% 68 Youth who advance one or more educational levels 100 Youth who completed a year + 22 who exited before completing a year = 122 [YOUTH LITERACY AND NUMERACY GAIN]

55. FY2007 Federal Performance for Managers55June 2006 Youth #2 EFL 1 EFL 2 EFL 3 EFL 4 EFL 5 Youth #1Youth #3Youth #4 Examples of EFL Improvement Pre-test to post-test, within one program year FY07 All Youth Performance Measures…

56. FY2007 Federal Performance for Managers56June 2006 FY07 All Youth Performance Measures… Placement Measure Certificate Measure Literacy Gain Measure Yes In Out ISY—has a summer job In Out ISY--earns HS diploma but nothing else In UE/OSY/BSD-- increases basic skills & earns GED In Out OSY, Not BSD -- gets job Out In Employed OSY/BSD-- increases basic skills In OSY/BSD--starts GED but drops out; no skill gain Youth Performance Measures Different Cohorts NEGATIVEPOSITIVE

57. FY2007 Federal Performance for Managers57June 2006 ADDITIONAL RESOURCES Workforce Issuance 06-35 - Performance Review Tool TEGL 17-05: Common Measures Policy for the Employment and Training Administration (ETA) http://wdr.doleta.gov/directives/corr_doc.cfm?DOCN=2195 PPV/SPRA Performance Enhancement Project On-Line Tutorials http://www.spra.com/PEP/online_intro.shtml (not 100% up to date for FY2007)