1. Workforce Investment Act Youth Measures Jacksonville Youth Workshop January 15, 2003 Agency for Workforce Innovation

2. Where are Workforce Investment Act Youth Measures Reported?  Red & Green Report quarterly.  Monthly Management Report monthly  USDOL Quarterly Summary Report ETA 9090  USDOL Annual Report ETA 9091

3. Where Does the Data Come From? Records entered into the state’s WIA Management Information System (MIS) for Customer Group “21”, Younger Youth participants, and Customer Group “22”, Older Youth participants. Unemployment Insurance wage records as reported by FETPIP FETPIP AWI RWB DATA

4. Color Code This presentation will be posted in color at http://www.floridajobs.org/pdg/wia/Default.htm  A red bullet usually means Red/Green, MMR  A Blue bullet usually means a USDOL measure Green text usually means Red/Green and/or MMR Measure/Report Text in this blue font is usually FETPIP Supplied Data or a USDOL Measure/Report Text in this red font is usually data entered into the administrative record by the RWBs

5. Older vs Younger Youth  Determined by age at registration  Once established – does NOT change  Older Youth are 19 and not yet 22 years old at registration  Younger Youth are 14 and not yet 19 years old at registration

6. Workforce Investment Act (WIA) Youth Performance Measures  Older Youth Entered Employment Rate  Older Youth Retention Rate  Older Youth Wage Change Rate  Older Youth Credential Rate  Younger Youth Retention Rate

7. Workforce Investment Act (WIA) Youth Performance Measures  Younger Youth Diploma Rate  Youth Positive Outcome Rate  Youth Goal Attainment  Rate

8. Exits during the BLUE quarters are reported. GREEN quarters are the quarters of the outcomes counted for the LATEST blue quarter. The PINK quarters are the quarters of the outcome information for the FIRST blue quarter. [Outcome quarter information for the other BLUE quarters have the same time relationships to exit as those of the FIRST & LATEST blue quarters.] Youth Measure Cohorts & Data Collection Periods PY2001-2002 Older Youth Wage Change RateBLUEGREEN Older Youth Wage Change Rate – Exiters from BLUE, Outcomes from PINK & GREEN Expanded example with one BLUE exit quarter per line Reg-3Reg-2Jul 00Oct 00Jan 01Apr 01 Jul 01 Oct 01Jan 02 Apr 02 Jul 02 Reg-3Reg-2Oct 00Jan 01Apr 01 Jul 01 Oct 01Jan 02 Apr 02 Jul 02 Reg-3Reg-2Jan 01Apr 01 Jul 01 Oct 01Jan 02 Apr 02 Jul 02 Reg-3Reg-2Apr 01 Jul 01 Oct 01Jan 02 Apr 02 Jul 02 Reg-3Reg-2Jul 00Oct 00Jan 01Apr 01 Jul 01 Oct 01Jan 02 Apr 02 Jul 02 Note: Reg-3 means the 3 rd quarter before the quarter of registration in WIA

9. Younger Youth Goal Attainments – Participants from BLUE, attainments during BLUE- data: [ Administrative Record] Older Youth Entered Employment Rate –Outcome from GREEN - data: (FETPIP) [ Exclusions Administrative Record] Oct 00Jan 01Apr 01Jul 01Oct 01Jan 02Apr 02Jul 02 Older Youth Retention – Outcome from GREEN - data: (FETPIP) [ Exclusions Administrative Record] Oct 00Jan 01Apr 01Jul 01Oct 01Jan 02Apr 02Jul 02 Older Youth Wage Change Rate – Outcome from PINK & GREEN - data: (FETPIP) [ Exclusions Administrative Record] Younger Youth Retention Rate – Outcome from GREEN - data: [ Administrative Record] & (FETPIP) Younger Youth Diploma Rate – Outcome from participation, BLUE & GREEN - data: [ Administrative Record] Younger Youth Positive Outcome Rate – Outcome during participation reported through exit or BLUE - data: [ Administrative Record] Oct 00Jan 01Apr 01Jul 01Oct 01Jan 02Apr 02Jul 02 Oct 00Jan 01Apr 01Jul 01Oct 01Jan 02Apr 02Jul 02 Youth Measure Cohorts & Data Collection Periods PY2001-2002 Exits during the BLUE quarters are reported. GREEN quarters are the quarters of the outcomes counted for the LATEST blue quarter. The PINK quarters are the quarters of the outcome information for the FIRST blue quarter. [Outcome quarter information for the other BLUE quarters have the same time relationships to exit as those of the FIRST & LATEST blue quarters.] Reg-3Reg-2Jul 00Oct 00Jan 01Apr 01 Jul 01 Oct 01Jan 02 Apr 02 Jul 02 Oct 00Jan 01Apr 01Jul 01Oct 01Jan 02Apr 02Jul 02 Oct 00Jan 01Apr 01Jul 01Oct 01Jan 02Apr 02Jul 02 Older Youth Credential Rate – Outcome through participation & GREEN - data: [ Administrative Record] & (FETPIP) Oct 00Jan 01Apr 01Jul 01Oct 01Jan 02Apr 02Jul 02

10. Global WIA Exclusions Exclusions from all performance measures:  Institutionalized at exit  Health/medical exits  Deceased  Reservists to Active Duty  National Emergency Grants only

11. Entered Employment Rate Number Employed in Quarter after Exit divided by Number of Exiters  Wage records the primary data source  Supplemental data may be used for persons not found in wage records (Florida does not use)  Exclude older youth employed at registration or in advanced training/ postsecondary education (but not employed) in quarter after exit.

12. YES NO YES Entered Employment Rate Older Youth Excluded Numerator Denominator Excluded YES Advanced Training In Advanced Training or Postsecondary Education in Quarter After Exit NO Employed in Quarter After Exit NO Employed at Registration Employed at Registration ?

13. Employment Retention  Based on those employed in quarter after exit  Wage records the primary data source  Excludes older youth in advanced training/ postsecondary education (but not employed) in 3rd quarter after exit. Of those employed in the quarter after exit: Number Employed in 3rd Quarter after Exit divided by Number Employed in Quarter after Exit

14. YES NO YES NO YES Employment Retention Rate Older Youth Employed Employed Quarter After Exit? Excluded Employed Employed 3rd Quarter After Exit Numerator Denominator Excluded NO Advanced Training In Advanced Training or Postsecondary Education in 3rd Quarter After Exit

15. Earnings Change Older Youth  Wage records only  Exclude older youth in advanced training/ postsec. education (but not employed) in 3rd quarter after exit. divided by Number employed in quarter after exit Of those employed in the quarter after exit: minus Earnings in 2nd and 3rd Quarters after Exit Earnings in 2nd and 3rd Quarters before Registration

16. Advanced Training In Advanced Training or Postsecondary Education in 3rd Quarter After Exit NO Earnings Change Older Youth Employed Employed Quarter After Exit? Included NO Employed Employed 3rd Quarter After Exit YES NO YES Yes YES Excluded

17. Older Youth Credential Rate Older Youth Credential Rate  Flexible definition of credential  Standard credentials  Certificates that recognize the successful completion of training services ….  Credential attainment by end of 3rd qtr after exit  Positive outcome is attain credential and:  Enter employment or in advanced training/postsec. educ. in quarter after exit (older youth)  Includes:  All older youth

18. attain credential Number who attain credential and enter employment [ enter employment or advanced trainingpostsecondary education are in advanced training or postsecondary education in the quarter after exit] divided by Number of exiters Credential--Older Youth

19. YES Credential Rate--Older Youth Employed Employed in Quarter After Exit NO YES Numerator NO Advanced Training In Advanced Training or Postsecondary Education Postsecondary Education in Quarter After Exit Attained a credential Denominator

20. Younger Youth Retention  In one or more qualifying outcomes in 3rd quarter after exit  Postsecondary education  Advanced training  Military service  Qualified apprenticeship  Unsubsidized employment  Exclude youth in secondary school at exit  Not related to outcome in quarter after exit

21. YES NO Younger Youth Retention Rate In Secondary School at Exit In Secondary School at Exit ? Numerator Denominator Excluded Retained Retained in 3rd Quarter After Exit

22. HS Diploma/GED Attainment  Diploma can be attained during participation or by end of quarter after exit  Exclude youth still in secondary school at exit Of youth who were not HS grads at registration: attained diploma Number of exiters who attained diploma divided by exiters not still in school at exit Number of exiters not still in school at exit

23. YES NO YES NO Younger Youth Diploma Rate HS grad at registration HS grad at registration ? Excluded Attained Diploma Attained Diploma ? Numerator Denominator Excluded YES In Secondary School at Exit In Secondary School at Exit ?

24. WIA Younger Youth Positive Outcome Rate In one or more qualifying outcomes at exit In one or more qualifying outcomes at exit  Postsecondary education  Advanced training  Military service  Qualified apprenticeship  Remained in secondary school  Entered Employment on the administrative record  Earned a HS diploma or equivalent during participation divided by divided by The number of younger youth exiters

25. WIA Younger Youth Goal Attainment Rate The percentage of basic skills, work readiness and occupational skills goals attained out of the number of goals due or required to be set in the report period Calculation: # of goals attained during a report period # of goals due or required to be set in a report period Data comes from the WIA MIS

26. WIA Younger Youth Goal Attainment Rate Groups of Goals Counted 1.Goals attained timely 2.Goals 1 year old and not attained 3.Goals not 1 year old and not attained at exit 4.Goals that should have been set and were not

27. Calculating the WIA Younger Youth Goal Attainment Rate Which goals count in the Numerator:  All goals attained in a report period  Exception: Goals attained late

28. Calculating the WIA Younger Youth Goal Attainment Rate Goals Counted In The Denominator:  Goals attained timely in the period (before due date <365 days) [Numerator]; +  Goals due, not attained in period; +  Exiters with goal set dates (less than 1 year) before exit and not attained at exit;+

29. Calculating the WIA Younger Youth Goal Attainment Rate Denominator factors continued:  Basic Skills Deficient carryovers without BS goal set between 7/01/01-6/30/02; +  BS deficient youth registered after 06/30/02 without BS goal set date = registration date; +

30. Calculating the WIA Younger Youth Goal Attainment Rate Denominator factors continued:  BSD on anniversary date; without BS goal set date = anniversary (month/day of registration); +  Youth Not Basic Skill Deficient; without goal set date = registration or anniversary date. Note: Permissible to set a Basic Skills goal for non-basic skills deficient youth

31. Strategies for Improving the Younger Youth Goal Attainment Rate Record a goal for all younger youth Record a goal for all younger youth with set date = Registration date with set date = Registration date Set first goal within 30 days of Registration Set first goal within 30 days of Registration Set only 3 goals per participation year Set only 3 goals per participation year Set Basic Skills as first goal set for youth who are Basic Skills Deficient Set Basic Skills as first goal set for youth who are Basic Skills Deficient Set a Basic Skills goal until youth is no longer basic skills deficient Set a Basic Skills goal until youth is no longer basic skills deficient

32. Strategies for Improving the Younger Youth Goal Attainment Rate (Cont.) Set a goal for every youth with a set date = anniversary date. [Goals set at anniversary (set date = anniversary date) can be set within 30 days before/after the anniversary date] Set a goal for every youth with a set date = anniversary date. [Goals set at anniversary (set date = anniversary date) can be set within 30 days before/after the anniversary date] Younger Youth should have an open goal at all times (Exception: up to 90 days without a goal may occur) Younger Youth should have an open goal at all times (Exception: up to 90 days without a goal may occur) If 3 goals set and attained in participation year, use ISS objectives to guide planning and skill tracking until anniversary date If 3 goals set and attained in participation year, use ISS objectives to guide planning and skill tracking until anniversary date

33. Strategies for Improving the Younger Youth Goal Attainment Rate (Cont.) Enter goal attainment data accurately and timely Enter goal attainment data accurately and timely Report all younger youth goal attainments as they occur Report all younger youth goal attainments as they occur Enter attainment/non-attainment within 365 days of the goal set date Enter attainment/non-attainment within 365 days of the goal set date Note: Attainments not recorded in the MIS timely are not counted in the performance measure.

34. Resources USDOL TEGL 7-99 Core and Customer Satisfaction Performance Measures http://wdr.doleta.gov/directives/attach/TEGL7-99.pdf http://wdr.doleta.gov/directives/attach/TEGL7-99.pdf AWI FG 02-029 9/20/2002 WIA Younger Youth Skill Attainment Rate WIA Younger Youth Skill Attainment Rate http://www.floridajobs.org/pdg/guidancepapers/029YoungerYouth.rtf TEGL 14-00, Change 1 WIASRDhttp://wdr.doleta.gov/directives/attach/TEGL14-00-ch1AttachE.pdf Reports & Performance Measures and Standards http://www.floridajobs.org/partsvcs.htm

35. Questions or Additional Information AWI Website for WIA reports http://floridajobs.org/awi/pdg/reports/default

36. Let’s Play Positive Or Negative?

37. Scenarios – Bob Bob was unemployed and had no diploma or GED when he was registered as an Adult for services. He was served in an OJT and exited the program working for that employer, Wilder Widgets, at $11.00 per hour. Bob’s region did not award credentials for OJT. His career Manager checked up on him all along and at 6 months found out that Bob had earned his GED. In the 3 rd quarter after exit, the Career Manager’s check in reveals that Bob has been promoted and is earning more money. This is positive, but why?

38. Why Bob is Positive…. 1.Bob was an unemployed Adult at entry who was found employed in the first quarter after exit – so, that’s an Entered Employment credit. 2.Bob was still employed in the 3 rd quarter, so that’s Retention. 3.Bob is earning more money, so that gives you a positive indicator on earnings gain. 4. Bob was served through training and earned a credential before the end of the 3 rd quarter!

39. Billie Ann Billie Ann was a Dislocated Worker served through training. She earned a RN degree and got a job after training and was employed in her first quarter after exit. THEN, she met a rich doctor, married him and quit her job two months after starting it. What impact does she have on your performance?

40. Billie Ann’s Impact 1.Billie Ann counts in the Employment rate equation. She was found employed by the end of the first quarter after exit, so that’s a positive. 2.Since Billie Ann was employed in the first quarter, she counts in your retention equation. But she quit before the 3 rd quarter after ext, so you lose on this one. 3.Again, since she was found employed in the first quarter she counts in earnings replacement – negative impact. 4. Billie Ann got a credential through training and was found employed in the first quarter, so that’s positive!

41. Jeannie – Older Youth Jeannie was unemployed when she entered the program – she had a job 3 months earlier making $6.25 an hour but ditched it to spend more time with her Beau. Her case manager enrolled her into Leadership Development activities, and Jeannie got bored with the program. She dropped out, and got a job for herself making $9.00 per hour balancing tires. Her case manager, though not the best at selecting services, was a good detective and found her in that job and closed her. The CM found her still employed at the 6 and 9 month marks. What do you think?

42. Jeannie, The Tire-Tosser 1.Since she was unemployed at entry, she’s in the entered employment rate and is POSITIVE! 2.Jeannie was still employed at the retention check, so you’re good there. 3.Based on Jeannie’s pre-program wage and what you know she’s making now, she should be good on earnings gain. 4.Here’s the bad part: Credential Attainment is based on ALL Older Youth, regardless of the service. Jeannie didn’t get a credential, so she counts against you here.

43. Marina Marina was a Dislocated Worker served through the One Stop in Intensive Services. Pre-program she earned $8.50 per hour. She got a new job at $9.50 per hour and exited the program in that job. Four months later her employer paid for her to get trained and credentialled as a Microsoft Certified Engineer. Her boss formed his own company and hired her at an even higher wage, which is where the Career Manager found her in the third quarter after exit. What do you think?

44. Marina’s Impact 1.Marina entered employment and was found employed in the first quarter after exit, so she’s good on the Entered Employment rate. 2.Marina was still employed in the 3 rd quarter after exit, another positive. Even though she moved to a new employer, she was still found employed. 3.You have a good indicator that if she entered the program at $8.50, left it at $9.50 and was earning more than that in the 3 rd quarter she’ll have a positive impact on earnings gain. 4. Sorry – no credential credit. She wasn’t served through training so she’s not in that equation.

45. Andrea Andrea was registered as an unemployed Adult and enrolled into training. She decided after a couple of days of being in the culinary arts program that she’d rather move to Albuquerque and watch Jerry Springer from the comfort of her mother’s couch. How does Andrea affect your performance?

46. Albuquerque Andrea 1.Since she was unemployed at entry, she’s in the Entered Employment rate - negative hit here. 2.No harm/no foul on Retention since she wasn’t found employed in the first quarter. 3.She won’t show up in earnings gain, either. 4.She was served in training and she neither earned a credential nor entered employment – so she’s negative here, too. So, Andrea is two negatives and two no harm/no fouls!

47. Maynette Maynette entered the program employed part time as a Kiddie Entertainment Specialist ($6.75 an hour) at one of the resorts on the beach. She was placed in an OJT with Publix to learn to be a butcher. She successfully completed her OJT and was awarded a certificate of completion as as result. She was closed earning $10.50 an hour at Publix. At the 6 month follow up, her case manager found that Maynette had won a scholarship to the Ft. Lauderdale Institute of Art. AT 9 months, Maynette was still attending school and not working. What do you think?

48. Maynette – the Next Monet? 1.Maynette was employed at registration, so she is not in the Entered Employment equation. 2.As to retention, though Maynette was found employed in the first quarter after exit but not in the 3 rd, she WAS found in post secondary education. She comes out of this equation. 3.Maynette comes out of earnings gain for the same reason as retention. 4.Maynette did get a credential, so she both counts and is positive in this equation.

49. Bryan the Burger Boss Bryan was working as a burger flipper for $6.75 an hour at the Big Cow Café when he registered as an adult for services. He was placed in training but only last through one semester, long enough to hit an OCP. That OCP helped him get hired as a manager for $9.75 an hour by Big Cow’s rival, the Kickin’ Chicken. Six months later his case manager found him back at Big Cow as the general manager making the princely sum of $15.00 per hour. This all sounds good and is good, but why?

50. Bryan’s Burger Success 1.Since Bryan was employed at entry, he won’t be in the entered employment rate. No harm here. 2.Even though Bruce changed employers, he was found employed – you get Retention points! 3.Bruce was earning more at placement and even more at six months, so you know your earnings gain should be good. 4. Bruce was both employed and had earned a credential (he hit an OCP) so he’s a positive on the employment and Credential measure!

51. Jeannie – Older Youth Jeannie was unemployed when she entered the program – she had had a job 3 months earlier making $6.25 an hour but ditched it to spend more time with her Beau. Her case manager enrolled her into Leadership Development activities, and Jeannie got bored with the program. She dropped out and got herself a job for $9.00 an hour balancing tires. Her case manager, though not the best at selecting services, was a good detective and found her in that job and closed her. The CM found her still employed at the 6 and 9 month marks. What do you think?

52. Jeannie 1.Since she was unemployed at entry, she’s in the Entered Employment rate and is positive! 2.Jeannie was still employed at the retention check, so you’re good here. 3.Based on Jeannie’s pre-program wage and what you know she was making now, she should be good on earnings gain. 4. Here’s the bad part : Credential attainment is based on ALL Older Youth, regardless of the service. Jeannie didn’t get a credential, so she counts against you here.

53. Bubba – Younger Youth Bubba was served through a school-based program. He was Basic Skills deficient and had that goal set for him every year ‘til he got up to speed and attained all of his goals. He graduated with the rest of the Bay High Tornadoes and headed off to Auburn with a football scholarship. His case manager faithfully followed up with Bubba and found at the 9 month mark that Bubba had decided he’d really rather be a marine and was currently enjoying boot camp. What does Bubba do to performance?

54. Bubba the Marine 1.Bubba hit those skill attainments every year, so that’s positive. 2.Bubba graduated, so there’s your diploma attainment! 3. Now even though Bubba started out in post secondary education and was found in the military at the 3 rd quarter mark, he’s positive on retention. He was found in one of the “good” places as far as that measure was concerned.

55. Maynette – Older Youth Maynette entered the program employed part time as a Kiddie Entertainment Specialist ($6.75 per hour) at one of the resorts on the beach. She was placed into an OJT with Publix as a butcher. She successfully finished the OJT and was awarded a certificate of completion as a result. She was closed earning $10.50 an hour at Publix. At the 6 month follow up, her case manager found that Maynette had won a scholarship to the Fort Lauderdale Institute of Art. At 9 months, Maynette was still attending school and not working. What do you think?

56. Maynette – the Next Monet? 1.Maynette was employed at registration, so she’s not in the entered employment equation. 2.As to retention, though Maynette was found employed in the first quarter after exit but not in the third, she was found in post secondary education. She comes out of the equation. 3.Maynette comes out of earnings gain for the same reason as retention. 4.Maynette did get a credential from you, so she both counts and is positive for this equation.

57. Reggie Reggie was an 18-year old Younger Youth who entered your program with a HS diploma. Reggie had some basic skills problems, so while you paid for Reggie’s tuition so that he could become an RN you also set basic skills goals and helped him attain one every year. Reggie exited the program after completing the RN degree to move to Gainesville and get himself a BSN. While he was there he worked part time at Shands and met a rich Doctor who married him less than two months after he left the program. Dr.Delightful insisted that Reggie drop out and become a stay-at-home husband. What do you think?

58. Reggie, the House Herr 1.Reggie attained his goals, which means you set at least one per year, so this is positive. 2.Reggie already had a diploma, so he doesn’t even count in the Diploma/GED measure. 3.We’ll take a hit on retention, Reggie was not found in any of the “good” places in the 3 rd quarter after exit since he was sitting at home planning his wife the doctor’s dinner.