The Application of Common Measures Back to Basics, The Application of Common Measures Lane Kelly Performance Specialist USDOL – ETA, Region 3 - Atlanta May 22, 2008 Ritz-Carlton Hotel Atlanta, Georgia

Focus of Today’s Session Clarification of Terms and Concepts Related to these Topics: Accountability for Grantees Required Definitions Exiting Participants -- Completion vs. Exit Gap in Service Basic Rules and Concepts Regarding Common Measures Incumbent Worker vs. Employed Worker Calculating the Common Measures

Accountability for Grantees General performance accountability for grantees consists of: Grant-specific training outcomes Capacity building impacts Common measures Other information to tell the story of your grant’s accomplishments Translation: Common Measures are a part of grantee accountability

Accountability for Grantees Common measures are required for ETA formula-funded grants and most discretionary grants. They improve the ability to: describe in a similar manner the core purposes of the workforce system collaborate between programs and reduce duplicative data entry analyze program outcomes and develop effective program strategies compare results of similar government programs Training and Employment Guidance Letter No. 17-05 details DOL policy for common measures See TEGL 17-05

Accountability for Grantees Which common measures apply to discretionary grantees? Adult Common Measures Youth Common Measures Entered Employment Rate Placement in Employment or Education Employment Retention Rate Attainment of Degree/Certificate Average Earnings Literacy/Numeracy Gains

Required Definitions—Participant Participant: An individual determined eligible to participate in the program who receives a service funded by the program in either a physical location (e.g., One-Stop Center) or remotely through electronic technologies Determined eligible to participate Receives a funded service In physical location or remotely You need to determine what services trigger participation in your grant You may need to capture information about services provided remotely

Required Definitions— Other Terms Related to Participation Participation Date: Date of first program-funded service Participation Quarter: Calendar quarter containing the participation date Participant Cohort: Group of individuals who share the same participation quarter

Required Definitions—Exiter Exiter: A participant who has not received a program or partner-funded service for 90 consecutive days and no future services are scheduled has exited the program Has not received a service No service received for 90 consecutive days No future services scheduled Example: The last day of class was 2/19/08, and it’s now 5/19/08 – if no services were provided between 2/19 and 5/19, the exit date is 2/19/08

Required Definitions— Other Terms Related to Exit Exit Date: Date of last program-funded service Exit Quarter: Calendar quarter containing the exit date Exit Cohort: Group of individuals who exit during the same calendar quarter

Additional Reasons for Exiting a Participant Six reasons for excluding an individual from performance calculations: Institutionalized; Missing or invalid SSN; Death; Reservist called to active duty; Relocation to a mandated program (for youth only); and Health/medical or family care 10

Exiting Participants Completion – a finishing or concluding action. Completion vs. Exit -- Not the same! Completion – a finishing or concluding action.  (Complete services from the program; still in the program) Exit – the last date of service. (Exit the program; no longer in the program) It is the reference point for all adult common measures.

Completion vs. Exit Question: What are the consequences of equating the two concepts? Classic Example – A grantee is delivering 2 “levels” of training; Level 1 must be completed before Level 2. After completion of both levels, the expectation is employment

Completion vs. Exit Scenario 1: – Participant is exited after completion of their service plan; in other words, the participant is exited after completion of Level 2 When are you accountable for reporting on the common measures for this participant? 13

Scenario #1 Q1 Post-Exit Q2 Post-Exit Q3 Post-Exit Participant completes Level 1, Level 2 and is exited; this is their EXIT QTR Q1 Post-Exit Q2 Post-Exit Q3 Post-Exit Accountable for Entered Employment Accountable for Employment Retention and Average Earnings

Completion vs. Exit Scenario 2: – Participant is exited after completing Level 1; they are then re-enrolled and exited after completing Level 2 When are you accountable for reporting on the common measures for this participant? 15

Scenario #2 Q1 Post-Exit Q2 Post-Exit Q3 Post-Exit Q1 Post-Exit Participant completes Level 1 and is exited; this is their EXIT QTR Q1 Post-Exit Q2 Post-Exit Q3 Post-Exit Participant completes Level 2 and is exited; this is their EXIT QTR Q1 Post-Exit Q2 Post-Exit The grantee is accountable for Employment (based on the first exit) when the individual is still in training, decreasing the likelihood of a positive outcome

“Gap in Service” A gap in service occurs if one of three allowable circumstances takes place that is expected to last more than 90 days: Training is delayed, A health/medical reason related to the participant or a family member, or A temporary move from the area NOTE: A gap in service prevents an unintentional exit from taking place.

“Gap in Service” -- Things to Know A gap in service … is dependent on the participant’s intent to continue services Should not exceed 180 calendar days. However, a subsequent gap of an additional 180 days is possible to resolve issues Must be documented in the participant case file, including the beginning and expected end date Is optional; it’s a judgment call.

Utilizing “Gap in Service” Example: Part I of a training program ended in 12/07, and Part II is not scheduled until 4/08; no services are provided between Part I and Part II After 90 days of inactivity (e.g., end of 3/08), participants would have to be exited unless there is a gap in service The gap in service stops the 90-day clock

A Gap in Service is Optional Scenario: Project participant has a health issue that will preclude further participation at this time for at least 90 days. What do you do? Does s/he intend to continue services? Based on the answer, you have an option – to either document a gap in service or exit the participant.

Exit or Continue Gap in Service Scenario: A participant has been placed in a gap in service of 120 days; at the 100th day, the individual decides they no longer wish to continue services What do you do? In this case, you have an exit, with the exit date being the last date of service. 21

Exit or Gap in Service Scenario: A participant finishes one class, disappears and then resurfaces later to attend another class. Can a “gap in service” be used for the time in between the participant’s disappearance and resurfacing? Generally, an exit would be more appropriate because the “delay before training” was due to the participant and not the training schedule.

Common Measures — Basic Rules and Concepts Only participants are included in performance calculations Exception: If one of six conditions exist, individuals are excluded from common measure calculations Exception: Statutory exclusion applying to WIA adults receiving self-service and informational activities only Calculations are based on exiters Only exception is Youth Literacy/Numeracy, which is based on one year from participation date Specific data sources must demonstrate the outcomes

Incumbent Worker vs. Employed Worker Incumbent Worker Training Programs --Typically… targets specific employers that are either experiencing a decline and have the potential to undergo workforce reductions, or are experiencing a skills gap that impacts their ability to compete, retain workers, and expand operations Employed Worker -- Talent development training activities – Target unemployed, underemployed, and employed persons seeking career advancement 24 24

Incumbent Worker Training What is Incumbent Worker Training (IWT)? Customized training to meet the needs of the employer and delivered to workers employed by the targeted business Employer is typically responsible for the assessment of workers and the design and implementation of the training Supports career advancement; promotes high skill, high wage positions; and promotes economic development within a region Delivered to targeted employers via contracts or written agreements IWT contracts between employers and the grantee should require employers to provide participation and outcome data 25 25

Entered Employment Rate Of those who are not employed at the date of participation: Number of adult participants who are employed in the 1st quarter after the exit quarter Number of adult participants who exit during the quarter

Entered Employment Rate Employed at Participation Date? YES Excluded from Measure NO Numerator Employed in 1st quarter after exit? YES Denominator NO

Employment Retention Rate Of those who are employed in the first quarter after the exit quarter: Number of adult participants who are employed in both the second and third quarters after the exit quarter Number of adult participants who exit during the quarter

Employment Retention Rate Employed in 1st quarter after exit quarter? NO Excluded from Measure YES Numerator Employed in 2nd and 3rd quarters after exit quarter? YES Denominator NO

Average Six-Month Earnings Of those adult participants who are employed in the first, second and third quarters after the exit quarter: Total earnings in the second quarter plus total earnings in the third quarter after the exit quarter Number of adult participants who exit during the quarter

Average Six-Month Earnings Employed in 1st quarter after exit quarter? No Excluded from Measure Yes No Numerator = Wages from quarters 2 & 3, post-exit included Employed in 2nd and 3rd quarters after exit quarter? Denominator = Number of adult participants who exit during the quarter Yes

The Bigger Picture– Adult Common Measures Participation Quarter(s) Exit Quarter Q1 Post-Exit Q2 Post-Exit Q3 Post-Exit Because the adult common measures are exit-based, participation dates, participation quarters, and participation cohorts are not needed for calculations The 3 adult common measures are exit-based. Therefore, the exit quarter is critical Entered Employment Rate is based on employment in Q1 post-exit; this is where we expect the positive outcome Employment Retention Rate is based on employment in Q1 and Q2 post-exit for those employed Q1 post-exit; we expect employment in both post-exit quarters Average Earnings is based on gross earnings in Q2 and Q3 post-exit for those employed Q1, Q2 and Q3 post-exit

Example -- Adult Common Measures Scenario: You have a participant who was employed at participation, employed in the first quarter after exit, and not employed in the third quarter after exit. For which adult common measure (s) would you be held accountable? For which adult common measure (s) would there be a positive outcome?

More on the Common Measures Training and Employment Guidance Letter (TEGL) 17-05: http://wdr.doleta.gov/directives/corr_doc.cfm?DOCN=2195 Back to Basics Webinars: Part I – Back to Basics, Part I: Data Collection and Adult Common Measures http://www.workforce3one.org/view.cfm?id=4809&info=1 Part II – Back to Basics, Part II: Data Processing and Reporting http://www.workforce3one.org/view.cfm?id=4824&info=1 Part III – Back to Basics, The Application of Common Measures to Community-Based and High-Growth Grants http://www.workforce3one.org/public/_shared/detail.cfm?id=4857&info=1&simple=false

Questions?

THANK YOU