1. The Application of Common Measures
Back to Basics,
The Application of Common Measures
Lane Kelly
Performance Specialist
USDOL – ETA, Region 3 - Atlanta
May 22, 2008
Ritz-Carlton Hotel
Atlanta, Georgia

2. Focus of Today’s Session
Clarification of Terms and Concepts Related to these Topics:
Accountability for Grantees
Required Definitions
Exiting Participants -- Completion vs. Exit
Gap in Service
Basic Rules and Concepts Regarding Common Measures
Incumbent Worker vs. Employed Worker
Calculating the Common Measures

3. Accountability for Grantees
General performance accountability for grantees consists of:
Grant-specific training outcomes
Capacity building impacts
Common measures
Other information to tell the story of your grant’s accomplishments
Translation: Common Measures are a part of grantee accountability

4. Accountability for Grantees
Common measures are required for ETA formula-funded grants and most discretionary grants. They improve the ability to:
describe in a similar manner the core purposes of the workforce system
collaborate between programs and reduce duplicative data entry
analyze program outcomes and develop effective program strategies
compare results of similar government programs
Training and Employment Guidance Letter No details DOL policy for common measures
See TEGL 17-05

5. Accountability for Grantees
Which common measures apply to discretionary grantees?
Adult Common Measures
Youth Common Measures
Entered Employment Rate
Placement in Employment or Education
Employment Retention Rate
Attainment of Degree/Certificate
Average Earnings
Literacy/Numeracy Gains

6. Required Definitions—Participant
Participant: An individual determined eligible to participate in the program who receives a service funded by the program in either a physical location (e.g., One-Stop Center) or remotely through electronic technologies
Determined eligible to participate
Receives a funded service
In physical location or remotely
You need to determine what services trigger participation in your grant
You may need to capture information about services provided remotely

7. Required Definitions— Other Terms Related to Participation
Participation Date:
Date of first program-funded service
Participation Quarter:
Calendar quarter containing the participation date
Participant Cohort:
Group of individuals who share the same participation quarter

8. Required Definitions—Exiter
Exiter: A participant who has not received a program or partner-funded service for 90 consecutive days and no future services are scheduled has exited the program
Has not received a service
No service received for 90 consecutive days
No future services scheduled
Example: The last day of class was 2/19/08, and it’s now 5/19/08 – if no services were provided between 2/19 and 5/19, the exit date is 2/19/08

9. Required Definitions— Other Terms Related to Exit
Exit Date:
Date of last program-funded service
Exit Quarter:
Calendar quarter containing the exit date
Exit Cohort:
Group of individuals who exit during the same calendar quarter

10. Additional Reasons for Exiting a Participant
Six reasons for excluding an individual from performance calculations:
Institutionalized;
Missing or invalid SSN;
Death;
Reservist called to active duty;
Relocation to a mandated program (for youth only); and
Health/medical or family care 10

11. Exiting Participants Completion – a finishing or concluding action.
Completion vs. Exit -- Not the same!
Completion – a finishing or concluding action. 
(Complete services from the program; still in the program)
Exit – the last date of service.
(Exit the program; no longer in the program)
It is the reference point for all adult common measures.

12. Completion vs. Exit
Question: What are the consequences of equating the two concepts?
Classic Example – A grantee is delivering 2 “levels” of training; Level 1 must be completed before Level 2. After completion of both levels, the expectation is employment

13. Completion vs. Exit
Scenario 1: – Participant is exited after completion of their service plan; in other words, the participant is exited after completion of Level 2
When are you accountable for reporting on the common measures for this participant? 13

14. Scenario #1 Q1 Post-Exit Q2 Post-Exit Q3 Post-Exit
Participant completes Level 1, Level 2 and is exited; this is their
EXIT QTR
Q1 Post-Exit
Q2 Post-Exit
Q3 Post-Exit
Accountable for Entered Employment
Accountable for Employment Retention and Average Earnings

15. Completion vs. Exit
Scenario 2: – Participant is exited after completing Level 1; they are then re-enrolled and exited after completing Level 2
When are you accountable for reporting on the common measures for this participant? 15

16. Scenario #2 Q1 Post-Exit Q2 Post-Exit Q3 Post-Exit Q1 Post-Exit
Participant completes Level 1 and is exited; this is their EXIT QTR
Q1 Post-Exit
Q2 Post-Exit
Q3 Post-Exit
Participant completes Level 2 and is exited; this is their EXIT QTR
Q1 Post-Exit
Q2 Post-Exit
The grantee is accountable for Employment (based on the first exit) when the individual is still in training, decreasing the likelihood of a positive outcome

17. “Gap in Service”
A gap in service occurs if one of three allowable circumstances takes place that is expected to last more than 90 days:
Training is delayed,
A health/medical reason related to the participant or a family member, or
A temporary move from the area
NOTE: A gap in service prevents an unintentional exit from taking place.

18. “Gap in Service” -- Things to Know
A gap in service …
is dependent on the participant’s intent to continue services
Should not exceed 180 calendar days. However, a subsequent gap of an additional 180 days is possible to resolve issues
Must be documented in the participant case file, including the beginning and expected end date
Is optional; it’s a judgment call.

19. Utilizing “Gap in Service”
Example: Part I of a training program ended in 12/07, and Part II is not scheduled until 4/08; no services are provided between Part I and Part II
After 90 days of inactivity (e.g., end of 3/08), participants would have to be exited unless there is a gap in service
The gap in service stops the 90-day clock

20. A Gap in Service is Optional
Scenario: Project participant has a health issue that will preclude further participation at this time for at least 90 days.
What do you do?
Does s/he intend to continue services?
Based on the answer, you have an option – to either document a gap in service or exit the participant.

21. Exit or Continue Gap in Service
Scenario: A participant has been placed in a gap in service of 120 days; at the 100th day, the individual decides they no longer wish to continue services
What do you do?
In this case, you have an exit, with the exit date being the last date of service. 21

22. Exit or Gap in Service
Scenario: A participant finishes one class, disappears and then resurfaces later to attend another class.
Can a “gap in service” be used for the time in between the participant’s disappearance and resurfacing?
Generally, an exit would be more appropriate because the “delay before training” was due to the participant and not the training schedule.

23. Common Measures — Basic Rules and Concepts
Only participants are included in performance calculations
Exception: If one of six conditions exist, individuals are excluded from common measure calculations
Exception: Statutory exclusion applying to WIA adults receiving self-service and informational activities only
Calculations are based on exiters
Only exception is Youth Literacy/Numeracy, which is based on one year from participation date
Specific data sources must demonstrate the outcomes

24. Incumbent Worker vs. Employed Worker
Incumbent Worker Training Programs --Typically… targets specific employers that are either experiencing a decline and have the potential to undergo workforce reductions, or are experiencing a skills gap that impacts their ability to compete, retain workers, and expand operations
Employed Worker --
Talent development training activities – Target unemployed, underemployed, and employed persons seeking career advancement 24 24

25. Incumbent Worker Training
What is Incumbent Worker Training (IWT)?
Customized training to meet the needs of the employer and delivered to workers employed by the targeted business
Employer is typically responsible for the assessment of workers and the design and implementation of the training
Supports career advancement; promotes high skill, high wage positions; and promotes economic development within a region
Delivered to targeted employers via contracts or written agreements
IWT contracts between employers and the grantee should require employers to provide participation and outcome data 25 25

26. Entered Employment Rate
Of those who are not employed at the date of participation:
Number of adult participants who are employed in the 1st quarter after the exit quarter
Number of adult participants who exit during the quarter

27. Entered Employment Rate
Employed at
Participation
Date?
YES
Excluded from Measure NO
Numerator
Employed in
1st quarter after exit?
YES
Denominator NO

28. Employment Retention Rate
Of those who are employed in the first quarter after the exit quarter:
Number of adult participants who are employed in both the second and third quarters after the exit quarter
Number of adult participants who exit during the quarter

29. Employment Retention Rate
Employed in 1st quarter
after exit
quarter? NO
Excluded from Measure
YES
Numerator
Employed in
2nd and 3rd quarters after exit quarter?
YES
Denominator NO

30. Average Six-Month Earnings
Of those adult participants who are employed in the first, second and third quarters after the exit quarter:
Total earnings in the second quarter plus total earnings in the third quarter after the exit quarter
Number of adult participants who exit during the quarter

31. Average Six-Month Earnings
Employed in
1st quarter after
exit quarter? No
Excluded from Measure
Yes No
Numerator = Wages from quarters 2 & 3, post-exit included
Employed in
2nd and 3rd quarters after exit quarter?
Denominator = Number of adult participants who exit during the quarter
Yes

32. The Bigger Picture– Adult Common Measures
Participation Quarter(s)
Exit Quarter
Q1 Post-Exit
Q2 Post-Exit
Q3 Post-Exit
Because the adult common measures are exit-based, participation dates, participation quarters, and participation cohorts are not needed for calculations
The 3 adult common measures are exit-based. Therefore, the exit quarter is critical
Entered Employment Rate is based on employment in Q1 post-exit; this is where we expect the positive outcome
Employment Retention Rate is based on employment in Q1 and Q2 post-exit for those employed Q1 post-exit; we expect employment in both post-exit quarters
Average Earnings is based on gross earnings in Q2 and Q3 post-exit for those employed Q1, Q2 and Q3 post-exit

33. Example -- Adult Common Measures
Scenario: You have a participant who was employed at participation, employed in the first quarter after exit, and not employed in the third quarter after exit.
For which adult common measure (s) would you be held accountable?
For which adult common measure (s) would there be a positive outcome?

34. More on the Common Measures
Training and Employment Guidance Letter (TEGL) 17-05:
Back to Basics Webinars:
Part I – Back to Basics, Part I: Data Collection and Adult Common Measures
Part II – Back to Basics, Part II: Data Processing and Reporting
Part III – Back to Basics, The Application of Common Measures to Community-Based and High-Growth Grants

35. Questions?

36. THANK YOU