1. 1 Program and Compliance Management Workshop: Documentation Desires, Duties, Dilemmas V I R T U A L L Y

2. Outline IntroductionIntroduction Eligibility vs. Data ValidationEligibility vs. Data Validation DV DocumentationDV Documentation Documenting Common Measure OutcomesDocumenting Common Measure Outcomes Best Case Scenario?Best Case Scenario? 2

3. Introduction 3

4. First Things First Documentation is not just paper (Utah, for instance, is entirely paperless) Requirements differ across programs and populations and pertain to: –Who you’re serving …documenting characteristics –What they’re getting …documenting services –With what results …documenting outcomes “THEY said…” – Make sure what you think is required really is 4

5. Potential Data Sources Who You’re Serving What They’re Getting With What Results Birth Certificate Baptismal Records Social Security Card Resident Card Driver’s License School/Student ID School Records Hospital Records Public Agency Records DD-214 Veterans’ Org. Records Assessment Records Pay Stubs [Others] Attendance Sheets Provider Forms Provider Invoices Sign-in Sheets School Records Activity Forms Progress Reports [Others] Assessment Results Completion dates Pay Stubs Documented employer call School Records Progress Reports Copy of Diploma Copy of Certificate Surveys [Others] 5

6. The Goldilocks Factor Just right.Not too much. Not too little. Just right. –Supports stewardship, accountability –Leaves audit trail –Consistent with data validation requirements 6

7. Issues from Data Validation Reviews Lack of documentation or insufficient documentation to support service provision (particularly for youth services) State and/or Local policy includes incorrect sources Issues with case files and case notes >> 7

8. Issues from Data Validation Reviews (2) Case Files –Can’t be located, documents missing –Inconsistencies between file and MIS –Lack of consistent file structure –One file for multiple periods of participation Case Notes –Quality (insufficient, illegible, irrelevant); Missing (where needed or required by policy) –Case notes contain info that differs from other documents or the MIS (e.g., different dates) 8

9. Eligibility vs. Data Validation 9

10. Are the Following Statements TRUE or FALSE? Federal policy elaborates on the eligibility requirements contained in the statute and regulations. Documentation for eligibility and documentation for data validation are the same. 10

11. EXAMPLE: Youth Eligibility Sec. 101(13) of the WIA states that eligible youth are low income and have at least one barrier: 1.Basic Skills Deficient 2.School Dropout 3.Homeless, Runaway or Foster Child 4.Pregnant or Parenting 5.Offender 6.Youth who requires additional assistance… 11 Documentation Disconnect

12. BSD is defined at Sec. 101(4) as being determined by a “generally accepted standardized test…” –…So eligibility is based on a standardized test but –Federal DV policy has school records and case notes in addition to standardized tests (TEGL 28-11, May 9, 2012 (page A-18)) Basic Skills Deficiency Documentation Disconnect 12

13. AND… if the youth is BSD as defined, then Literacy/Numeracy requirements must also be followed –…which means the ‘standardized test’ has to be acceptable for L/N or another test administered –…which means reporting several WIASRD data elements and following the associated DV documentation requirements Basic Skills Deficiency Documentation Disconnect (2) 13

14. The 6 th Youth Eligibility Criterion Documentation Disconnect “…requires additional assistance to complete an educational program or to secure and hold employment as defined by State or local policy. If the State Board defines a policy, the policy must be included in the State Plan.” Many states don’t define, leaving it up to locals How is it defined in your state? 14

15. DV reviews have cited more than lack of documentation for this eligibility item Including insufficient case notes to validate Area of Concern often included in final report recommending a thorough review of local definitions/policy on this item (e.g., too vague) The 6 th Youth Eligibility Criterion Documentation Disconnect (2) 15

16. Data Validation Documentation 16

17. Types of Allowable Source Documentation The document itself (e.g., certificate) State MIS (management information system) Cross-Match Self-Attestation Case Notes 17

18. “State MIS” Specific, detailed information stored in the state’s management information system that supports an element –An indicator, such as a checkmark on a computer screen, is not acceptable source documentation in and of itself A deeper level of inquiry is required 18

19. Example of Using “State MIS” Validating Date of Exit After the exit date is noted in the state MIS, what additional information would you look for in order to pass this data element? –The service provided and last date received –No other qualifying services provided within 90 days of the last service –Individual not scheduled for future service and has no gap in service 19

20. “Cross-Match” Requires accessing a non-WIA database to find detailed supporting evidence for the data element –An indicator or presence of an SSN or checkmark in a non-WIA database is not sufficient evidence TANF participation can be determined by a cross-match with the state’s public assistance database but it is not sufficient to just find the SSN in the database 20

21. Example of Using “Cross-Match” Validating Youth Placement A state uses a cross-match (computerized record) with their Higher Education Administration as the source document for verifying placement into post-secondary education What information would you need to pass this data element? –Name or some other personal identifier –Information showing enrollment in post-secondary education –Date that indicates the youth was enrolled in first quarter after exit quarter 21

22. “Self-Attestation” When a participant states his/her status for a particular data element, then signs and dates a form acknowledging this status Key elements for self-attestation: –Participant identifying his/her status for permitted data elements –Signing and dating a form attesting to this self-identification 22

23. Example of Using “Self-Attestation” Validating Displaced Homemaker Status What type of information/language must be included on a self-attestation form to be considered an acceptable source document for this data element? –The individual has been providing unpaid services to family members in the home, and –Has been dependent on the income of another family member, and –Is no longer supported by that income, and –Is unemployed or underemployed, and –Is experiencing difficulty obtaining/upgrading employment. –Signature and date 23

24. “Case Notes” Paper or electronic statements by the case manager that identify, at a minimum, the following –Participant’s status for a particular data element, –Date the information was obtained (and from who, as appropriate) –Name of case manager 24

25. Example of Using “Case Notes” Validating Other Reasons For Exit What type of information must case notes include? –Participant information (likely moot point given electronic case notes) –Individual will no longer be able to participate due to [state reason*] *Institutionalization; Health/Medical; Deceased; Family Care; Reservist called to Active Duty; Relocation to mandated facility for youth 25

26. Documenting Common Measure Outcomes 26

27. Data Sources According to TEGL 17-05, these are the five basic data sources used to demonstrate common measure outcomes A.Wage Records B.WRIS, FEDES C.Administrative Records D.Assessment Instruments E.Supplemental Data 27

28. Documenting Outcomes Using Supplemental Data In the absence of wage recordsIn the absence of wage records, can be used to demonstrate employment and retention only (i.e., not earnings) All data and methods are documented and subject to audit –“Allowable sources... include case management notes, automated data base systems, One-Stop operating system’s administrative records, surveys of participants, and contact with employers.” (TEGL 17-05, page 26) The data and methods to obtain it can be clarified in state policy guidance (e.g., maintaining correct contact information is job one) 28

29. Documenting Outcomes Using Supplemental Data (2) Some states use supplemental data more than others (and some states don’t use it at all) –SD and UT do not use supplemental data –For the first quarter of PY12, supplemental data accounted for what percentage of the numerator of the Adult EER? MT – 6% ND – 2% WY – 9% 29

30. Documenting Outcomes Common Measures 1.Entered Employment Rate 2.Employment Retention Rate 3.Average Earnings 4.Placement in Employment or Education 5.Attainment of Degree/Certificate 6.Literacy/Numeracy A.Wage Records B.WRIS, FEDES C.Supplemental Data D.Administrative Records E.Assessment Instruments According to TEGL 17-05, these are the five basic data sources used to demonstrate common measure outcomes (A—E). Question: Which common measure utilizes four of the allowable data sources? 30

31. Best Case Scenario 31

32. The Ideal ? Use federal DV source documentation as foundation State provides additional guidance to clarify federal sources where necessary State provides guidance on “other” documentation to ensure statewide consistency Local Boards/One-Stop Operators provide additional guidance where needed (consistent with State and federal) 32

33. The Ideal ? (2) Irrespective of level, documentation policies and procedural requirements are part of staff orientation and general data management –Consistency –Accountability 33

34. 34