1. Other Data Issues Improving Data Quality American Institutes for Research February 2005

2. Other Data Issues 2 Quality Issues Raised by Data Quality Issues Raised by Data Wide variations among states in key data elements High intra-state variance on some data across years Data have internal inconsistencies

3. Other Data Issues 3 Wide Variation Among States

4. Other Data Issues 4 Importance of Accurate Attendance Measures Needed to understand intensity and duration of participation WIA requirement to have programs of “sufficient intensity and duration” Needed to track time between pre- and post-testing Depending on basis for funding—need accurate data to: Monitor local program grant compliance Ensure providers are treated fairly based on service

5. Other Data Issues 5 Internal Inconsistencies— Employment Data Employed1,022,394 Unemployed1,044,223 Not in Labor Force 398,632 Total Students2,465,249 Total Students Reported 2,736,192

6. Other Data Issues 6 Improving Data Quality State policies, training technical assistance Local implementation procedures Local staff roles and resources Data collection procedures Staff training Local student record and data systems

7. Other Data Issues 7 Local Staff Roles and Resources Staff assigned to data collection have clear understanding of responsibilities Time, resources for data collection Access to enter and retrieve data at the local level Data flow procedures Well-defined roles, procedures for all staff Decision points identified and resolved

8. Other Data Issues 8 Local Data Collection Model Teacher 1Teacher 2 Intake Clerical Data Entry Data System Reporting/Error Checking State Data System/Report Federal Report Program Administrative Review

9. Other Data Issues 9 Data Collection Procedures Clear definitions of measures and procedures (manuals, data dictionaries) Standard forms with coding categories, tied to database Timely data entry procedures Regular data review and error checking procedures for quality control

10. Other Data Issues 10 Implications for Staff Training and Data System Ongoing training—train the right staff and enough staff at each program Technical support for questions Student level, relational database Local access to data with reporting capabilities that support program management decisions Scaled test scores in database

11. Other Data Issues 11 Discussion—Quality Control Timely data submission Ongoing data monitoring Site visits and data auditing Support to locals through training and resources Other??