1 Assessing the Quality of Administrative Data 2012 FCSM Statistical Policy Seminar Session 7: New Perspectives on the Quality of Administrative Data December 5, 2012 Bill Iwig, USDA/NASS (on detail to OMB/SSP)

Development of the Data Quality Assessment Tool Data Quality ~ Fitness for Use Intended to facilitate decisions by a potential data user about the suitability of the data for identified purpose. Intended to prompt additional questions. Based on the information gathered, the data user may decide to: Use the data as planned Alter plans Accommodate some quality weakness in the data Not use the data 2

Development of the Data Quality Assessment Tool Multi-Dimensional Relevance Accessibility Interpretability Coherence Accuracy Institutional Environment Questions within each dimension are designed to address quality characteristics of micro-level administrative data. Example answers provided based on characteristics of a Supplemental Nutrition Assistance Program (SNAP) data set. 3

Development of the Data Quality Assessment Tool Discovery Phase Evaluation period leading to approval for developing a data-sharing MOU. Initial Acquisition Phase Evaluation period from MOU development approval to first- time receipt of data. Repeated Acquisition Phase Continuing periodic receipt of data. 4

Putting the Phases and Dimensions Together Discovery Phase: 12 questions covering Relevance, Accessibility, and Interpretability, including a request for a Data Dictionary Initial Acquisition Phase: 29 questions covering Accessibility, Interpretability, Coherence, Accuracy, and Institutional Environment Repeated Acquisition: 11 questions covering Interpretability (updated Data Dictionary), Coherence, Accuracy, and Institutional Environment 5

What it is and What it isn’t Evaluates quality characteristics of an administrative data file. Doesn’t address quality characteristics of linked data. Doesn’t provide guidance for making fitness for use decisions. Doesn’t provide quality improvement recommendations. Not specifically designed to address quality of commercial data. 6

Status of Pilot Testing Obtained feedback from three federal/state reviewers prior to pilot testing. Pilot tested on: Child Care data files from the Office of Child Care Information Systems at DHHS – Initial Acquisition Phase Wisconsin Child Care Payment System files – All Phases Expecting a response for two additional files. Planning to conduct follow-up interview after completion of assessment tool. 7

General Comments Completing the Data Quality Assessment Tool is perceived to be burdensome to the administrative agency. Some agencies may be concerned about data quality implications. Many administrative data files are compiled from various data sources. Right scope and content to be incorporated into agency administrative record procedures? 8

Next Steps Update based on feedback received. Make available to the public. Pursue opportunities to promote use. 9

Thank You Bill Iwig Detailee to the Statistical and Science Policy Office, OMB