U.S. Information Quality Standards Shannon Joyce White House Office of Information and Regulatory Affairs U.S./Colombia RIA and GRP Workshop Bogota, Colombia April 11, 2018 Any views expressed here are solely those of the presenter, and do not necessarily reflect the position of the Office of Management and Budget or the Executive Office of the President.

The Importance of Quality Information as the Foundation of Policy Our Nation and our economy both rely on the flow of objective, credible information to inform the decisions of governments, businesses, organizations, and individuals. Information protection and dissemination policies are crucial to maintaining public trust in government information and decisions. The United States promotes data-driven decisions and evidence-based policy through its government-wide policies for data quality, access, protection, and sharing.

Types of Government Data Statistical Administrative (program) Surveillance Research Analytic information

Information Oversight Privacy Policy Enhance trust in the Federal government’s protection of personal information Oversight surrounding stewardship of government information Statistical Policy Ensure the efficiency, effectiveness, and credibility of the Federal statistical system. Directives promote the objectivity, reliability, relevance, timeliness of data, and consistent measurement methods.

Information Oversight Information Collection Review Maximize the usefulness of information collected from the public Minimize burden on the public – only collect what is absolutely necessary and nothing more. Dissemination of Government Information Increase public access to government data to drive private innovation. Increase transparency of data underlying policies. Increase quality of data used to drive policies (e.g., peer review of scientific assessments)

U.S. Information Quality Guidelines The quality of information be appropriate for the context in which it is used Pre-dissemination review appropriate to type of information. In the case of scientific information, peer review. Opportunity for challenging the quality of information in the context in which it was used.

What is Quality Information? Not a static standard. Consider the: Fitness for purpose rigor of design, completeness of the data, external validity Utility Usefulness of the information to the intended users Objectivity is the information presented in an accurate, clear, complete, and unbiased manner? Are your assumptions transparent?

Highest Standard for Information: Influential Information Information is ‘influential’ when the information “will have or does have a clear and substantial impact on important public policies or important private sector decisions. Influential information needs to meet additional quality standards including reproducibility Agencies define criteria for influential within context of their mission. Each agency has defined influential in ways appropriate for that agency. Education and HHS use broad OMB Guidelines definition. EPA has specified that this definition includes information which: supports agency rulemakings, is used in support of Economically Significant actions as defined in EO 12866, or is a major work product undergoing peer review. Influential information should be disseminated in a manner transparent enough to facilitate reproducibility. Purpose is to cultivate a consistent agency commitment to transparency. It is not OMB’s intent that each analytic result must be reproduced before it is disseminated. A standard of ‘capable of being substantially reproduced’ is applied. Means that independent analysis of the original or supporting data using identical methods would generate similar analytic results, subject to an acceptable degree of imprecision or error.

Tools to Increase Information Quality Survey design guidance Measurement and classification standards Review of Information Collection Requests Analytic guidance (risk principles) Regulatory review – scientific/technical analysis Transparency of methods and limitations Pre-dissemination review (e.g., peer review for scientific analyses)

Assessing Information Quality in an RIA Is the justification supported with data, analysis, or literature or is the agency simply making contentions? Is the set of literature or facts cited balanced and complete? Is the tone of the presentation objective? Are the assumptions clear? Is the literature review masquerading as an evaluation? If so, was there peer review? Does the discussion distinguish between science and policy?

Useful Links – Information Policy Paperwork Reduction Act: https://www.reginfo.gov/public/reginfo/pra.pdf Privacy Act: https://www.justice.gov/opcl/overview-privacy-act-1974-2015- edition E-Government Act: https://www.gpo.gov/fdsys/pkg/PLAW- 107publ347/pdf/PLAW-107publ347.pdf Information Quality Act: Section 515 of the Treasury and General Government Appropriations Act for Fiscal Year 2001 (Public Law 106-554) https://www.gpo.gov/fdsys/pkg/PLAW-106publ554/html/PLAW- 106publ554.htm Government-wide Information Quality Guidelines: https://www.gpo.gov/fdsys/pkg/FR-2002-02-22/pdf/R2-59.pdf CIPSEA guidance: https://www.gpo.gov/fdsys/granule/FR-2007-06-15/E7- 11542/content-detail.html Commission on Evidence-Based Policymaking: https://www.cep.gov/content/dam/cep/report/cep-final-report.pdf

Shannon Joyce: sjoyce@omb.eop.gov Questions? Thank you! Contact information: Shannon Joyce: sjoyce@omb.eop.gov Not sure about providing contact info. I’m fine to read off either email or phone numbers. If there is concern, we could do the main phone line. Deliberative staff materials. Subject to further revision. Do not distribute/duplicate.