Standards and Guidelines For Cognitive Interviews

Slides:

Advertisements

Similar presentations

T IME & E FFORT R EPORTING AND S UPPLEMENT VERSUS S UPPLANT ABE Coordinators February 2012.

Advertisements

1 Developing EPA’s Peer Review Program Joint JIFSAN/SRA/RAC Symposium Dorothy E. Patton, Ph.D., J.D. September 30, 2003.

PISA FOR DEVELOPMENT Technical Workshops Components and input for ToR of the International Contractor(s) 9 th April 2014 OECD Secretariat 1.

1 A View of the United States Federal Statistical System from OMB Katherine K. Wallman Chief Statistician U. S. Office of Management and Budget.

Brian A. Harris-Kojetin, Ph.D. Statistical and Science Policy

Washington Headquarters Services Executive Services Directorate Information Management Division OMB Collection Number Paperwork Reduction Act – DoD Public.

A New Directive, a New Opportunity: How a New Policy Expands Communication at the National Center for Health Statistics Presented by Megan Cox Public Affairs.

FDA Research: Clearance Requirements and Implications Steven L. Bradbard, Ph.D. Team Leader, Consumer Studies CFSAN/ORPSS.

From Cutting Red Tape to Maximizing Net Benefits Alexander T. Hunt U.S. Office of Management and Budget Challenges on Cutting Red Tape Rotterdam, The Netherlands.

BLS U.S. Bureau of Labor Statistics The Paperwork Reduction ACT (PRA) What You Need to Know for Government UX Work Jean Fox Presented to the.

U.S. Army Records Management & Declassification Agency Privacy Act/System of Records Policies.

CENDI/NFAIS Quality Workshop: The Importance of Quality and Integrity Kevin Kirby, Enterprise Data Architect US Environmental Protection Agency Office.

ExxonMobil Biomedical Sciences, Inc. (LJH) SRA / FDA Peer Review Workshop - 9/30/03 Peer Review: Challenges Raised by OMB’s Draft Guidelines Leslie Hushka,

OMB A-123 Update CRT April 20, 2015 Mike Wetklow

Archived File The file below has been archived for historical reference purposes only. The content and links are no longer maintained and may be outdated.

1 OAR Guidance on -- “Consulting with Indian Tribal Governments” May 21, 2012.

The Executive Office of the President (EOP). Office of Management and Budget (OMB)

Jeff Morris Associate Director for Science Office of Science Policy Symposium on Peer Review of Risk Assessments and Related Activities September 30, 2003.

United States Department of Agriculture Food Safety and Inspection Service 1 National Advisory Committee on Meat and Poultry Inspection August 8-9, 2007.

American Community Survey ACS Content Review Webinar State Data Centers and Census Information Centers James Treat, ACSO Division Chief December 4, 2013.

Strengthening Science Supporting Fishery Management  Standards for Best Available Science  Implementation of OMB’s Peer Review Bulletin  Separation.

1 Assessing the Quality of Administrative Data 2012 FCSM Statistical Policy Seminar Session 7: New Perspectives on the Quality of Administrative Data December.

The Integrity of Data Management Responsible Conduct of Research Workshop Howard University November 2, 2010 Peggy Carr, Ph.D. Associate Commissioner National.

Presentation at HSPD-12 Workshop Ms. Jeanette Thornton May 4, 2005.

Audit Evidence Process

19 October 2004Enterprise Architecture in WSRP Portal 1 Foreword: Building Enterprise Architecture Through WSRP in Sample EPA Regional Portal FEA Goals:

Howard University October 10,2012 The Integrity of Data Management Responsible Conduct of Research Workshop Peggy Carr, Ph.D. Associate Commissioner National.

Preparation Plan. Objectives Describe the role and importance of a preparation plan. Describe the key contents of a preparation plan. Identify and discuss.

Imagery For The Nation Cost Benefit Analysis October 24, 2007.

THE RESEARCH PROCESS. STEP 1: IDENTIFY THE PROBLEM The first step in the process is to identify a problem or develop a research question. The research.

Harmonized Approach to Cash Transfers (HACT) to Implementing Partners

Washington Headquarters Services Executive Services Directorate Directives Division DoD Information Collection Program Public Collections Training: SORNs,

Washington Headquarters Services Executive Services Directorate Directives Division Office of Information Management (OIM) Public Collections Training:

Compliance with Framework of Quality Control - General & Specific Controls CA Vimal Chopra, Ex Chairman of CIRC of ICAI.

Multiple Indicator Cluster Surveys Survey Design Workshop

Use of Administrative Data in EIA’s Data Program

Public School Monitoring Roadmap

Research Methods Lesson 1 choosing a research method types of data

DOE Nuclear Safety Research and Development Program

Standards and Guidelines for Conducting Cognitive Interviews

Interoperability of Data Systems Administration for Children & Families Office of Planning, Research & Evaluation Robert (Bob) Garcia Regional Administrator,

Disasters, CRVS and the importance of statistics, health and privacy

Starter Look at the photograph, As a sociologist, you want to study a particular group in school. In pairs think about the following questions… Which group.

IS4550 Security Policies and Implementation

Harmonisation process of anonymisation of microdata

Research Proposal January 1, 2013 Consulted books

Consumer Satisfaction Data Collection for Information Dissemination

Organization of efficient Economic Surveys

DoD-Internal Collections

Informed Consent (SBER)

Information Management for Effective Emergency Response

UNODC-UNECE Manual on Victimization Surveys: Content

Harmonized Approach to Cash Transfers (HACT) to Implementing Partners

An Update of COSO’s Internal Control–Integrated Framework

U.S. Information Quality Standards

Quality Assurance in the European Statistical System

Guidelines on Integrated Economic Statistics

OMB Super-Circular Proposal

Preschool Development Grant (B-5)

Added flexibility for districts

Quality Reporting in CBS

The role of metadata in census data dissemination

PERFORMANCE AUDIT MINI-SURVEY RESULTS Tribunal de Contas da União Brazil May,

Uniform Guidance and Grants Accounting

International Standards for Compilation of Statistics: The Gap between Standards Adoption and Standards Implementation Katherine K. Wallman Chief Statistician.

2.7 Annex 3 – Quality reports

Étienne Saint-Pierre, Statistics Canada

« Survey on PRODCOM data collection in other MS

Fundamental Science Practices (FSP) of the U.S. Geological Survey

Presentation transcript:

Standards and Guidelines For Cognitive Interviews Bob Sivinski Statistical and Science Policy Office of Management and Budget

Contents Information Quality and the PRA Cognitive Interviews and Instrument Design Stat Policy Directive #2 and the PRA Clearance Process Questions Start general -> specific to the policy Comments aren’t meant to set OMB policy, see the actual guidance.

Information Quality and the PRA Burden Reduction Compliance with Federal policies Cross-government Coordination Information Quality - Focus on the objectivity, utility, integrity, and reproducibility of information - PRA and IQG emphasize ‘fitness for purpose’ Not just about burden.

Cognitive Interviews and Instrument Design Concepts, Methods & Design Data Collection Analysis Processing & Editing Review Dissemination, including policy decisions based on data or analysis • Scope Choice of Methods Sampling (inc Frame) Instrument Design Quality Control Procedures Quality Assurance Practices Statistical Standards Model Choice Assumptions Information Quality Guidelines Principles for Risk Peer Review Bulletin The Information Quality Roadmap Instrument design as one of the first steps in the info quality process. Everything else downstream. 90% of estimation takes place before data collection. Non-survey data becoming more available and important to stat agencies, what sets surveys apart from admin/big data and why are they still important? Measurable error/uncertainty/bias Ability to ask the exact question you want answered!

Statistical Policy Directive #2 And the PRA Clearance Process Federal agencies are required to adhere to all standards for every statistical survey, but OMB recognizes that these standards cannot be applied uniformly or precisely in every situation. The agency should include in its documentation the reasons why the standard could not be met and what actions the agency is taking to address any resulting issues. The standards and guidelines are not meant to be exhaustive of all information quality efforts, and are not meant to substitute for the extensive existing literature. Update on the status of the OMB website. Addendum has the same implications as the original guidance.

Questions? Bob Sivinski rsivinski@omb.eop.gov (202) 395-1205