1. Standards and Guidelines For Cognitive Interviews
Bob Sivinski
Statistical and Science Policy Office of Management and Budget

2. Contents Information Quality and the PRA
Cognitive Interviews and Instrument Design
Stat Policy Directive #2 and the PRA Clearance Process
Questions
Start general -> specific to the policy
Comments aren’t meant to set OMB policy, see the actual guidance.

3. Information Quality and the PRA
Burden Reduction
Compliance with Federal policies
Cross-government Coordination
Information Quality
- Focus on the objectivity, utility, integrity, and reproducibility of information
- PRA and IQG emphasize ‘fitness for purpose’
Not just about burden.

4. Cognitive Interviews and Instrument Design
Concepts,
Methods
& Design
Data
Collection
Analysis
Processing
& Editing
Review
Dissemination,
including
policy decisions
based on data
or analysis •
Scope
Choice of Methods
Sampling (inc Frame)
Instrument Design
Quality Control Procedures
Quality Assurance Practices
Statistical Standards
Model Choice
Assumptions
Information Quality
Guidelines
Principles for Risk
Peer Review
Bulletin
The Information Quality Roadmap
Instrument design as one of the first steps in the info quality process.
Everything else downstream.
90% of estimation takes place before data collection.
Non-survey data becoming more available and important to stat agencies, what sets surveys apart from admin/big data and why are they still important?
Measurable error/uncertainty/bias
Ability to ask the exact question you want answered!

5. Statistical Policy Directive #2 And the PRA Clearance Process
Federal agencies are required to adhere to all standards for every statistical survey, but OMB recognizes that these standards cannot be applied uniformly or precisely in every situation.
The agency should include in its documentation the reasons why the standard could not be met and what actions the agency is taking to address any resulting issues.
The standards and guidelines are not meant to be exhaustive of all information quality efforts, and are not meant to substitute for the extensive existing literature.
Update on the status of the OMB website.
Addendum has the same implications as the original guidance.

6. Questions?
Bob Sivinski
(202)