1. Standards and Guidelines for Conducting Cognitive Interviews
Jennifer Edgar
WSS Seminar
February 23, 2017

2. OMB Standards
Intended to document principles and practices Federal agencies must adhere to
Not many have been produced (15)
Conducting statistical surveys
CI Standards are the 2nd addendum to that
Not a roadmap or playbook, agencies still need experienced practitioners to carry them out

3. Standards and Guidelines
Standards – methodological requirement
What needs to be done
Guidelines – best practices on how to fulfill the standards
How to do it

4. Cognitive Interviewing Standards
Methodological Plan
Sample
Protocol for Interview
Analysis
Transparency
Reporting
Public Accessibility

5. 1. Methodological Plan
A plan must be developed prior to conducting a CI Study
Plan should include at least
Study objective
Sampling plan
Recruitment plan
Location of interviews
Development of interview guide
Analysis plan

6. 2. Sample Selection
Sample selection must be done in a purposeful way to ensure that respondents have the characteristics necessary to provide data required to meet the study objectives

7. Sampling Guidelines Sample Size Sample Characteristics
Recruitment Plan

8. 3. Interview Protocol
A protocol is required and must be designed to collect the processes by which a respondent interprets and responds to a question as well as any difficulties experienced by a respondent in providing and answer

9. Protocol Guidelines Iterative Process Interview as Analyst
Mode and Context Effects

10. 4. Systematic Analysis
The data produced must undergo a systematic analysis
Ensure no particular case is overemphasized and that findings represent the full range of responses

11. Analysis Guidelines
Empirical Evidence
Process for Conducting Analysis

12. 5. Transparent Analysis
Analysis must be transparent such that an outsider can understand and assess the findings
Each step must be documented
Findings can be traced back to raw data
Allows readers to examine and judge the quality of the data and results

13. Transparent Analysis Guidelines
Audit Trail
Maintenance of Raw Data

14. 6. Final Report
Methods, results and conclusions must be documented in a final report
Improves the rigor and credibility of a study
Maximizes transparency of analysis
Offers clear and conceptually adequate descriptions of all aspects of the study and its execution

15. Report Guidelines
Issues addressed in report
Contents of a report

16. 7. Publically Available
Final reports must be made available to the public if results are referenced in publications or data collection documentation