1. Q-Bank: Using Information Technology for Collaboration to Enhance Survey Methodology
2008 PHIN Conference
August 25, 2008
Aaron Maitland, NCHS
Justin Mezetin, NCHS

2. Background Why evaluate survey questions?
Ensure questions capture intended concepts
Comprehension
Recall
Judgment
Response selection
Identify comparability problems
To fix problems

3. Question Evaluation Methods
Qualitative
Cognitive Interviewing
Focus Groups
Quantitative
Behavior coding
Statistical modeling
Mixed method approach

4. Cognitive Interviewing Example
Would you say {your} health in general is excellent, very good, good, fair, or poor?
“Many times [respondents] answers seemed to depend on who they were comparing themselves to, which in part seemed to be due to this question’s placement right after the physical limitations section. People sometimes compared themselves to those who could not get around or take care of themselves; others compared themselves to those their own age or those with similar health situations. This also results in incomparable data.”

5. The Problem of Accessibility
Historically, cognitive test findings have been relatively inaccessible.
Implications
Knowledge is lost
Resources are wasted
Lack of transparency and accountability
Important need for a medium to make findings available

6. What is Q-Bank Q-Bank houses:
Test reports
Test findings
Methods used to arrive at those findings
Q-Bank is NOT a database of good questions.

11. Who should use Q-Bank? Question designers Survey data users
Survey methodologists

12. History and Development
2002 NCHS began development
2005 interagency effort
Steering Committee
Designed to incorporate:
Interviewer administered surveys
self-administered surveys
establishment surveys
Establish common lexicon
Establish report standards

13. Contributors Bureau of Labor Statistics
National Center for Health Statistics
National Science Foundation
Office of Behavioral and Social Sciences Research and National Cancer Institute of the National Institutes of Health
US Census Bureau

14. Current Organization
Hosted by the National Center for Health Statistics (NCHS)
NCHS Staffing
Q-Bank Administrator
Dedicated Programmer
Steering Committee Chair
Steering Committee
NCHS, Census, BLS, NCI, NSF, GAO

15. Future Goals Full inclusion of evaluation methods
Ongoing programming revisions
Usability testing
Increase participation among existing agencies and collaboration with additional agencies
Developing standards for evaluation reports
Potential integration with other survey methods tools

16. Q-Bank Website: http://wwwn. cdc. gov/qbank/home
Q-Bank Website: Q-Bank