Q-Bank: Using Information Technology for Collaboration to Enhance Survey Methodology 2008 PHIN Conference August 25, 2008 Aaron Maitland, NCHS Justin Mezetin, NCHS

Background Why evaluate survey questions? Ensure questions capture intended concepts Comprehension Recall Judgment Response selection Identify comparability problems To fix problems

Question Evaluation Methods Qualitative Cognitive Interviewing Focus Groups Quantitative Behavior coding Statistical modeling Mixed method approach

Cognitive Interviewing Example Would you say {your} health in general is excellent, very good, good, fair, or poor? “Many times [respondents] answers seemed to depend on who they were comparing themselves to, which in part seemed to be due to this question’s placement right after the physical limitations section. People sometimes compared themselves to those who could not get around or take care of themselves; others compared themselves to those their own age or those with similar health situations. This also results in incomparable data.”

The Problem of Accessibility Historically, cognitive test findings have been relatively inaccessible. Implications Knowledge is lost Resources are wasted Lack of transparency and accountability Important need for a medium to make findings available

What is Q-Bank Q-Bank houses: Test reports Test findings Methods used to arrive at those findings Q-Bank is NOT a database of good questions.

Who should use Q-Bank? Question designers Survey data users Survey methodologists

History and Development 2002 NCHS began development 2005 interagency effort Steering Committee Designed to incorporate: Interviewer administered surveys self-administered surveys establishment surveys Establish common lexicon Establish report standards

Contributors Bureau of Labor Statistics National Center for Health Statistics National Science Foundation Office of Behavioral and Social Sciences Research and National Cancer Institute of the National Institutes of Health US Census Bureau

Current Organization Hosted by the National Center for Health Statistics (NCHS) NCHS Staffing Q-Bank Administrator Dedicated Programmer Steering Committee Chair Steering Committee NCHS, Census, BLS, NCI, NSF, GAO

Future Goals Full inclusion of evaluation methods Ongoing programming revisions Usability testing Increase participation among existing agencies and collaboration with additional agencies Developing standards for evaluation reports Potential integration with other survey methods tools

Q-Bank Website: http://wwwn. cdc. gov/qbank/home Q-Bank Website: http://wwwn.cdc.gov/qbank/home.aspx Q-Bank e-mail: qbank@cdc.gov