1 Renee M. Gindi NCHS Federal Conference on Statistical Methodology Statistical Policy Seminar December 4, 2012 Responsive Design on the National Health Interview Survey: Opportunities and Challenges Division of Health Interview Statistics National Center for Health Statistics

2 Objectives  National Health Interview Survey (NHIS) background  Potential features of responsive design on NHIS  Opportunities  Challenges

3  Conducted by National Center for Health Statistics  Nationally representative  Representative monthly sample  In-person interviews  35-40,000 household interviews/year  Fielded by U.S. Census Bureau  ~700 interviewers in 6 regional offices The National Health Interview Survey (NHIS)  1 hour face-to-face interview – no incentives

4 Refusal Rates, NHIS

5 Sources of Paradata on NHIS  Contact History Instrument (CHI)  Used on other surveys fielded by Census  Front/Back sections of the survey instrument  Tailored to NHIS  Blaise audit trails  Used to produce item/interview times

6 Recent Paradata Research from NHIS  Using Statistical Process Control to monitor data quality estimates (item nonresponse, item time) over time  Using CHI variables to estimate response propensity  response propensity and measurement error  response propensity and survey outcomes

7 Looking Ahead: Responsive Design on NHIS  Some elements of responsive design  Monitoring performance indicators  Change design based on monitoring survey outcomes  Target interventions to subsets using response propensity  Timeline: 2016 sample redesign

8 Looking Ahead: Responsive Design on NHIS  Opportunities  Real-time access to operations data  New ways to estimate survey quality  Challenges  Selecting and prioritizing survey outcome estimates  How, when, and where data collection phases should shift

9 Real-Time Access to Operations Data  Census Bureau’s Unified Tracking System (UTS)  More information to make better decisions quickly  Daily data update and historical data  Flexibility in reports  NHIS-specific indicators on tracked on UTS Demographic Race Income Education Employment Health Usual place of care Needs help with personal care Response quality First /Last Name Consent for linkage Adult SSN Telephone number

10 New Ways to Estimate Survey Quality  Trying to identify measures that can help assess, reduce, and correct for nonresponse bias in our health estimates  Adding new interviewer observation questions on responders and non-responders  Physical condition of the sample unit  Household income, employment status  Health-related indicators

11 Identifying priority estimates: 15 Selected Health Measures  Lack of health insurance coverage and type of coverage  Usual place to go for medical care  Obtaining needed medical care  Receipt of influenza vaccination  Receipt of pneumococcal vaccination  Obesity  Leisure-time physical activity  Current smoking  Alcohol consumption  Human immunodeficiency virus (HIV) testing  General health status  Personal care needs  Serious psychological distress  Diagnosed diabetes  Asthma episodes and current asthma

12 “Phase Shifts”: How, When, and Where?  How can we “sufficiently alter” NHIS protocol?  Mode shift? Shift to “core” survey? Introduce incentives?  When should protocol be altered given a monthly sample and production cycle?  Is a 7-10 day window wide enough to achieve response goals?  Where should protocol be altered?  Nationally? Regional Office? State?

13 Renee M. Gindi, Ph.D. Phone: Thank you!

14 EXTRA SLIDES

15 SPC: Sample Adult Interview Pace (seconds per question) Control Chart, Regional Office 1, Cluster 4

16 Response Propensity and Measurement Bias

17 Correlations between CHI Measures and Participation and Health Outcomes : NHIS, 2010 CHI Measure Absolute Value of Correlation with Family/Sample Adult Participation Absolute Value of Correlations with 76 Health Outcomes RangeAverage Refusal concerns.73/ Time constraints.15/ Privacy/trust.45/ Gate keeping.27/ Number of concerns.62/ Number of contact attempts.49/ Health problem.19/ =.70: Strong/very strong