1 Assessment of Potential Bias in the National Immunization Survey (NIS) from the Increasing Prevalence of Households Without Landline Telephones Meena Khare (NCHS), James A. Singleton (NCIRD), Abera Wouhib (NCHS), and Nidhi Jain (NCIRD) National Immunization Conference, 2008 The findings and conclusions in this paper are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.

2 National Immunization Survey (NIS) Ages months Monitors vaccination coverage in US states and selected local areas Conducted present Large random-digit-dialing landline telephone survey Vaccination history from child’s provider(s) Annual sample size ~30,000 children with interview data ~21,000 with provider reported data ~ 65% household response rate

3 NIS-Teen Ages years Uses the NIS sampling frame of landline telephone number Conducted Q (national sample) 5,468 household interviews conducted 2,882 teens had adequate provider reported vaccination histories 56.2% household response rate Repeated Q (national sample) annual sample by state and selected local areas

4 National Health Interview Survey (NHIS) Conducted annually since 1957 Nationally representative sample Civilian non-institutionalized US population Face-to-face interviews Selects ~40,000 households and ~100,000 persons Detailed health information for 1 adult & 1 child per family Oversamples NH-blacks, Hispanics; Asians (in 2006) Response rates 87.3% household 78.8% sample child (<18 years)

5 NHIS Covers both landline telephone and non-landline telephone households Households connected by landline telephone (LT) Households without landline telephone service (NLT) Wireless-only households (2003+) Households without any phone service

6 Ref: Blumberg et al., Wireless substitution: Early release of estimates based on data from the NHIS, July-December 2006, NCHS ; children<=17 yrs

7 Percentage of Children Living in Landline, Wireless Only, and Phoneless Households, NHIS 2006 Children aged 1 – 4 years (N = 2,201)Landline Telephone Wireless OnlyPhoneless All GenderBoys Girls Hispanic Race/EthnicityNH White NH Black < 100% Poverty Level % % %

8 Teens aged 13 – 17 years (N = 3,044)Landline Telephone Wireless OnlyPhoneless All GenderBoys Girls Hispanic Race/EthnicityNH White NH Black NH Other < 100% Poverty Level % % % Percentage of Teens Living in Landline, Wireless Only, and Phoneless Households, NHIS 2006

9 Methods Assess bias due to exclusion of non-landline households using 2006 NHIS sample Outcome measure: parental report of child’s influenza vaccination in the past 12 months Compared estimates from all NHIS sample to NHIS landline (LT) sample estimates Used weighting methodology similar to NIS to adjust LT sample estimates for exclusion of children from the non-landline sample Compared NHIS estimates with NIS-Teen 2006 estimates based on parental report (benchmarking)

10 Influenza Vaccination Rates Children 1- 4 Years

11 Influenza Vaccination Rates among Children aged 1-4 Years Landline vs. Non-landline Households, NHIS 2006

12 Influenza Vaccination Rates among Children 1-4 Years Overall vs. Adjusted Landline (LT) Estimates, NHIS 2006

13 Influenza Vaccination Rates among Children 1-4 Years Comparisons by Selected Factors, NHIS 2006 *No differences were statistically significant (p > 0.05) Children 1-4 years (N = 2,201) Overall NHIS 2006 (%) Landline Telephone (LT) (%) Non- Landline (NLT) (%) Adjusted LT (%) Difference* = Adjusted LT - Overall NHIS (%) All Poverty Level: < 100% % - 199% % - 399% % Dr. office visits: None – 5 office visits office visits

14 Continued… *No differences were statistically significant (p > 0.05) Children 1-4 years (N = 2,201) Overall NHIS 2006 (%) Landline Telephone (LT) (%) Non- Landline (NLT) (%) Adjusted LT (%) Difference* = Adjusted LT - Overall NHIS (%) All Ever been told child had asthma? Yes No Uninsured? Yes No Ever had chickenpox? Yes No

15 Influenza Vaccination Rates Adolescents Years

16 Influenza Vaccination Rates among Adolescents Years Landline vs. Non-landline Households, NHIS 2006

17 Influenza Vaccination Rates among Adolescents Years Overall vs. Adjusted Landline (LT) Estimates, NHIS 2006

18 Influenza Vaccination Rates among Adolescents Years Comparisons by Selected Factors, NHIS 2006 *No differences were statistically significant (p > 0.05) Adolescents years (N = 3,044) Overall NHIS 2006 (%) Landline Telephone (LT) (%) Non- Landline (NLT) (%) Adjusted LT (%) Difference* = Adjusted LT - Overall NHIS (%) All Poverty Level: < 100% % - 199% % - 399% % Dr. office visits: None – 5 office visits office visits

19 Continued… *No differences were statistically significant (p > 0.05) Adolescents years (N = 3,044) Total US, NHIS 2006 (%) Landline Telephone (LT) (%) Non- Landline (NLT) (%) Adjusted LT (%) Difference* = Adjusted LT - Overall NHIS (%) All Ever been told child had asthma? Yes No Uninsured? Yes No Ever had chickenpox? Yes No

20 Summary In 2006, 18% of young children and 10% of teens lived in non-landline telephone households ~78% of these households had access to wireless telephones In 2006, no significant bias was observed in the landline telephone sample estimates after adjusting for exclusion of non-landline telephone households

21 Limitations Not a direct evaluation of bias in NIS Uses parental-report of influenza vaccination as proxy measure for other vaccines Limited NHIS sample sizes for children in non-landline households, esp. for teens Differences between NIS and NHIS: Telephone vs. face-to-face interviews Different recall time period Different weighting methods

22 Benchmarking 2006/Q4 NIS-Teen to 2006 NHIS Compared estimated household-reported influenza vaccination rates Evaluated bias from NIS-Teen for exclusion of non- landline households and nonresponse Influenza vaccination rates among teens years 13.6% NIS-Teen, 11.9% NHIS Influenza vaccination rates among teens with asthma 23.8% NIS-Teen, 22.7% NHIS Limitations: differences in survey modes & periods of influenza vaccinations; NHIS is an address-based sample & does not have 100% response rates

23 Conclusions Findings from NHIS are reassuring but more direct evaluation of bias in NIS is needed We will continue to monitor trends in non-landline prevalence and potential bias in NIS estimates using NHIS Starting with 2007 data, we will also benchmark NIS household-reported influenza vaccinations among children mo to NHIS estimates

24 Charting the course for NIS… where are we headed?

25 The Challenge Develop short and long term strategies for addressing exclusion of non-landline households from the NIS sample frame, while… Maintaining a credible survey that will provide the information needed to monitor the U.S. immunization program with reliable, valid estimates

26 Strategy Evaluate potential bias NHIS analysis Evaluate NIS weighting enhancements Planning vaccination provider record check on NHIS sample in 2008 (infant, teen) Compare vaccination coverage estimates from 2007 NIS infant cell phone pilot study Add sampling frame with non-landline households e.g., cell phone sample, birth certificates/IIS, school-based sample for teens Replace current landline telephone frame? e.g., American Community Survey

27 Potential Bias in Landline Sample * Landline Bias = (% of population non-landline)*(Landline estimate – non-landline estimate)

28 Thank you Contact: Meena Khare or Jim Singleton