Does Inclusion of Both Partial and Complete Interviews from the Source Sampling Frame Have an Effect on Nonresponse Error and Measurement Error in a National Health Survey? Trena M. Ezzati-Rice, Frederick Rohde, Steven B. Cohen International Total Survey Error Workshop 2010 Stowe, VT June 13-16, 2010

Motivation for research Large observed differences in response rates between two types of previously interviewed cases included in the Medical Expenditure Panel Survey (MEPS) sample as a result of its integration with another national health survey Large observed differences in response rates between two types of previously interviewed cases included in the Medical Expenditure Panel Survey (MEPS) sample as a result of its integration with another national health survey

MEPS survey background Annual survey since 1996; nationally representative sample of households Annual survey since 1996; nationally representative sample of households 5 rounds of data collection covering 2 calendar years 5 rounds of data collection covering 2 calendar years Used to estimate medical care utilization, access to care, and health care expenses for the U.S. civilian noninstitutionalized population Used to estimate medical care utilization, access to care, and health care expenses for the U.S. civilian noninstitutionalized population Widely used to inform health care policy Widely used to inform health care policy Integrated survey design Integrated survey design – Each annual panel of households is a subsample of responding households (from prior year) from another large ongoing U.S. health survey, the National Health Interview Survey (NHIS)

Integration of NHIS and MEPS MEPS inherits the NHIS complex sample design with its oversampling of selected minority populations MEPS inherits the NHIS complex sample design with its oversampling of selected minority populations Reduces household screening costs for MEPS Reduces household screening costs for MEPS Provides valuable sampling frame information to adjust for survey nonresponse Provides valuable sampling frame information to adjust for survey nonresponse Provides enhanced analytical potential (NHIS and MEPS linked analyses) Provides enhanced analytical potential (NHIS and MEPS linked analyses) MEPS response rate is conditioned on the NHIS response rate MEPS response rate is conditioned on the NHIS response rate MEPS inherits the composition of the NHIS interviewed sample MEPS inherits the composition of the NHIS interviewed sample – Completed household interviews – “Partially Completed” household interviews

Definition of an NHIS “partially completed” interview All questionnaire modules are not completed All questionnaire modules are not completed NHIS questionnaire composed of 5 major sections NHIS questionnaire composed of 5 major sections 1. Household composition (demographic information) 1. Household composition (demographic information) 2. Family core (basic health and sociodemographic data) 2. Family core (basic health and sociodemographic data) Family relationships, marital status Family relationships, marital status Health status, activity limitations, injury, poisoning Health status, activity limitations, injury, poisoning Access and utilization and citizenship Access and utilization and citizenship Health insurance Health insurance Educational attainment STOP Educational attainment STOP Employment, earnings, or income data for partials NO Data – 3. Sample adult qx. NO Data – 4. Sample child qx. NO Data – 5. Immunization section NO Data

Data Matrix: NHIS and MEPS sample integration NHIS complete Interviews MEPS Respondents MEPS Nonrespondents MEPS Respondents MEPS Nonrespondents NHIS partially completed interviews NHIS Sample Frame (NHIS Respondents eligible for MEPS)

Background for this research Previously conducted research (Chiu et al, 2001) indicated that “late”/difficult” NHIS interviews were more likely to be partial completes. Previously conducted research (Chiu et al, 2001) indicated that “late”/difficult” NHIS interviews were more likely to be partial completes. Partially completed NHIS cases in the MEPS sample observed to: Partially completed NHIS cases in the MEPS sample observed to:  require more field effort  have lower response rates compared to the NHIS complete cases

Research questions Do the NHIS partially completed cases as included in the MEPS Do the NHIS partially completed cases as included in the MEPS – Have lower response propensity at Round 1 and subsequent rounds? – Adversely affect data collection burden? Are there differences in the demographic and socioeconomic characteristics of the MEPS respondents when stratified by their NHIS interview status (partial versus complete)? Are there differences in the demographic and socioeconomic characteristics of the MEPS respondents when stratified by their NHIS interview status (partial versus complete)? Do the NHIS partially completed cases as carried over to MEPS affect survey data quality? Do the NHIS partially completed cases as carried over to MEPS affect survey data quality? – Do they have higher item nonresponse rates on MEPS variables? – Do MEPS survey estimates differ when stratified by partial vs. complete NHIS interview status?

Source of Data MEPS MEPS – Historical data for examining response rate trends – Panels 11 & 12 for detailed analyses – Paradata (survey burden assessment) – Survey data (item nonresponse and weighted survey estimates) NHIS NHIS – Paradata (NHIS interview status and other data) – Survey data (response propensity modeling )

Percentage of MEPS reporting units with a previous NHIS partially completed interview: MEPS Panels 3-14 Year MEPS Panel Started Panel NHIS “partially completed” interview (%)

MEPS Round 1 response rates (reporting unit level) by NHIS interview completion status:

Percent refused MEPS interview (at the RU level) by round and NHIS interview status, MEPS Panel 12 Round NHIS Partial (%) NHIS Complete (%) 128.7* * *p<.001

Percent ever refused (reluctant respondent or final refusal) by Round and NHIS interview completion status: RU level MEPS Panel 12

Percent of noncontacts and true contacts by NHIS interview status: MEPS Panel 12, Round 1 ParadataNHISpartialNHIScompleteSignif Round 1 noncontacts p<.001 Round 1 “true” contacts p<.001

Other paradata measures by NHIS interview status: MEPS Panel 12, Round 1 Paradata NHIS partial NHIS complete Signif Reluctant respondent (%) p<.001 Not located (%) p<.001 MEPS interview type In person (%) In person (%) Phone (%) Phone (%) p<.001 Mean interview time (minutes) p=.232 Any break-offs during the interview (%) p=.006

Bivariate analysis of R1 respondent characteristics (reference person) by NHIS interview completion status: MEPS Panels 11 & 12 combined Bivariate analysis of R1 respondent characteristics (reference person) by NHIS interview completion status: MEPS Panels 11 & 12 combined Characteristic Overall (%) Partial (%) Complete(%) Age of Reference person* Race/ethnicity * Hispanic Hispanic Non-Hispanic Black Non-Hispanic Black Non-Hispanic Asian Non-Hispanic Asian Non-Hispanic Other Non-Hispanic Other Marital Status* Married Married Widowed/separated/divorced Widowed/separated/divorced Never married Never married Year 1 Poverty Status* In/near poverty In/near poverty Low income Low income Middle income Middle income High income High income Missing Missing * significant at alpha = 0.05

Bivariate analysis (cont.) of R1 respondent characteristics (MEPS reference person) by NHIS interview completion status: MEPS Panels 11 & 12 combined (* = significant at alpha =.05) Characteristic Overall (%) Partial (%) Complete (%) Size of RU* Employment status* Employed Employed Region* Northeast Northeast Midwest Midwest South South West West MSA status* MSA MSA Non MSA Non MSA

Logistic regression analysis of Round 1 (RU) respondent (reference person charac) being a prior NHIS partial complete: MEPS Panels 11 & 12 EffectDFWald X 2 Pr > X 2 EffectDFWald X 2 Pr > X 2 Race/ethnicity352.1<.0001 Yr1 poverty status541.9<.0001 No. of people in RU Family structure528.5<.0001 Employment status Region MSA status115.3<.0001

Odds ratios of logistic model predicting a “NHIS Partial Complete” respondent at MEPS Round 1, Panels 11 & 12 Race/ethnicityMSA Asian1.61* MSA 1.31* Asian1.61* MSA 1.31* Black1.31* Non-MSA1.00 Black1.31* Non-MSA1.00 Hispanic1.45* Hispanic1.45* White/Other1.00 White/Other1.00 Reporting Unit SizePoverty Status 10.26* In poverty * In poverty * Low income * Low income * Middle income * Middle income Missing income1.52* Missing income1.52* Near poverty Near poverty1.10 High income1.00 High income1.00Region Midwest0.84*Employment Status Midwest0.84*Employment Status Northeast0.95 Employed1.14* Northeast0.95 Employed1.14* South0.85* Missing/<16 yr2.42* South0.85* Missing/<16 yr2.42* West1.00 Unemployed1.00 West1.00 Unemployed1.00

Predicting Round 1 dwelling unit level response propensity potential (NHIS) covariates as used in MEPS weights production Demographic Household Characteristics Socio- Economic Status Geographic Health- related Age ref. person DU size Poverty status Census region Health status Race/ethnicity Has phone Education MSA size Need help Marital status Working or not IncomeMSA/nonMSA # nts. in hospital Gender Type of PSU Employment status Urban/Rural Healthcare coverage Any Asian Type of home – house, apt., etc. Home ownership Medical expenses category Any Black Time w/ no phone Interview language U.S. citizen U.S. born

Additional NHIS para data included in logit regression (predicting round 1 response propensity) -- * available for MEPS panel 12 only NHIS interview completion status (partial vs. complete) NHIS interview completion status (partial vs. complete) – Panels 11 and 12 Household cooperativeness* Household cooperativeness* How likely respond to later linked survey* How likely respond to later linked survey* # of contacts and # of non-contacts* # of contacts and # of non-contacts* Language problem* Language problem* Health problem* Health problem* Time constraints* Time constraints* Content/privacy concerns* Content/privacy concerns* Hostility mentioned* Hostility mentioned*

Results of logistic regression predicting round 1 DU level response propensity -- MEPS Panel 11 EffectDF Wald X 2 Pr > X 2 EffectDF Wald X 2 Pr > X 2 Predicted poverty Age Race/ethnicity <.0001 Marital status Income4 48.3<.0001 DU size4 36.5<.0001 MSA status2 21.0<.0001 Region Has phone3 27.0<.0001 Home type Interview language U.S. born NHIS interview completion status1105.4<.0001

Results of logistic regression predicting round 1 response propensity -- MEPS Panel 12 (additional NHIS paradata available) EffectDFWald X 2 Pr > X 2 EffectDFWald X 2 Pr > X 2 Race/ethnicity <.0001 Marital status Income DU size MSA status Region Has phone PSU type Interview language Health status NHIS interview completion status Med expend category HH cooperativeness428.3 <.0001 Likely to respond to later survey # contacts w/ sample unit635.5 <.0001 # noncontacts530.7 <.0001 Privacy concerns120.6 <.0001 Hostility mentioned

RU level item nonresponse rates, MEPS Panel 12 (if any respondent in the RU had a missing value in year 1) VariableNHIS Partial (%) NHIS Complete (%) Signif Yr1 poverty status p<.001 Education3.92.1p=.002 High blood pressure (>17) p<.001 High cholesterol (>17) p=.025 Diabetes diagnosis p=.059 How often dental check up p<.001 How long last routine check up (>17) p<.001 How long last flu shot (>17) p<.001 How long since mammogram (>29) p<.001 How long since last PSA (>39, M) p<.001 Usual source of care p<.001 Employment status p=.014 Mental health status p=.014 Health status p=.059

Comparison of selected Year 1 estimates (percents) according to NHIS interview completion status: MEPS Panels 11 & 12 (* = signif p <.05) Person level health Panel 11 Panel 12 measurePartialCompletePartialComplete Insurance (<65) Any private Any private * Public only Public only Uninsured Uninsured Any activity limitation* High cholesterol dx (>17)* Office based provider visit* Rx (including refills)* Total mean expenditures* $2,760$3,815$3,171$3, contacts 5+ contacts$2,484 $3,390* $3,390*$3,021$3,823

MEPS Year 1 estimates (in percent) for All MEPS respondents versus excluding NHIS partial cases (person weights for NHIS completes re-calculated treating partials as nonrespondents) (* = signif p<.05) Person level health Panel 11 Panel 12 measureAllCompletes Only (rewt) All Completes Only (rewt) Insurance (<65) Any private Any private Public only Public only Uninsured Uninsured Any activity limitation * High cholesterol dx (>17)* Office based provider visit* Rx (including refills)* Total mean expenditures $3,593$3,723*$3,802$3, contacts 5+ contacts$3,162$3,313*$3,647$3,702

Summary We examined the impact of the carry-over of two types of prior interview cases in the MEPS as a result of its integration with the NHIS. We examined the impact of the carry-over of two types of prior interview cases in the MEPS as a result of its integration with the NHIS. In particular, we wanted to examine if the partially completed NHIS cases may bias MEPS survey estimates and impact survey burden. In particular, we wanted to examine if the partially completed NHIS cases may bias MEPS survey estimates and impact survey burden. NHIS partial cases had significantly higher ever refused rates, higher contact rates, lower response rates at MEPS Round 1, and higher attrition rates. NHIS partial cases had significantly higher ever refused rates, higher contact rates, lower response rates at MEPS Round 1, and higher attrition rates. Partial versus complete was a significant predictor of MEPS Round 1 response propensity controlling for other variables. Partial versus complete was a significant predictor of MEPS Round 1 response propensity controlling for other variables.

Summary (cont.) Item nonresponse rates were higher for the partially completed cases relative to the NHIS completes for a number of MEPS key variables. Item nonresponse rates were higher for the partially completed cases relative to the NHIS completes for a number of MEPS key variables. Estimates for several selected health items were different between the NHIS partials and completes as carried over in the MEPS sample. Estimates for several selected health items were different between the NHIS partials and completes as carried over in the MEPS sample. Excluding the NHIS partial cases from the MEPS survey estimates (and re-weighting) resulted in slightly higher estimates of selected conditions, medical events, and expenditures. Excluding the NHIS partial cases from the MEPS survey estimates (and re-weighting) resulted in slightly higher estimates of selected conditions, medical events, and expenditures. Thanks Thanks

Discussion Initial exploratory analysis Initial exploratory analysis – What additional analyses can be carried out to assess potential nonresponse bias and measurement error bias related to the two types of interview cases in the MEPS? – What approaches could be explored to correct for nonresponse or measurement error bias? Should MEPS consider excluding or subsampling the NHIS partials in future panels of the MEPS? Should MEPS consider excluding or subsampling the NHIS partials in future panels of the MEPS? Ideas/suggestions for future research? Ideas/suggestions for future research?

Logistic regression analysis of the insured (<65 years) reference person: testing for NHIS interview status effect, MEPS, Panels 11 and 12 (RU level analysis) Panel 11 Panel 12 Panel DF Wald F p-valueDF P-valueDF p-value Age <.0001 Sex <.0001 Race/ethnicity399.39< < <.0001 Highest year of education < < <.0001 Poverty status (Y1) < < <.0001 Region339.01< < <.0001 Total health care Expenditures < < <.0001 NHIS Interview Completion Status