Trena M. Ezzati-Rice, Frederick Rohde, Robert Baskin Evaluation of Respondents’ Reporting of Medical Events: Relationship to Response Propensities and Other Multiple Measures in the Medical Expenditure Panel Survey (MEPS) Trena M. Ezzati-Rice, Frederick Rohde, Robert Baskin International Total Survey Error Workshop Tallberg, Sweden June 14-17, 2009

Motivation for research This research effort was motivated by results presented at 2008 Joint Statistical Meetings (Rohde) Some observed tendency for reduced reporting of mean number of medical events after Round 1 in the longitudinal MEPS with 5 rounds of interviews covering a 2 year period

Why is it important to investigate reduced reporting of medical events? Reports of medical events are a key survey statistic in MEPS Reduced event reporting can lead to lower estimates of total health care expenditures MEPS health care expenditure data are used to inform health policy

Goal of this research To explore multiple dimensions hypothesized to potentially affect the quality of medical event reporting in MEPS Respondent socio-demographic characteristics Respondent health-related characteristics Household and environmental characteristics Respondents’ Round 1 survey response propensity Para data (survey administrative data)

Research questions Is reduced event reporting after Round 1 related to respondents’ Round 1 response propensity? Is reduced event reporting after Round 1 related to length of the Round 1 interview and/or number of household contacts? Is type of respondent (cooperative vs. reluctant) at Round 1 related to reduced event reporting post Round 1? What role do respondent socio-demographic characteristics play in reduced event reporting after controlling for other variables? Is there a relationship between reduced event reporting and the length of interview in the prior survey? MEPS sampling frame – subsample of respondents to the prior year’s National Health Interview Survey (NHIS)

MEPS survey background Annual since 1996; nationally representative sample of households 5 rounds of data collection covering a 2 year period Used to estimate medical care utilization, access to care, and health care expenses Primary source of individual and family level health care expenditure information for U.S. noninstitutionalized population

Medical events in MEPS During household interview, respondents report all their medical events for defined reference period Event reporting relies on respondent reports (1 respondent reports for entire family) Respondents report charges and payments for each medical event Expenditure data also collected from sample of medical providers to improve accuracy and reduce amount of missing data Key: Expenditures linked to reporting of medical events.

Data 4 MEPS panels Panel 9 thru Panel 11 (all 5 rounds): 2004-2006 Panel 12 (3 rounds currently available): 2007 Sample: N = 34,443 respondents with event data in Round 1 and 1+ round after Round 1 4 individual events Hospital inpatient stays Dental visits Prescription medicine purchases Ambulatory visits (office-based doctor visits, hospital outpatient visits, emergency room visits)

Analysis methods Descriptive statistics on mean event reporting across rounds by panel and individual events Events standardized for varying number of days in survey round (events per 100 round days) Calculate Round 1 person level respondents’ response propensities Logit estimated using set of covariates used to adjust for year 1 attrition in MEPS (scores grouped into quintiles) SAS stepwise multiple logistic regression models for the 4 individual events Dichotomous dependent variable (Y): decreased event reporting in at least 1 Round after Round 1: 1 = yes; 0 = no 4 types of independent variables (X)

Independent variables in the models (multiple types of data) 1. Round 1 respondent characteristics (demographic, health) Age, sex, race/ethnicity, education Health status, insurance status 2. Environmental characteristics Round 1 family size Geographic region Metropolitan/non-metropolitan area status 3. Round 1 person level response propensity (calculated from logit model) 4. Para data (next slide) 5. MEPS Panel (9-12) Specifically we explored replicated imputation as a methodology for estimating the effect of imputation on variances of expenditure estimates associated with one particular medical event. This presentation is a followup to last year’s initial study.

Para data included in models Round 1 (R1) interview time Number of household contacts made at R1 R1 interview type (in-person versus phone) R1 level of cooperation (cooperative vs. reluctant) R1 number of break-offs during the interview NHIS (MEPS sampling frame source) interview time NHIS interview status (partial vs. complete)

Descriptive Statistics MEPS is a longitudinal panel survey that has been ongoing since 1996. it is a nationally representative survey of the civilian noninstitutionalized population. For sampling efficiency and to enhance analytical capacity, our sample is a subsample of respondents to the previous year’s NHIS. The sample is a stratified multi-stage probability proportional to size selection of HHs and for most years the sample consists of about 200 PSUs.

Mean number of Rx and ambulatory events per 100 round days, by Panel MEPS is a longitudinal panel survey that has been ongoing since 1996. it is a nationally representative survey of the civilian noninstitutionalized population. For sampling efficiency and to enhance analytical capacity, our sample is a subsample of respondents to the previous year’s NHIS. The sample is a stratified multi-stage probability proportional to size selection of HHs and for most years the sample consists of about 200 PSUs.

Impact of differential reporting after Round 1 Prior 2008 analysis (Rohde) If no differential reporting after Round 1, prediction model showed that estimates of total medical utilization and expenses for 1996-2005 would have increased by 4-22 percent Thus, those results motivated this analysis Individual event level analysis Examine multiple dimensions potentially related to reduced event reporting Panel 11 Round 3 Specifically we explored replicated imputation as a methodology for estimating the effect of imputation on variances of expenditure estimates associated with one particular medical event. This presentation is a followup to last year’s initial study.

Percent with Rx event among respondents in both R1 and 1+ rounds after R1 MEPS is a longitudinal panel survey that has been ongoing since 1996. it is a nationally representative survey of the civilian noninstitutionalized population. For sampling efficiency and to enhance analytical capacity, our sample is a subsample of respondents to the previous year’s NHIS. The sample is a stratified multi-stage probability proportional to size selection of HHs and for most years the sample consists of about 200 PSUs.

Percent with ambulatory events among respondents in both R1 and 1+ rounds after R1 MEPS is a longitudinal panel survey that has been ongoing since 1996. it is a nationally representative survey of the civilian noninstitutionalized population. For sampling efficiency and to enhance analytical capacity, our sample is a subsample of respondents to the previous year’s NHIS. The sample is a stratified multi-stage probability proportional to size selection of HHs and for most years the sample consists of about 200 PSUs.

Conditional mean number of Rx events (per 100 days in the round) for R1 versus R2-5 MEPS is a longitudinal panel survey that has been ongoing since 1996. it is a nationally representative survey of the civilian noninstitutionalized population. For sampling efficiency and to enhance analytical capacity, our sample is a subsample of respondents to the previous year’s NHIS. The sample is a stratified multi-stage probability proportional to size selection of HHs and for most years the sample consists of about 200 PSUs.

Conditional mean number of ambulatory events (per 100 days in the round) for R1 versus R2-5 MEPS is a longitudinal panel survey that has been ongoing since 1996. it is a nationally representative survey of the civilian noninstitutionalized population. For sampling efficiency and to enhance analytical capacity, our sample is a subsample of respondents to the previous year’s NHIS. The sample is a stratified multi-stage probability proportional to size selection of HHs and for most years the sample consists of about 200 PSUs.

Initial model (12 variables) Stepwise logistic regression models of lower event reporting after Round 1 (4 individual medical events) Initial model (12 variables) Panel Response propensity rank Demographics (age, sex, race/ethnicity, education, family size) Interview type (in-person vs. phone) Health status and health insurance Region and MSA Stepwise inclusion of para data to determine final model (6 variables) 0.05 significance for stepwise selection into the model Specifically we explored replicated imputation as a methodology for estimating the effect of imputation on variances of expenditure estimates associated with one particular medical event. This presentation is a followup to last year’s initial study.

Lower Ambulatory visits Selected results of logistic regression predicting fewer standardized events after Round 1: Demographics and health   Lower Rx purchases Lower Ambulatory visits Indep Variable Estimate p-value Odds ratio R1 Age 0-17 -0.08 0.173 0.97 0.05 0.394 1.17 18-24 0.06 0.163 1.12 0.03 0.513 1.44 25-44 0.07 0.016 1.13 0.025 1.18 45-64 0.01 0.821 1.06 -0.03 0.218 1.08 Sex Female 0.018 0.376 1.02 R1 Health Fair-Poor 0.10 0.000 1.21 Race/ethnicity Hispanic 0.009 0.99 0.11 1.26 Non-H Asian -0.14 0.003 0.80 -0.07 0.155 1.05 Non-H Black -0.02 0.547 0.90 0.08 0.005 1.22 MEPS is a longitudinal panel survey that has been ongoing since 1996. it is a nationally representative survey of the civilian noninstitutionalized population. For sampling efficiency and to enhance analytical capacity, our sample is a subsample of respondents to the previous year’s NHIS. The sample is a stratified multi-stage probability proportional to size selection of HHs and for most years the sample consists of about 200 PSUs.

Lower Ambulatory visits Selected results of logistic regression predicting fewer standardized events after Round 1: propensity and selected para data   Lower Rx purchases Lower Ambulatory visits Indep Variable Estimate p-value Odds ratio Propensity Rank .056-.831 0.07 0.030 1.13 0.04 0.146 1.12 .831-.869 -0.01 0.601 1.04 0.00 0.960 1.07 .869-.891 0.02 0.423 1.08 0.918 .891-.909 -0.02 0.442 0.504 R1 Cooperation Level Nonreluctant 0.15 0.000 1.35 0.11 1.25 R1 Interview Time  31-45 min -0.33 0.83 -0.46 0.71 46-60 -0.10 0.001 -0.20 0.92 61-75 0.458 0.523 1.11 76-90 1.33 0.09 1.24 91-120 0.16 0.25 1.46 121+ 0.27 1.50 0.45 1.78 R1 Number contacts 0-1 0.20 1.27 1.10 2-5 1.14 0.003 1.06 11-15 -0.14 0.004 0.91 -0.04 0.379 0.95 16+ -0.11 0.053 0.94 -0.15 0.006 0.85 MEPS is a longitudinal panel survey that has been ongoing since 1996. it is a nationally representative survey of the civilian noninstitutionalized population. For sampling efficiency and to enhance analytical capacity, our sample is a subsample of respondents to the previous year’s NHIS. The sample is a stratified multi-stage probability proportional to size selection of HHs and for most years the sample consists of about 200 PSUs.

Significant predictors of reduced event reporting after Round 1 by event type: results of stepwise logistic regression Rx Dental Ambulatory Inpatient Effect Adjusted R2 Initial: .0140 Final: .0251 Initial: .0689 Final: .0712 Initial: .0200 Final: .0395 Initial: .0551 Final: .0897 Panel √ R propensity ns Age Sex Race/ethnicity Education Family size Interview type Health status Health insurance Region MSA R1 interview time 1 √ R1 break offs 4 √ 3 √ R1 reluctant responder 2 √ R1 number of contacts MEPS is a longitudinal panel survey that has been ongoing since 1996. it is a nationally representative survey of the civilian noninstitutionalized population. For sampling efficiency and to enhance analytical capacity, our sample is a subsample of respondents to the previous year’s NHIS. The sample is a stratified multi-stage probability proportional to size selection of HHs and for most years the sample consists of about 200 PSUs.

Summary 1 We analyzed respondents’ Round 1 survey response propensity along with respondents demographic and health characteristics, environmental characteristics, and survey para data. Data were analyzed to explore whether they contribute to reduced medical event reporting after Round 1 ( 4 separate events). Concern for reduced medical event reporting -- can lead to reduced estimates of total medical expenditures. Key component of this study: first use of MEPS para data. Results of individual event level models suggest some important correlates of reduced event reporting after Round 1. Specifically we explored replicated imputation as a methodology for estimating the effect of imputation on variances of expenditure estimates associated with one particular medical event. This presentation is a followup to last year’s initial study.

Summary 2 While the explained variance (R2) of the models was low ranging from 2.5% for Rx purchases to about 9% for inpatient stays, the para data did increase the predictive power of the models of reduced event reporting. Respondent Round 1 response propensity was only significant for one event – inpatient stays. NHIS interview time and partial vs. complete interview status were not significant predictors of reduced medical event reporting in the MEPS follow-on survey. In the stepwise logistic regression analysis, length of the Round 1 interview entered the model early and was highly significant and with highest Wald Chi-square. Number of contacts at Round 1 was also highly significant. Specifically we explored replicated imputation as a methodology for estimating the effect of imputation on variances of expenditure estimates associated with one particular medical event. This presentation is a followup to last year’s initial study.

Limitations One limitation of this analysis is that it does not directly address measurement error. Secondly, analysis ignores fact that measurement error can be associated with a number of the covariates included in the models. Another major limitation is lack of information about response for persons not in the dataset, thus modeled response propensity is likely a weaker model. For this analysis, Panel 12, Round 3 was only available for the first year of data collection. Round 3 overlaps two calendar years. Specifically we explored replicated imputation as a methodology for estimating the effect of imputation on variances of expenditure estimates associated with one particular medical event. This presentation is a followup to last year’s initial study.

Future research Refine the models Incorporate interactions Determine best categories for various variables Incorporate interviewer characteristics in the models (demographics, experience level, etc.) Include NHIS para data Conduct more in-depth analysis of how Round 1 interview time relates to reduced event reporting and potential impact on expenditure estimates. Continued evaluations of data quality for key MEPS analytical variables. Specifically we explored replicated imputation as a methodology for estimating the effect of imputation on variances of expenditure estimates associated with one particular medical event. This presentation is a followup to last year’s initial study.

Conclusion Thanks Questions/comments Ideas/suggestions for future research Specifically we explored replicated imputation as a methodology for estimating the effect of imputation on variances of expenditure estimates associated with one particular medical event. This presentation is a followup to last year’s initial study.