1. Trena M. Ezzati-Rice, Frederick Rohde, Robert Baskin
Evaluation of Respondents’ Reporting of Medical Events: Relationship to Response Propensities and Other Multiple Measures in the Medical Expenditure Panel Survey (MEPS)
Trena M. Ezzati-Rice, Frederick Rohde, Robert Baskin
International Total Survey Error Workshop
Tallberg, Sweden
June 14-17, 2009

2. Motivation for research
This research effort was motivated by results presented at 2008 Joint Statistical Meetings (Rohde)
Some observed tendency for reduced reporting of mean number of medical events after Round 1 in the longitudinal MEPS with 5 rounds of interviews covering a 2 year period

3. Why is it important to investigate reduced reporting of medical events?
Reports of medical events are a key survey statistic in MEPS
Reduced event reporting can lead to lower estimates of total health care expenditures
MEPS health care expenditure data are used to inform health policy

4. Goal of this research
To explore multiple dimensions hypothesized to potentially affect the quality of medical event reporting in MEPS
Respondent socio-demographic characteristics
Respondent health-related characteristics
Household and environmental characteristics
Respondents’ Round 1 survey response propensity
Para data (survey administrative data)

5. Research questions
Is reduced event reporting after Round 1 related to respondents’ Round 1 response propensity?
Is reduced event reporting after Round 1 related to length of the Round 1 interview and/or number of household contacts?
Is type of respondent (cooperative vs. reluctant) at Round 1 related to reduced event reporting post Round 1?
What role do respondent socio-demographic characteristics play in reduced event reporting after controlling for other variables?
Is there a relationship between reduced event reporting and the length of interview in the prior survey?
MEPS sampling frame – subsample of respondents to the prior year’s National Health Interview Survey (NHIS)

6. MEPS survey background
Annual since 1996; nationally representative sample of households
5 rounds of data collection covering a 2 year period
Used to estimate medical care utilization, access to care, and health care expenses
Primary source of individual and family level health care expenditure information for U.S. noninstitutionalized population

7. Medical events in MEPS
During household interview, respondents report all their medical events for defined reference period
Event reporting relies on respondent reports (1 respondent reports for entire family)
Respondents report charges and payments for each medical event
Expenditure data also collected from sample of medical providers to improve accuracy and reduce amount of missing data
Key: Expenditures linked to reporting of medical events.

8. Data 4 MEPS panels Panel 9 thru Panel 11 (all 5 rounds): 2004-2006
Panel 12 (3 rounds currently available): 2007
Sample: N = 34,443 respondents with event data in Round 1 and 1+ round after Round 1
4 individual events
Hospital inpatient stays
Dental visits
Prescription medicine purchases
Ambulatory visits (office-based doctor visits, hospital outpatient visits, emergency room visits)

9. Analysis methods
Descriptive statistics on mean event reporting across rounds by panel and individual events
Events standardized for varying number of days in survey round (events per 100 round days)
Calculate Round 1 person level respondents’ response propensities
Logit estimated using set of covariates used to adjust for year 1 attrition in MEPS (scores grouped into quintiles)
SAS stepwise multiple logistic regression models for the 4 individual events
Dichotomous dependent variable (Y): decreased event reporting in at least 1 Round after Round 1: 1 = yes; 0 = no
4 types of independent variables (X)

10. Independent variables in the models (multiple types of data)
1. Round 1 respondent characteristics (demographic, health)
Age, sex, race/ethnicity, education
Health status, insurance status
2. Environmental characteristics
Round 1 family size
Geographic region
Metropolitan/non-metropolitan area status
3. Round 1 person level response propensity (calculated from logit model)
4. Para data (next slide)
5. MEPS Panel (9-12)
Specifically we explored replicated imputation as a methodology for estimating the effect of imputation on variances of expenditure estimates associated with one particular medical event.
This presentation is a followup to last year’s initial study.

11. Para data included in models
Round 1 (R1) interview time
Number of household contacts made at R1
R1 interview type (in-person versus phone)
R1 level of cooperation (cooperative vs. reluctant)
R1 number of break-offs during the interview
NHIS (MEPS sampling frame source) interview time
NHIS interview status (partial vs. complete)

12. Descriptive Statistics
MEPS is a longitudinal panel survey that has been ongoing since it is a nationally representative survey of the civilian noninstitutionalized population.
For sampling efficiency and to enhance analytical capacity, our sample is a subsample of respondents to the previous year’s NHIS.
The sample is a stratified multi-stage probability proportional to size selection of HHs and for most years the sample consists of about 200 PSUs.

13. Mean number of Rx and ambulatory events per 100 round days, by Panel
MEPS is a longitudinal panel survey that has been ongoing since it is a nationally representative survey of the civilian noninstitutionalized population.
For sampling efficiency and to enhance analytical capacity, our sample is a subsample of respondents to the previous year’s NHIS.
The sample is a stratified multi-stage probability proportional to size selection of HHs and for most years the sample consists of about 200 PSUs.

14. Impact of differential reporting after Round 1
Prior 2008 analysis (Rohde)
If no differential reporting after Round 1, prediction model showed that estimates of total medical utilization and expenses for would have increased by 4-22 percent
Thus, those results motivated this analysis
Individual event level analysis
Examine multiple dimensions potentially related to reduced event reporting Panel 11 Round 3
Specifically we explored replicated imputation as a methodology for estimating the effect of imputation on variances of expenditure estimates associated with one particular medical event.
This presentation is a followup to last year’s initial study.

15. Percent with Rx event among respondents in both R1 and 1+ rounds after R1
MEPS is a longitudinal panel survey that has been ongoing since it is a nationally representative survey of the civilian noninstitutionalized population.
For sampling efficiency and to enhance analytical capacity, our sample is a subsample of respondents to the previous year’s NHIS.
The sample is a stratified multi-stage probability proportional to size selection of HHs and for most years the sample consists of about 200 PSUs.

16. Percent with ambulatory events among respondents in both R1 and 1+ rounds after R1
MEPS is a longitudinal panel survey that has been ongoing since it is a nationally representative survey of the civilian noninstitutionalized population.
For sampling efficiency and to enhance analytical capacity, our sample is a subsample of respondents to the previous year’s NHIS.
The sample is a stratified multi-stage probability proportional to size selection of HHs and for most years the sample consists of about 200 PSUs.

17. Conditional mean number of Rx events (per 100 days in the round) for R1 versus R2-5
MEPS is a longitudinal panel survey that has been ongoing since it is a nationally representative survey of the civilian noninstitutionalized population.
For sampling efficiency and to enhance analytical capacity, our sample is a subsample of respondents to the previous year’s NHIS.
The sample is a stratified multi-stage probability proportional to size selection of HHs and for most years the sample consists of about 200 PSUs.

18. Conditional mean number of ambulatory events (per 100 days in the round) for R1 versus R2-5
MEPS is a longitudinal panel survey that has been ongoing since it is a nationally representative survey of the civilian noninstitutionalized population.
For sampling efficiency and to enhance analytical capacity, our sample is a subsample of respondents to the previous year’s NHIS.
The sample is a stratified multi-stage probability proportional to size selection of HHs and for most years the sample consists of about 200 PSUs.

19. Initial model (12 variables)
Stepwise logistic regression models of lower event reporting after Round 1 (4 individual medical events)
Initial model (12 variables)
Panel
Response propensity rank
Demographics (age, sex, race/ethnicity, education, family size)
Interview type (in-person vs. phone)
Health status and health insurance
Region and MSA
Stepwise inclusion of para data to determine final model (6 variables)
0.05 significance for stepwise selection into the model
Specifically we explored replicated imputation as a methodology for estimating the effect of imputation on variances of expenditure estimates associated with one particular medical event.
This presentation is a followup to last year’s initial study.

20. Lower Ambulatory visits
Selected results of logistic regression predicting fewer standardized events after Round 1: Demographics and health
Lower Rx purchases
Lower Ambulatory visits
Indep Variable
Estimate
p-value
Odds ratio
R1 Age
0-17
-0.08
0.173
0.97
0.05
0.394
1.17
18-24
0.06
0.163
1.12
0.03
0.513
1.44
25-44
0.07
0.016
1.13
0.025
1.18
45-64
0.01
0.821
1.06
-0.03
0.218
1.08
Sex
Female
0.018
0.376
1.02
R1 Health
Fair-Poor
0.10
0.000
1.21
Race/ethnicity
Hispanic
0.009
0.99
0.11
1.26
Non-H Asian
-0.14
0.003
0.80
-0.07
0.155
1.05
Non-H Black
-0.02
0.547
0.90
0.08
0.005
1.22
MEPS is a longitudinal panel survey that has been ongoing since it is a nationally representative survey of the civilian noninstitutionalized population.
For sampling efficiency and to enhance analytical capacity, our sample is a subsample of respondents to the previous year’s NHIS.
The sample is a stratified multi-stage probability proportional to size selection of HHs and for most years the sample consists of about 200 PSUs.

21. Lower Ambulatory visits
Selected results of logistic regression predicting fewer standardized events after Round 1: propensity and selected para data
Lower Rx purchases
Lower Ambulatory visits
Indep Variable
Estimate
p-value
Odds ratio
Propensity Rank
0.07
0.030
1.13
0.04
0.146
1.12
-0.01
0.601
1.04
0.00
0.960
1.07
0.02
0.423
1.08
0.918
-0.02
0.442
0.504
R1 Cooperation Level
Nonreluctant
0.15
0.000
1.35
0.11
1.25
R1 Interview Time 
31-45 min
-0.33
0.83
-0.46
0.71
46-60
-0.10
0.001
-0.20
0.92
61-75
0.458
0.523
1.11
76-90
1.33
0.09
1.24
91-120
0.16
0.25
1.46
121+
0.27
1.50
0.45
1.78
R1 Number contacts
0-1
0.20
1.27
1.10
2-5
1.14
0.003
1.06
11-15
-0.14
0.004
0.91
-0.04
0.379
0.95
16+
-0.11
0.053
0.94
-0.15
0.006
0.85
MEPS is a longitudinal panel survey that has been ongoing since it is a nationally representative survey of the civilian noninstitutionalized population.
For sampling efficiency and to enhance analytical capacity, our sample is a subsample of respondents to the previous year’s NHIS.
The sample is a stratified multi-stage probability proportional to size selection of HHs and for most years the sample consists of about 200 PSUs.

22. Significant predictors of reduced event reporting after Round 1 by event type: results of stepwise logistic regression Rx
Dental
Ambulatory
Inpatient
Effect
Adjusted R2
Initial:
Final:
Initial:
Final: .0712
Initial:
Final:
Initial:
Final:
Panel √
R propensity ns
Age
Sex
Race/ethnicity
Education
Family size
Interview type
Health status
Health insurance
Region
MSA
R1 interview time
1 √
R1 break offs
4 √
3 √
R1 reluctant responder
2 √
R1 number of contacts
MEPS is a longitudinal panel survey that has been ongoing since it is a nationally representative survey of the civilian noninstitutionalized population.
For sampling efficiency and to enhance analytical capacity, our sample is a subsample of respondents to the previous year’s NHIS.
The sample is a stratified multi-stage probability proportional to size selection of HHs and for most years the sample consists of about 200 PSUs.

23. Summary 1
We analyzed respondents’ Round 1 survey response propensity along with respondents demographic and health characteristics, environmental characteristics, and survey para data.
Data were analyzed to explore whether they contribute to reduced medical event reporting after Round 1 ( 4 separate events).
Concern for reduced medical event reporting -- can lead to reduced estimates of total medical expenditures.
Key component of this study: first use of MEPS para data.
Results of individual event level models suggest some important correlates of reduced event reporting after Round 1.
Specifically we explored replicated imputation as a methodology for estimating the effect of imputation on variances of expenditure estimates associated with one particular medical event.
This presentation is a followup to last year’s initial study.

24. Summary 2
While the explained variance (R2) of the models was low ranging from 2.5% for Rx purchases to about 9% for inpatient stays, the para data did increase the predictive power of the models of reduced event reporting.
Respondent Round 1 response propensity was only significant for one event – inpatient stays.
NHIS interview time and partial vs. complete interview status were not significant predictors of reduced medical event reporting in the MEPS follow-on survey.
In the stepwise logistic regression analysis, length of the Round 1 interview entered the model early and was highly significant and with highest Wald Chi-square. Number of contacts at Round 1 was also highly significant.
Specifically we explored replicated imputation as a methodology for estimating the effect of imputation on variances of expenditure estimates associated with one particular medical event.
This presentation is a followup to last year’s initial study.

25. Limitations
One limitation of this analysis is that it does not directly address measurement error.
Secondly, analysis ignores fact that measurement error can be associated with a number of the covariates included in the models.
Another major limitation is lack of information about response for persons not in the dataset, thus modeled response propensity is likely a weaker model.
For this analysis, Panel 12, Round 3 was only available for the first year of data collection. Round 3 overlaps two calendar years.
Specifically we explored replicated imputation as a methodology for estimating the effect of imputation on variances of expenditure estimates associated with one particular medical event.
This presentation is a followup to last year’s initial study.

26. Future research Refine the models Incorporate interactions
Determine best categories for various variables
Incorporate interviewer characteristics in the models (demographics, experience level, etc.)
Include NHIS para data
Conduct more in-depth analysis of how Round 1 interview time relates to reduced event reporting and potential impact on expenditure estimates.
Continued evaluations of data quality for key MEPS analytical variables.
Specifically we explored replicated imputation as a methodology for estimating the effect of imputation on variances of expenditure estimates associated with one particular medical event.
This presentation is a followup to last year’s initial study.

27. Conclusion Thanks Questions/comments
Ideas/suggestions for future research
Specifically we explored replicated imputation as a methodology for estimating the effect of imputation on variances of expenditure estimates associated with one particular medical event.
This presentation is a followup to last year’s initial study.