1. The Impact of Survey Design Modifications on Health Care Utilization Estimates in a National Longitudinal Health Care Survey Steven B. Cohen, Ph.D. Trena Ezzati-Rice, M.S. Marc Zodet, M.S.

2. Presentation Need for essential data on health care utilization to inform health care policy and practice Need for essential data on health care utilization to inform health care policy and practice Description of the Medical Expenditure Panel Survey (MEPS): purpose, design and analytical capacity Description of the Medical Expenditure Panel Survey (MEPS): purpose, design and analytical capacity Nonresponse and post-stratification adjustments Nonresponse and post-stratification adjustments Recent survey design modifications: Recent survey design modifications: (1) CAPI upgrade; (2) Sample Redesign Evaluation of impact of design modifications on health care utilization estimates Evaluation of impact of design modifications on health care utilization estimates Impact of design modifications on model-based analyses of health care use Impact of design modifications on model-based analyses of health care use Discussion Discussion

3. Medical Expenditure Panel Survey (MEPS) Annual Survey of 14,000 households: provides national estimates of health care use, expenditures, insurance coverage, sources of payment, access to care and health care quality Permits studies of: Distribution of expenditures and sources of payment Distribution of expenditures and sources of payment Role of demographics, family structure, insurance Role of demographics, family structure, insurance Expenditures for specific conditions Expenditures for specific conditions Trends over time Trends over time

4. Key Features of MEPS-HC Survey of U.S. civilian noninstitutionalized population Survey of U.S. civilian noninstitutionalized population Sub-sample of respondents to the National Health Interview Survey (NHIS) Sub-sample of respondents to the National Health Interview Survey (NHIS) Oversample of minorities and other target groups Oversample of minorities and other target groups Panel Survey – new panel introduced each year Panel Survey – new panel introduced each year – Continuous data collection over 2 ½ year period – 5 in-person interviews (CAPI) – Data from 1st year of new panel combined with data from 2nd year of previous panel

5. MEPS Overlapping Panels (Panels 13 and 14) MEPS Household Component MEPS Panel 13 2008- 2009 Round 2Round 3Round 4Round 5 Round 1Round 2Round 3 MEPS Panel 14 2009-2010 1/1/20081/1/2009 Round 1 NHIS2007 NHIS2008 Round 4Round 5

6. HC - Purpose Estimates annual health care use and expenditures Estimates annual health care use and expenditures Provides distributional estimates Provides distributional estimates Supports person and family level analysis Supports person and family level analysis Tracks changes in insurance coverage and employment Tracks changes in insurance coverage and employment Longitudinal design; linkage to National Health Interview Survey (NHIS) Longitudinal design; linkage to National Health Interview Survey (NHIS)

7. Tool Chest of Methods to Maximize Survey Response Recruitment of experienced interviews and bilingual Recruitment of experienced interviews and bilingual 10+ days training (including procedures for obtaining signed consents) 10+ days training (including procedures for obtaining signed consents) Uses of MEPS data as reference materials for interviewers Uses of MEPS data as reference materials for interviewers Periodic retraining and special trainings (e.g. methods to improve response rates) Periodic retraining and special trainings (e.g. methods to improve response rates) Respondent remuneration Respondent remuneration Advance mailings from co-sponsors of survey Advance mailings from co-sponsors of survey Monthly planning calendar and MEPS DVD Monthly planning calendar and MEPS DVD Daily emails to interviewers regarding interviewing progress Daily emails to interviewers regarding interviewing progress Multiple contacts for refusal conversions Multiple contacts for refusal conversions

8. MEPS Response Rates Multiplicative response rates (RR): product of Multiplicative response rates (RR): product of NHIS RR and NHIS RR and MEPS RR (multiplicative function of round specific RR): MEPS RR (multiplicative function of round specific RR): MEPS rounds 1-3 of panel (YR1 estimates) MEPS rounds 1-3 of new panel (YR1 estimates) MEPS rounds 3-5 of panel (YR2 estimates) MEPS rounds 3-5 of old panel (YR2 estimates)

9. MEPS Response Rates (RR) Overall annual RR (~65%) Overall annual RR (~65%) Highest RR 1 st year, new panel (~66-71%) Highest RR 1 st year, new panel (~66-71%) Lowest RR 2 nd year, old panel (~63-65%) Lowest RR 2 nd year, old panel (~63-65%) Post-survey nonresponse adjustments Post-survey nonresponse adjustments – Dwelling unit level – Person level survey attrition

10. NHIS variables used as potential covariates in forming DU NR adjustment cells Demographic Household Characteristic Socio- Economic Status GeographicHealth Age DU size Poverty status Census region Health status Race/ethnicity Has phone Education MSA size Need help Marital status Working/reason not work (e.g., attending school, retired, etc,) IncomeMSA/nonMSA Gender Type of PSU Employment status Urban/Rural Any Asian Any Black

11. NEW NHIS variables added as potential covariates in forming DU NR adjustment cells Demographic Household Characteristic Socio- Economic Status Health Interview language Type of home – house, Apt., etc. Category of medical expense Number of nights in hospital U.S. Citizenship Time no phone Home ownership Healthcare coverage Born in US

12. Adjustment factor Within each adjustment cell: Within each adjustment cell: A(c) = ratio of the sum of weights of all eligible (E) units in the cell to the sum of weights of only the respondents (R) in the cell

13. Person Level Adjustments: Annual Estimates Each panel weighted separately Each panel weighted separately Nonresponse adjustment for survey attrition Nonresponse adjustment for survey attrition Final Poststratification adjustment – CPS 12/31: Final Poststratification adjustment – CPS 12/31: age, race/ethnicity, sex, region, MSA status, poverty status age, race/ethnicity, sex, region, MSA status, poverty status

14. Person Level (survey attrition) Nonresponse Adjustment Covariates Factors associated with survey attrition (after R1) Factors associated with survey attrition (after R1) – Indicator for initial refusal to R1interview – Family size – Age – MSA, census region – Marital status (family reference person) – Race/ethnicity – Education of reference person – Employment status – Health insurance status – Total expenditures (in yr 1 for yr 2 adj.) – # doctor visits (in yr 1) – Self reported health status

15. Round 1Round 2Round 3Round 4Round 5 2009 2010 Longitudinal Estimation Strategy Individuals in the 2009 sample with positive weights that left the civilian population prior to 2010, with no return 2009 sample also responding in 2010 with complete information for both 2009 and 2010 &

16. MEPS Redesign in 2007 Re-engineered CAPI Interview: Windows-based Platform replaces DOS-based system for Panel 12 Re-engineered CAPI Interview: Windows-based Platform replaces DOS-based system for Panel 12 New NHIS Sample Design Introduced in 2006: MEPS Panel 12 selected from redesigned NHIS sample New NHIS Sample Design Introduced in 2006: MEPS Panel 12 selected from redesigned NHIS sample Year 2 of MEPS Panel 11 based on original MEPS survey design Year 2 of MEPS Panel 11 based on original MEPS survey design The overlapping panel structure in MEPS allows for a comparison of survey estimates across the alternative designed for the same time period The overlapping panel structure in MEPS allows for a comparison of survey estimates across the alternative designed for the same time period

17. Evaluation of Concordance of Healthcare Utilization Estimates: Comparison of results from new and original designs MEPS has overlapping panel design: 1st year of new panel combined with data from 2nd year of previous years panel to yield annual data MEPS has overlapping panel design: 1st year of new panel combined with data from 2nd year of previous years panel to yield annual data Multiplicative response rates: product of NHIS RR and MEPS RR (multiplicative function of round specific RR: 3 rounds for new panel/5 rounds for old panel) Multiplicative response rates: product of NHIS RR and MEPS RR (multiplicative function of round specific RR: 3 rounds for new panel/5 rounds for old panel) Detailed adjustments for survey nonresponse and poststratification: Detailed adjustments for survey nonresponse and poststratification: Compare 2007 health care utilization estimates based on new design (MEPS Panel 12 – Year 1) with original design (MEPS Panel 11-Year 2) Compare 2007 health care utilization estimates based on new design (MEPS Panel 12 – Year 1) with original design (MEPS Panel 11-Year 2)

18. Testing for Survey Redesign Effects Comparisons of panel specific national Comparisons of panel specific national health care utilization estimates derived from the MEPS for the following health care services: ambulatory visits (office- based visits and outpatient facility visits) in-patient stays ER visits dental visits prescribed medicine purchases For the overall population, and further subset by age classification (0-17, 18-64, 65+) Model-based tests for survey redesign effects Model-based tests for survey redesign effects

23. Capacity of MEPS to Produce Comparable NHIS Estimates of Health Care Utilization The following NHIS measures of health care utilization were selected in support of these analyses: Have you been hospitalized OVERNIGHT in the past 12 months? (yes; no; refused/not ascertained/DK) Have you been hospitalized OVERNIGHT in the past 12 months? (yes; no; refused/not ascertained/DK) How many different times did you stay in any hospital overnight or longer DURING THE PAST 12 MONTHS? (#; refused/not ascertained/DK) How many different times did you stay in any hospital overnight or longer DURING THE PAST 12 MONTHS? (#; refused/not ascertained/DK) Altogether how many nights were you in the hospital DURING THE PAST 12 MONTHS? (#; refused/not ascertained/DK) Altogether how many nights were you in the hospital DURING THE PAST 12 MONTHS? (#; refused/not ascertained/DK) During the past 12 MONTHS did you receive care from doctors or other health care professionals 10 or more times? Do not include telephone calls. (yes; no; refused/not ascertained/DK) During the past 12 MONTHS did you receive care from doctors or other health care professionals 10 or more times? Do not include telephone calls. (yes; no; refused/not ascertained/DK) DURING THE PAST 12 MONTHS, have you delayed seeking medical care because of worry about the cost? (yes; no; refused/not ascertained/DK) DURING THE PAST 12 MONTHS, have you delayed seeking medical care because of worry about the cost? (yes; no; refused/not ascertained/DK) DURING THE PAST 12 MONTHS, was there any time when you needed medical care, but did not get it because you/the family couldn't afford it? (yes; no; refused/not ascertained/DK) DURING THE PAST 12 MONTHS, was there any time when you needed medical care, but did not get it because you/the family couldn't afford it? (yes; no; refused/not ascertained/DK)

30. Options for aligning redesign-based estimates with the original design Option Period of Applicability Constraints Restrict time trend analyses to sample with old design Year(s) with overlap between redesign and prior design Loss in precision Implementation of measure specific adjustments to a set of estimation weights When the survey redesign is implemented and subsequent years Introduction of greater variability in resultant survey estimates ; complicates model based analyses Implementation of bridging adjustments to the primary survey estimation weight When the survey redesign is implemented and subsequent years Dependence on within survey adjustments or availability of comparable external data source for national control totals No additional adjustments for redesign When the survey redesign is implemented and subsequent years Possibility of differences detected in trends partially attributable to redesign

31. Summary Need for accurate and reliable national data on health care utilization to inform policy and practice Need for accurate and reliable national data on health care utilization to inform policy and practice MEPS design features and analytical capacity MEPS design features and analytical capacity Statistical, methodological and operational design features to adjust for nonresponse and attrition Statistical, methodological and operational design features to adjust for nonresponse and attrition Evaluation of impact MEPS redesign on health care utilization estimates Evaluation of impact MEPS redesign on health care utilization estimates Impact on model based studies Impact on model based studies Some evidence of redesign effect Some evidence of redesign effect

32. Strategies to Improve Accuracy MEPS includes a linked survey of medical providers for expenditures: use of medical event information to evaluate household reports of health care use MEPS includes a linked survey of medical providers for expenditures: use of medical event information to evaluate household reports of health care use MEPS data periodically linked to Medicare claims data for evaluations: permits examination of accuracy of household reported data MEPS data periodically linked to Medicare claims data for evaluations: permits examination of accuracy of household reported data Implement additional improvements to the CAPI interview and enhanced post-survey adjustment strategies Implement additional improvements to the CAPI interview and enhanced post-survey adjustment strategies