1. Frances M. Chevarley David W. Keer
How the Way the Construct of Disability is Measured Affects the Estimates Obtained in Selected National Surveys
Frances M. Chevarley David W. Keer
2016 QDET2 Session CS35 November 12, 11:00 AM to 12:25 PM
The views expressed in this presentation are those of the authors and no official endorsement by the Department of Health and Human Services or the Agency for Healthcare Research and Quality is intended or should be inferred.

2. How to measure disability?
Two approaches
Disability Measure, AHRQ’s National Quality/Disparity Reports — NQDR reports
Disability Questions, Census Bureau’s 2008 American Community Survey—2008+ ACS
Separate but overlapping workgroups for each
Different goals
Increase disability data in NQDR using similar groups
Better disability measure for ACS
We will briefly discuss two different approaches associated with disability measures.
We first describe the process used to develop a disability measure for use in AHRQ’s National Quality/Disparity Reports
The second is the Census Bureau’s process to come up with the 2008 Amercian Community Survey Disability questions.
Both used separate but overlapping workgroups for each
Different goals for each…

3. AHRQ’s National Quality/Disparity Reports Disability Measure
Background of the National Healthcare Quality and Disparities Reports (NQDR)
Priority populations including persons with Special Health Care Needs
NQDR disability working group (NQDR DWG)
Measurement of disability
Implemented with the 2007 NQDR reports

4. Interagency Disability Working Group of the NQDR IWG
AHRQ convenes group with help from:
Interagency Subcommittee of Disability Statistics
Representatives from over 16 agencies
Including HHS, VA, Census, Social Security Administration, NCD, and Department of Education
Goal to recommend similar measurements of individuals with disabilities for NQDR
Applicable across multiple data sources
MEPS, NHIS, MCBS
Can be described to lay persons in a half page
Allows comparisons by race, ethnicity, and income
Broad and inclusive-sensitive to sample size limitations

5. Interagency Disability Working Group of the NQDR IWG
Representatives from:
Agency for Healthcare Research and Quality (AHRQ)
Administration on Aging (AoA)
Office of The Assistant Secretary for Planning and Evaluation (ASPE)
National Center for Health Statistics (CDC-NCHS)
National Center on Birth Defects and Developmental Disabilities (CDC-NCBDDD)
Centers for Medicare & Medicaid Services (CMS)
DHHS Office on Disability
National Institute on Aging (NIH-NIA)
National Center for Medical Rehabilitation Research (NIH-NCMRR)
Health Resources and Services Administration (HRSA)
Substance Abuse and Mental Health Services Administration (SAMHSA)
National Council on Disability (NCD)
National Institute on Disability and Rehabilitation Research (DoE-NIDRR)
Social Security Administration
U.S. Census Bureau
U.S. Department of Veterans Affairs (VA)

6. Objective of Disability Working Group
Reflect the ADA definition of Disability
Locate the areas of consistency in questions across data sets used in the NQDR
MEPS
NHIS
MCBS
Used the International Classification of Functioning, Disability, and Health (ICF) as a guide to the conceptual components of disability
Compare questions/estimates to ICF Activities and Participation Component’s Chapters/Domains
Determine domains/estimates to include in disability measures

7. In comparing disability questions across surveys note:
Disability is a continuum and not a dichotomous concept
Disability manifests itself across multiple domains
Disability can be short-lived or can be chronic or sporadic

8. In comparing disability questions across surveys note (Cont.):
Surveys have different purposes
capture different # of disability questions
capture different domains and at different points on the severity continuum
Survey tools to measure disability are not as precise as measurement tools for physical sciences

9. NHIS, MEPS, SIPP NHANES, MCBS all capture domains of:
Compare Questions/Estimates to ICF Activities and Participation Component Chapters/Domains
NHIS, MEPS, SIPP NHANES, MCBS all capture domains of:
Learning and applying knowledge (C1)
Purposeful sensory experience
Applying knowledge
Mobility (C4)
Self care (C5)
Domestic life (C6)
Major life areas (C8)
Community, social and civic life (C9)
Ref.— crosswalk lead was Cille Kennedy, ASPE

10. Comparison of Mobility Domain Questions
2003 MEPS
Family Screener
Severity for individuals that screen in for 8 items – 4 categories
no difficulty, some difficulty, a lot of difficulty, or completely unable to do it
2003 NHIS
Severity asked for each of 9 items – 6 categories
not at all difficult, only a little difficult, somewhat difficult, very difficult, can’t do at all, unable to do it
Broader range of categories in NHIS changes how people answer ‘somewhat difficult’ in NHIS vs. ‘some difficulty’ in MEPS
To make comparisons
Limit NHIS to 8 items most similar to items in MEPS
Use severity in NHIS to match severity in MEPS

11. NQDR Disability Working Group recommendation
Use two measures of disability
Basic Activity Limitations (BAL)
Mobility, self care (ADLs), domestic life (IADLs), watching and listening activities
Complex Activity Limitations (CAL)
Work activities, and social and civic life activities
BAL and CAL are not mutually exclusive
Neither BAL or CAL used for comparison purposes
Implemented in NQDR starting in 2007

12. Census Bureau’s 2008 American Community Survey Disability Measure
Background Census and ACS Disability Questions
ACS Subcommittee on Disability Measurement
Measurement of disability
Implemented with the 2008 ACS
Ref.— Brault, 2009; Altman, 2006; Brault, Stern, and Raglin, 2007

13. Background—2000 Census and 2003-2007 American Community Survey Disability Questions
6 Concepts, 3 Questions
Question 1-- ages 5+
Sensory disability—“blindness, deafness or a severe vision or hearing impairment”
Physical disability—“condition that limits one or more basic physical acitivities…”
Question 2—ages 5+, physical mental, or emotional condition lasting 6 months or more, any difficulty:
Mental disability—“Learning, remembering, or concentrating”
Self-care disability—“Dressing, bathing, or getting around inside the home”
Question 3—ages 15+, physical mental, or emotional condition lasting 6 months or more, any difficulty
Go-outside-home disability—“shop or visit a doctor’s office”
Employment disability—“working at a job or business”

14. ACS Subcommittee on Disability Measurement
Convened by the OMB Interagency Committee for the ACS in 2002
NCHS to take the lead, all agencies invited
Representatives from NCHS, U.S. Census Bureau, AHRQ, ASPE, AoA, BLS, ED, ED- NIDRR, FCC, NSF, NCD, NIMH, OMB, SSA, VA, HUD, IHS
Objective: To assess the adequacy of the current ACS (former 2000 Census) disability questions

15. ACS Measurement of Disability
Constraints on number of questions & legislature mandates
Disability involves social factors—both internal and external -- Difficult to measure in surveys
Focused on population likely to experience restrictions in participation due to physical, social and other environmental barriers—Easier to measure in surveys
Goal to identify broad representation of population at risk of restrictions in participation at the person level
Less influenced by social and structural contexts which can vary across groups
Want to identify younger as well as older people

16. ACS Measurement of Disability
Came up with initial questions
Cognitively tested initial questions
Revised questions
Part of the 2006 ACS Content Test
Revised questions performed equal or better than ACS disability questions
nonresponse rates
Measures of item by item and overall disability recode reliability

17. ACS Subcommittee on Disability Measurement recommendation
2008 ACS 6 disability questions
Measured by 4 domains of functioning
Seeing—ages 1+
Hearing—ages 1+
Cognitive functioning –ages 5+
Ambulation—ages 5+
Identify persons with problems of independence by 2 areas of self-care
Difficulty bathing and dressing—ages 5+
Ability to get around outside the home without assistance—ages 15+

18. Compare 2014 MEPS and NHIS estimates using these two approaches
NQDR disability measure
2014 MEPS and 2014 NHIS estimates for ages 18+
2008 ACS disability questions

19. Percent with a disability: 2014, ages 18+ NQDR Disability Measures
* Difference significant at .05 level

20. Percent with a disability: 2014, Ages 18+ 2008 ACS Disability Questions
* Difference significant at .05 level

21. How did we do with our two approaches for disability measures (Cont.)?
Goal of both approaches to produce outcomes for similar groups of disabled persons across surveys
In general MEPS and NHIS produce similar estimates for both approaches, though there are statistically significant differences along some domains
Which approach is better?

22. Strengths and limitations of two disability measures
NQDR disability measures
Limited to ages 18+
Within a domain had to choose severity that was most similar in the surveys (not necessarily the one we would want to choose)
2008 ACS disability questions
Estimates for broad range of ages
Used in a number of surveys so outcomes for disabled can be compared across many more surveys
Neither approach has a severity level within the domains they consider
Will there be a standard set of questions for more detailed disability information?

23. Thank you. Questions. MEPS Website: http://MEPS. AHRQ
Thank you! Questions? MEPS Website: Contact: Frances M. Chevarley David W. Keer