Disability Data in the American Community Survey Presentation for the 2006 State-of-the-Science Conference - The Future of Disability Statistics: What We Know and Need to Know Sharon M. Stern Poverty and Health Statistics Branch U.S. Census Bureau October 5, 2006

2 Overview What is the American Community Survey (ACS)? ACS Operations Current disability items 2006 ACS Content Test –How content for test was developed –Review of test operations –Selection criteria Update on SIPP and DEWS

3 What is the American Community Survey (ACS)? The ACS is a nationwide survey designed to provide communities a fresh look at how they are changing. The ACS provides a statistical snapshot of the community

4 How is ACS household data collected? Surveys are mailed every month to a systematic sample of addresses in each county If a household does not respond in six weeks, Census Bureau staff will attempt to contact the respondent by telephone to complete the survey. If that, too, fails, a differential sample of remaining addresses will be visited by Census Bureau staff for an in person interview.

5 Response Rates By Mode

6 Group Quarters in ACS Implemented in 2006 Questionnaires, introductory letter, FAQ’s … Technical Paper – housing units and group quarters…

7 ACS – Paper Form – Disability Items Bottom of page 7 Top of page 8

8 ACS Interagency Committee Subcommittee for the Disability Questions Under the auspices of Office of Management and Budget Chaired by the National Center for Health Statistics Prepared recommendations for the ACS 2006 Content Test –Reviewed the legislative need for the data –Determined the main purposes of the data –Focused on meeting the needs given constraints –Conducted cognitive testing on questions

9 Federal Agencies Reviewed for Statutory and/or Programmatic Requirements for Disability Data Examples: Department of Commerce – Telecommunications Act of 1996 Department of Education – National Education Reform Administration on Aging – Older Americans Act Department of Housing and Urban Development – National Affordable Housing Act Department of Transportation – Mass transportation Elderly and Persons with Disabilities Program

10 Federal Agencies Reviewed for Statutory and/or Programmatic Requirements for Disability Data - Findings Examples: Agencies focus on individuals with limitations in functioning who are more likely to experience a limitation in participation as well. Agencies use the data for two purposes. –Distribution of benefits, such as SSDI Insurance or Veteran health benefits –Provision of opportunities such as access to education, housing, and communication.

11 Definition of Disability Adapted by the Subcommittee Based on orientation of agency mandates and using IOM Model of Disability and ICF Model of Functioning and Disability (WHO) Disability - the restriction in participation that results from a lack of fit between the individual’s functional limitations and the characteristics of the physical and social environment. –Measuring disability then means analyzing the component concepts that make up the process.

12 Purpose of Disability Measure Recognized by the Subcommittee Equalization of opportunity - identify those who, without accommodation, are likely to experience restrictions in participation due to limitations in functioning Identify the population needing assistance to maintain independence

13 Basis of questions chosen for cognitive testing Domains of functioning that identify the largest component of the population with disabilities –Vision –Hearing –Mobility (walking, climbing stairs) –Cognitive functioning Monitoring independent living –Self-care activities (bathing, dressing) –Ability to move about the community without help Limitation in kind or amount of work –Included for testing based on its previous use in the Census and assumptions of its ability to capture persons with mental health related limitations

14 Cognitive Testing “Report of Cognitive Research on Proposed American Community Survey Disability Questions” by Kristen Miller of the National Center for Health Statistics and Theresa DeMaio of U.S. Census Bureau –Five rounds of testing between June 2004 and February 2005 –Testing at NCHS face-to-face and telephone interviews –Testing at Census paper questionnaires –Several wordings for each domain

15 Cognitive Testing – Goals Identify respondent interpretations Identify potential response errors Improve test questions Investigate question performance within the context of three ACS modes: self- administered paper, telephone interview, in- person interview

16 Cognitive Testing – Recurring Themes Regardless of the type of question, respondents have an internal calculation of whether a condition or limitation is “severe enough” to report Some people report limitation status with assistance other report status without assistance Mode tended not to impact respondent’s interpretation

17 Recommendations 16 a. Is this person deaf or does he/she have serious difficulty hearing? b. Is this person blind or does he/she have serious difficulty seeing even when wearing glasses? F) Answer question 17a if this person is 5 years old or over. Otherwise skip to the questions for Person 2 on page a. Because of a physical, mental, or emotional condition, does this person have serious difficulty concentrating, remembering, or making decisions? b. Does this person have serious difficulty walking or climbing stairs? c. Does this person have difficulty dressing or bathing? G) Answer questions 18 if this person is 15 years old or over. Otherwise skip to the questions for Person 2 on page Because of a physical, mental, or emotional condition, does this person have difficulty doing errands alone such as visiting a doctor's office or shopping?

ACS Content Test A variety of objectives for testing –Content: e.g.. disability and health insurance –Form layout: e.g.. horizontal v. vertical roster Two versions of disability items –Control: ACS current production questions –Test: Recommendation from interagency work group Research Questions –Does the new set have more consistent reporting? –Does the new set have improved item response? –Where do these people’s disabilities fall on a spectrum of difficulty with specific activities?

19 How was the ACS Content Test data collected? Surveys were mailed to a sample of addresses. If no mail response was received, Census Bureau staff will attempt to contact the address in person to complete the survey. All respondents were part of a content follow-up conducted by telephone.

20 Summary of Disability Topics 17bMobility Disability16bPhysical Disability 17aCognitive Disability17aMental Disability 17cSelf-care Disability17bSelf-care Disability 18 Independent-living Disability 18a Go-outside-home Disability b Employment Disability 16bSeeing Disability 16aHearing Disability 16aSensory Disability Item #LabelItem #Label Test (Interagency Recommendation) Control (ACS Current Production)

21 Summary of How “Test” Differs from “Control”– Part 1 The hearing and vision concepts are separate questions in the test version The hearing and vision questions on the test version include children under 5 years old, since the skip instruction was after these items. The vision question includes the qualifier “even when wearing glasses.” The hearing, vision, and mobility questions are simpler, omitting key terms/phrases of “long-lasting condition” and “impairment.”

22 Summary of How “Test” Differs from “Control”– Part 2 The phrase “Does this person have [serious] difficulty” is included in the test version before each function or activity. The test does not use terms like “substantially limits” or “long-lasting” or “lasting 6 months or more” The test set does not include a work disability item. Please see the information sheet for more details.

23 Selection Criteria Is the reliability for the test version equal to or better than the control? –For example, is the reliability for the vision and hearing questions in the test version equal to or better than the vision and hearing question in the control version? –Adjusted Simple Response Variance Are the item nonresponse rates for the test version less than or equal to that of the control?

24 Survey of Income and Program Participation – Dynamics of Economic Well Being System SIPP General Information: Topical Modules –Functional Limitations and Disability, June to September 2005 –Panels od/top_mods_chart.html DEWS dews.html Reengineering the SIPP - PDF presentation by David Johnson, Chief, HHES at the U.S. Census Bureau Reengineering the SIPP: Brookings/Census Roundtable (MS Word document) August 24, 2006 Stakeholder Presentation Presentation Stakeholder matrix

25 Contacts Sharon M. Stern Matthew Brault