Can We Trust Data Users to Consider Data Quality? Presented at the 2008 European Conference on Quality in Official Statistics
Background The American Community Survey (ACS) is an innovative approach for collecting and publishing demographic, social, economic, and housing data National sample of about 3 million addresses each year
Background Combining ACS samples over time permits publication for smallest geographic areas Combining ACS samples over space permits publication for shorter time periods
ACS Data Release Schedule Type of Estimate Year of Data Release yearNA yearNA year NA: Not Available
Dissemination Options 5-year estimates released for all geographic areas to produce data similar to census sample data 1-year and 3-year estimates released only for a subset of these geographic areas
ACS Data Users Technically advanced users have the experience and can usually be trusted to consider quality Novice users who lack this experience may not understand or take quality into account
Consequences Release of data that are not perceived as credible leads to loss of trust in the integrity of the survey in general
“ ACS strikes again! Its hard to believe that the Census Bureau expects users to accept these numbers.”
ACS Dissemination Philosophy Release as many data as possible to as many areas as possible while being certain that confidentiality is retained Produce accompanying information on sampling error and educational materials for users
Methods 1-year estimates are only published for geographic areas with a minimum population of 65,000 Products reflect the use of a table-based data release rule and the availability of detailed and collapsed tables
Example of a Collapsed Table
Example - Margins of Error
Example - Confidence Intervals
Example - Statistical Testing
Other Educational Materials Website includes numerous documents describing survey methods and survey quality Separate ACS web page on Quality Measures
Review of 2006 ACS Data Summary of the total estimates produced from the ACS sample Reliability of published estimates Effectiveness of publication thresholds and data release rules
Estimate Size – 2006 ACS Number of published (blue) and not published (grey) estimates (in millions)
Reliability of 2006 ACS Estimates Number of published estimates (in millions) with CVs of less than 30% (blue) and CVs of 30% or greater (red)
Effectiveness of Thresholds and Release Rules – 2006 ACS Number of estimates (in millions) with CVs of 30% or greater with release rules (red) or without release rules (grey) given varying publication thresholds
Effect of Threshold Changes on Scope of Publication Number of geographic areas receiving 1-year estimates given varying publication thresholds
Conclusions Continued release of 1-year estimates based on 65,000 threshold and use of data release rule Expansion of educational materials for users with emphasis on quality
New Initiatives Survey of users to obtain feedback on measures of sampling error Development of on-line calculator Testing of alternative visual display of ACS data
New Initiatives Data user guides for targeted audiences On-line tutorial Training materials and train-the-trainer sessions
More information Deborah H. Griffin U.S. Census Bureau (301)