1. Can We Trust Data Users to Consider Data Quality? Presented at the 2008 European Conference on Quality in Official Statistics

2. Background The American Community Survey (ACS) is an innovative approach for collecting and publishing demographic, social, economic, and housing data National sample of about 3 million addresses each year

3. Background Combining ACS samples over time permits publication for smallest geographic areas Combining ACS samples over space permits publication for shorter time periods

4. ACS Data Release Schedule Type of Estimate Year of Data Release 200620072008200920102011 5-yearNA 2005- 2009 2006- 2010 3-yearNA 2005- 2007 2006- 2008 2007- 2009 2008- 2010 1-year200520062007200820092010 NA: Not Available

5. Dissemination Options 5-year estimates released for all geographic areas to produce data similar to census sample data 1-year and 3-year estimates released only for a subset of these geographic areas

6. ACS Data Users Technically advanced users have the experience and can usually be trusted to consider quality Novice users who lack this experience may not understand or take quality into account

7. Consequences Release of data that are not perceived as credible leads to loss of trust in the integrity of the survey in general

8. “ ACS strikes again! Its hard to believe that the Census Bureau expects users to accept these numbers.”

9. ACS Dissemination Philosophy Release as many data as possible to as many areas as possible while being certain that confidentiality is retained Produce accompanying information on sampling error and educational materials for users

10. Methods 1-year estimates are only published for geographic areas with a minimum population of 65,000 Products reflect the use of a table-based data release rule and the availability of detailed and collapsed tables

11. Example of a Collapsed Table

12. Example - Margins of Error

13. Example - Confidence Intervals

14. Example - Statistical Testing

15. Other Educational Materials Website includes numerous documents describing survey methods and survey quality Separate ACS web page on Quality Measures

16. Review of 2006 ACS Data Summary of the total estimates produced from the ACS sample Reliability of published estimates Effectiveness of publication thresholds and data release rules

17. Estimate Size – 2006 ACS Number of published (blue) and not published (grey) estimates (in millions)

18. Reliability of 2006 ACS Estimates Number of published estimates (in millions) with CVs of less than 30% (blue) and CVs of 30% or greater (red)

19. Effectiveness of Thresholds and Release Rules – 2006 ACS Number of estimates (in millions) with CVs of 30% or greater with release rules (red) or without release rules (grey) given varying publication thresholds

20. Effect of Threshold Changes on Scope of Publication Number of geographic areas receiving 1-year estimates given varying publication thresholds

21. Conclusions Continued release of 1-year estimates based on 65,000 threshold and use of data release rule Expansion of educational materials for users with emphasis on quality

22. New Initiatives Survey of users to obtain feedback on measures of sampling error Development of on-line calculator Testing of alternative visual display of ACS data

23. New Initiatives Data user guides for targeted audiences On-line tutorial Training materials and train-the-trainer sessions

24. More information Deborah H. Griffin U.S. Census Bureau Deborah.H.Griffin@census.gov (301) 763-2855