Developing Survey Handbooks as Educational Tools for Data Users Presented at the European Conference on Quality in Official Statistics May 2010 Deborah Griffin U.S. Census Bureau

2 What I’ll Cover Background on the American Community Survey Producing and Disseminating the Handbooks Innovative Approaches Used Quantitative and Qualitative Assessments Challenges and Lessons Learned

3 The American Community Survey Since 1790, decennial censuses in the United States have counted the population to support apportionment The decennial census also served as the data collection vehicle for detailed demographic, social, economic, and housing data

4 The American Community Survey The 2010 Census is only collecting critical data required of the full population The detailed demographic, social, economic, and housing data are now collected and disseminated using the American Community Survey

5 The American Community Survey While ACS content is very similar to the census, the design and methodology differ ACS data are collected continuously throughout the year and estimates are released once-a-year in the form of 1-year, 3-year, and 5-year aggregates

6 The American Community Survey The ACS has a diverse set of data users including federal, state, and local governments, academia, media, and the private sector Uses range from a single estimate for a grant application or homework assignment to complex research using microdata files

7 Census Bureau Strategy The diversity of ACS data users and user applications lead to the decision to produce several different types of educational materials, including a series of ACS handbooks – each written for a different audience including audience-specific issues and case studies

8 Handbook Audiences General Data Users Federal Agencies Business Community Researchers Congress High School Teachers Public Use Microdata Sample Users State and Local Governments

9 Handbook Audiences Media Users of Data for Rural Areas Users of Data about American Indians and Alaska Natives Puerto Rico Community Survey Data Users

10 Producing the Handbooks Experienced data users from each audience were identified to develop draft handbooks Required topics, reference materials, and several core tables were provided A common set of technical appendices and a glossary were developed

11 Generic Table

12 Technical Appendices

13 Technical Appendices

14 Innovative Approaches We encouraged each content developer to use a style and case studies that would work best with their audience Challenge was to maintain common look-and- feel and consistent messaging across all approaches

15 Media

16 State and Local Governments

17 General Users

18 Researchers

19 High School Teachers

20 Rural Area Data Users

21 Producing the Handbooks Each handbook was peer reviewed by a set of individuals who were a part of the target audience Census Bureau staff conducted a detailed review of all handbooks to ensure accuracy and consistency of messaging Final stage of graphics design and review

22 Disseminating the Handbooks Handbooks were released online beginning in October 2008 Webpage created for users to access and download handbooks Printed copies also produced and disseminated

23 Quantitative Assessment AudiencePDF views each month General Data Users269 Congress129 High School Teachers583 Puerto Rico Community Survey Users58 Business Community111 Public Use Microdata Sample Users150 Researchers327 Users of Data for Rural Areas185 Users of Data for American Indians & Alaska Natives109 Federal Agencies100 Media260 State and Local Governments157

24 Quantitative Assessment AudienceHard copies distributed General Data Users1,200 Congress400 High School Teachers400 Business Community1,000 Public Use Microdata Sample Users500 Researchers800 Users of Data for Rural Areas700 Users of Data for American Indians & Alaska Natives800 Federal Agencies400 Media500 State and Local Governments1,500 TOTAL8,200

25 Lessons Learned Striking a balance between technical accuracy and simple language that will be understood by non-technical audiences Gaining agreement within Census on detailed guidance Gaining Census Bureau acceptance of a non- traditional approach to user materials

26 Lessons Learned Use of external content developers was a huge benefit to producing user-friendly handbooks Use of peer reviewers was an important way to gauge the ability of each handbook to reach key audiences

27 Conclusion Production required a great investment of time and resources Audience-specific was the right decision Dissemination and assessment of utility are a challenge Users want more

Contact Information