Technical Issues Associated with the American Community Survey Lisa Neidert NPC Poverty/American Community Survey Workshop June 22-26, 2009

Statistical Notes  Reference purposes Do not need to use for presentation on Friday Often ignored for overview talks Often ignored, unless required by reviewer  Statistical issues are not ACS-specific Census Bureau is more sensitive to statistical issues in 2008 than they were in 2000, 1990  Confidentiality  Standard errors/Margins of Error in ACS, but not census products  Resources: ass_Appendix.pdf ass_Appendix.pdf ngReplicateWeights.html ngReplicateWeights.html

Technical Appendices  Each Compass guide has the same glossary and set of appendices  douts/Compass_Appendix.pdf douts/Compass_Appendix.pdf

Big Picture  Appendix 3 – Definitions Standard Errors  How to get a SE from MOE Margin of Error  Reported with summary tabulations  How to change from 90% to 95%; 99% Confidence Interval Coefficient of Variation  Recommendations

Coefficient of Variation  CV = Standard Error/Population Estimate Note: Need to recover SE (Appendix 3)  The smaller the CV the more reliable the estimate is  Preferable that CV < 10%  If using a proportion, the closer the proportion is to 0, the more unstable the CV is

Coefficient of Variation...  1-year estimate vs 3-year estimate CV will be 70% larger for the 1-year estimate  Decision is precision vs currency  Use multi-year estimate if single year has CV > 10%  If single year CV is > 50 percent, do not use

Minimum population thresholds  Rule of thumb: Population estimate (weighted) should be:  >3,000 for 1-year  >1,000 for 3-year  >600 for 5-year  [From older draft of Appendix 6]

Multiyear Estimates  When comparing multiyear estimates, use non-overlapping estimates vs  Do not compare a single year estimate against a multiyear estimate Both should be multiyear estimates The single year estimate could be at the peak or trough of an economic cycle vs the more moderated multiyear estimate

Margins of Error for Derived Estimates  Summary data provide a margin of error for every cell  Researchers often combine cells Geography like NYC community districts comprised of multiple census tracts Cells within a geography  Combine several age groups

Spreadsheet  Illustrate New Derived MOE  group/34a72eab-daa4-4d14-80e aed6c/Assignments/Technical% 20Appendix_Appendix_5.xls group/34a72eab-daa4-4d14-80e aed6c/Assignments/Technical% 20Appendix_Appendix_5.xls

Statistical Significance  Appendix 4 ACS vs ACS ACS vs Census

Replicate Weights  To correct for the fact that the samples are not random  Census Bureau’s replicate weights are not handled by statistical packages (as an option, like JK or BRR)  SAS program that has the algorithm oup/34a72eab-daa4-4d14-80e aed6c/Technical%20- %20Statistical/sas_code_example.txt oup/34a72eab-daa4-4d14-80e aed6c/Technical%20- %20Statistical/sas_code_example.txt

Census Bureau recommendations for replicate weights  Garrett, B. Dale and Michael Starsinic “ACS Public Use Microdata Samples of 2005 and 2006 – How to Use Replicate Weights.” Presentation at AAPOR Conference, New Orleans, May 16,  4a72eab-daa4-4d14-80e aed6c/Technical%20- %20Statistical/How%20to%20Use%20PUMS% 20Replicate%20Weights.ppt 4a72eab-daa4-4d14-80e aed6c/Technical%20- %20Statistical/How%20to%20Use%20PUMS% 20Replicate%20Weights.ppt

Theoretical Basis for Successive Difference Model  Fay, R. and Train, G “Aspects of Survey and Model-Based Postcensal Estimation of Income and Poverty Characteristics for States and Counties,” Proceedings of the Section on Governmental Statistics, American Statistical Association, pp