Statistical Significance and Population Controls Presented to the New Jersey SDC Annual Network Meeting June 6, 2007 Tony Tersine, U.S. Census Bureau.

Slides:



Advertisements
Similar presentations
1 The Foreign-Born Population in the United States: 2003.
Advertisements

Chapter 1 The Study of Body Function Image PowerPoint
1 Copyright © 2013 Elsevier Inc. All rights reserved. Appendix 01.
STATISTICS INTERVAL ESTIMATION Professor Ke-Sheng Cheng Department of Bioenvironmental Systems Engineering National Taiwan University.
BUS 220: ELEMENTARY STATISTICS
1 ACS Statistical Issues and Challenges: One-, Three-, and Five-year Period Estimates Michael Beaghen U.S. Census Bureau New Jersey State Data Center Annual.
Jeopardy Q 1 Q 6 Q 11 Q 16 Q 21 Q 2 Q 7 Q 12 Q 17 Q 22 Q 3 Q 8 Q 13
Jeopardy Q 1 Q 6 Q 11 Q 16 Q 21 Q 2 Q 7 Q 12 Q 17 Q 22 Q 3 Q 8 Q 13
Title Subtitle.
Measurements and Their Uncertainty 3.1
Copyright © 2010 Pearson Education, Inc. Slide
FACTORING ax2 + bx + c Think “unfoil” Work down, Show all steps.
Year 6 mental test 10 second questions
Lecture 2 ANALYSIS OF VARIANCE: AN INTRODUCTION
1 Contact details Colin Gray Room S16 (occasionally) address: Telephone: (27) 2233 Dont hesitate to get in touch.
7.1 confidence Intervals for the Mean When SD is Known
Chapter 7 Sampling and Sampling Distributions
Understanding Multiyear Estimates from the American Community Survey Updated February 2013.
1 Understanding Multiyear Estimates from the American Community Survey.
Solve Multi-step Equations
REVIEW: Arthropod ID. 1. Name the subphylum. 2. Name the subphylum. 3. Name the order.
Biostatistics Unit 5 Samples Needs to be completed. 12/24/13.
County-level Estimates of Leisure Time Physical Inactivity among Adults aged 20 years old Trends
Estimating Proportions with Confidence
6. Statistical Inference: Example: Anorexia study Weight measured before and after period of treatment y i = weight at end – weight at beginning For n=17.
PP Test Review Sections 6-1 to 6-6
American Community Survey Data Products Updated February 2013.
5-1 Chapter 5 Theory & Problems of Probability & Statistics Murray R. Spiegel Sampling Theory.
1 Slides revised The overwhelming majority of samples of n from a population of N can stand-in for the population.
VOORBLAD.
Hypothesis Tests: Two Independent Samples
Chapter 10 Estimating Means and Proportions
Comparing Two Groups’ Means or Proportions: Independent Samples t-tests.
Copyright © 2012, Elsevier Inc. All rights Reserved. 1 Chapter 7 Modeling Structure with Blocks.
1 Surveys in Humanitarian Emergencies. 2 Methods of Data Collection AssessmentSurveySurveillance Objective Rapid appraisal Medium-term appraisal Continuous.
Measurements and Their Uncertainty 3.1
Factor P 16 8(8-5ab) 4(d² + 4) 3rs(2r – s) 15cd(1 + 2cd) 8(4a² + 3b²)
Basel-ICU-Journal Challenge18/20/ Basel-ICU-Journal Challenge8/20/2014.
Hours Listening To Music In A Week! David Burgueño, Nestor Garcia, Rodrigo Martinez.
1..
© 2012 National Heart Foundation of Australia. Slide 2.
Lets play bingo!!. Calculate: MEAN Calculate: MEDIAN
Statistical Analysis SC504/HS927 Spring Term 2008
Understanding Generalist Practice, 5e, Kirst-Ashman/Hull
DB analyzer utility An overview 1. DB Analyzer An application used to track discrepancies and other reports in Sanchay Post Constantly updated by SDC.
25 seconds left…...
Equal or Not. Equal or Not
Putting Statistics to Work
Januar MDMDFSSMDMDFSSS
Statistical Inferences Based on Two Samples
©Brooks/Cole, 2001 Chapter 12 Derived Types-- Enumerated, Structure and Union.
Chapter Thirteen The One-Way Analysis of Variance.
Chapter 8 Estimation Understandable Statistics Ninth Edition
1 Accessing Census Bureau Population Estimates for the United States Ryan Burson Population Division U.S. Census Bureau July 16, 2014.
Intracellular Compartments and Transport
PSSA Preparation.
Chapter 11: The t Test for Two Related Samples
Experimental Design and Analysis of Variance
Essential Cell Biology
Multiple Regression and Model Building
January Structure of the book Section 1 (Ch 1 – 10) Basic concepts and techniques Section 2 (Ch 11 – 15): Inference for quantitative outcomes Section.
Slide Slide 1 Copyright © 2007 Pearson Education, Inc Publishing as Pearson Addison-Wesley. Section 7-2 Estimating a Population Proportion Created by Erin.
1 Case Study 2: Choosing between ACS 1-, 3-, and 5-Year Estimates 2009 Population Association of America ACS Workshop April 29, 2009.
Calculating Statistical Significance and Margins of Error Using American Community Survey Data Montgomery County Census Workshop Mark Goldstein Maryland.
1 Things That May Affect Estimates from the American Community Survey.
Things that May Affect the Estimates from the American Community Survey Updated February 2013.
Presentation transcript:

Statistical Significance and Population Controls Presented to the New Jersey SDC Annual Network Meeting June 6, 2007 Tony Tersine, U.S. Census Bureau

2 Overview of the Session Basic Concepts Margin of Error Confidence Intervals Standard Error Formulas Statistical Testing Population Controls

3 Basic Concepts - 1 Sampling error is introduced due to sampling, selection of a subset of the population to draw inferences about the entire population. Standard error is an estimate of the precision of the estimates. It measures the variability of an estimate due to sampling.

4 Basic Concepts - 2 The sampling error is often reported as the estimate plus or minus the margin of error, a measure of how precise the estimate is. The margin of error describes the precision of the estimate at a given confidence level.

5 Basic Concepts - 3 The confidence level measures the likelihood that the true value is within the margin of error of the sample estimate. The Census Bureau statistical standard for published data is to use the 90 percent confidence level.

6 Margin of Error The margin of error is important because relying on statistical inference can save you from drawing incorrect conclusions from data based on a sample. It can help prevent you from interpreting small or nonexistent differences as important.

7 Margin of Error (MOE) MOE = 1.65 * Standard Error 1.65 is used for the 90 percent confidence level Standard Error = MOE / 1.65 Starting in 2006 ACS will use 1.645

8 Confidence Interval Confidence Interval Estimate ± Margin of Error 90 percent confidence level Margin of Error = * Std Error 95 percent confidence level Margin of Error = 1.96 * Std Error

9 Confidence Interval The confidence interval tells you the upper and lower bounds of a range of values that may contain the true value. It provides important information about the true value or the population parameter. It tells you the limitations on using the estimates.

10 MOE / Confidence Interval Median Family Income – $30,000 Standard Error – $1,500 90% MOE = * $1,500 = $2,468 90% CI = $30,000 ± $2,468 = $27,532 to $32,468

11 Standard Error – Sum/Difference Standard Error of X + Y or X – Y SE(X+Y) = SE(X-Y)

12 Standard Error – Sum SE(X 1 +X 2 +…+X n )

13 Standard Error – Proportions P= X / Y – X is a subset of Y SE(P)

14 Standard Error – Ratios X / Y – X is not a subset of Y SE(X / Y)

15 Statistical Testing Two estimates are "significantly different" at the 90 percent confidence level if the difference between them is large enough to infer that there was a less than 10 percent chance that the difference was purely random. Users may want to compare estimates across years or geographies. It is important to note that small differences, which may be statistically significant, may not have any practical significance.

16 Statistical Testing - Steps 1.State that two estimates are statistically different if the difference between the two estimates is statistically different from zero. 2.Calculate the standard error of the difference.

17 Statistical Testing - Steps 3.Calculate the margin of error of the difference. 4.Compare the original difference between the estimates to the margin of error of the difference.

18 Statistical Testing - Steps 5.If the difference is greater than the margin of error, then you conclude that the two estimates are significantly different. 6.If the difference is less than the margin of error, you conclude that the two estimates are not significantly different.

19 Statistical Testing - Example Percent with Bachelors Degree or Higher GeographyPercentMOE CI Area ± Area ± Difference = 20.0 – 12.3 = 7.7

20 Statistical Testing - Example MOE of the Difference - Standard Errors for Each Estimate SE = MOE / 1.65 SE(Area 1) = 5.0 / 1.65 = 3.03 SE(Area 2) = 4.7 / 1.65 = 2.85

21 Statistical Testing - Example Standard Error of the Difference Margin of Error of the Difference MOE(X - Y) = 1.65 * 4.16 = 6.9

22 Statistical Testing - Example Compare the Difference to MOE –Difference = 7.7% –MOE = 6.9% Difference > MOE Conclude that the two estimates are significantly different with 90 percent confidence

23 Census 2000 – Example Percent Bachelors Degree or Higher – Alexandria, VA 51,982 / 95,730 = 54.3% DF = 1.2 – (13.4% in sample) 90% MOE =1.65 * 0.4 = % CI = 54.3 ± 0.7 = 53.6 to 55.0

24 Rules to Remember Dont make a big deal of small differences. If the confidence intervals overlap you cannot conclude the difference is not statistically significant. Always talk to subject matter experts before making any conclusions.

25 Population Controls - Rational Correct for coverage –Higher undercoverage in surveys than in census Reduce variance estimates

ACS Coverage Rates - US Total Pop MaleFemaleHispanic Non-Hispanic WhiteBlackAIANAsianNHOPI

27 Population Controls Intercensal estimates are produced by updating the previous census results using various administrative records data and ACS data on foreign-born In a multi-stage process, HU, GQ, and population adjustment ratios are applied to the weights

28 GQ Controls GQ population controls applied at the STATE level by 7 major types. Collapsing across types if not enough sample Always control to at least Institutional / Non-Institutional Population

29 Housing Unit Controls Applied at a weighting area level New step to make all 3 agree –Households –Householders –Occupied Housing Units Housing Units will not be controlled

30 Weighting Areas Controls applied at the weighting area (county or group of counties) level 1343 weighting areas consist of a single county –All 21 New Jersey counties are weighting areas The other 607 weighting areas are made up of 1798 counties

31 HU Population Controls Controls applied by race/ethnicity and age/sex groups ACS GQ estimates subtracted from population estimates to obtain controls Collapsing of race/ethnicity and age/sex groups

32 Why Do Place Estimates Differ ACS does not control subcounty areas 1-person households –Lower response rate Multi-Unit Structures –Conversion of single to multi-unit

33 Room: 4H477 Phone: U.S. DEPARTMENT OF COMMERCE U.S. Census Bureau Washington, DC Anthony Tersine Contact Information