This will help you understand the limitations of the data and the uses to which it can be put (and the confidence with which you can put it to those.

Slides:



Advertisements
Similar presentations
Probability models- the Normal especially.
Advertisements

Sampling.
Evaluating Hypotheses Chapter 9. Descriptive vs. Inferential Statistics n Descriptive l quantitative descriptions of characteristics.
Intro to Statistics for the Behavioral Sciences PSYC 1900 Lecture 9: Hypothesis Tests for Means: One Sample.
Full time and part time employment Coventry population in employment by gender Source: Annual Population Survey, Office for National Statistics
Confidence Interval A confidence interval (or interval estimate) is a range (or an interval) of values used to estimate the true value of a population.
Inferential Statistics
AM Recitation 2/10/11.
 The situation in a statistical problem is that there is a population of interest, and a quantity or aspect of that population that is of interest. This.
Topics: Statistics & Experimental Design The Human Visual System Color Science Light Sources: Radiometry/Photometry Geometric Optics Tone-transfer Function.
PARAMETRIC STATISTICAL INFERENCE
Section 8.1 Estimating  When  is Known In this section, we develop techniques for estimating the population mean μ using sample data. We assume that.
© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
Confidence Intervals: The Basics BPS chapter 14 © 2006 W.H. Freeman and Company.
Foundations of Sociological Inquiry Statistical Analysis.
Estimating a Population Mean
1 Copyright © 2010, 2007, 2004 Pearson Education, Inc. All Rights Reserved. Example: In a recent poll, 70% of 1501 randomly selected adults said they believed.
Sample Size Mahmoud Alhussami, DSc., PhD. Sample Size Determination Is the act of choosing the number of observations or replicates to include in a statistical.
Statistical Inference for the Mean Objectives: (Chapter 8&9, DeCoursey) -To understand the terms variance and standard error of a sample mean, Null Hypothesis,
As a data user, it is imperative that you understand how the data has been generated and processed…
Tom.h.wilson Department of Geology and Geography West Virginia University Morgantown, WV.
Tom.h.wilson Department of Geology and Geography West Virginia University Morgantown, WV.
Dr.Theingi Community Medicine
BA5001 Business Decision Making Autumn 2014 Session 4 – Lecture Estimation Module Leader: Dr Francisca Tej.
Class Six Turn In: Chapter 15: 30, 32, 38, 44, 48, 50 Chapter 17: 28, 38, 44 For Class Seven: Chapter 18: 32, 34, 36 Chapter 19: 26, 34, 44 Quiz 3 Read.
Lecture 5.  It is done to ensure the questions asked would generate the data that would answer the research questions n research objectives  The respondents.
CHAPTER 6: SAMPLING, SAMPLING DISTRIBUTIONS, AND ESTIMATION Leon-Guerrero and Frankfort-Nachmias, Essentials of Statistics for a Diverse Society.
AC 1.2 present the survey methodology and sampling frame used
Marketplace: 2017 Cell Phone Risk-Knowledge Study
GOVT 201: Statistics for Political Science
And distribution of sample means
GS/PPAL Research Methods and Information Systems
Module 9: Choosing the Sampling Strategy
Chapter Eight Estimation.
Chapter 6 Inferences Based on a Single Sample: Estimation with Confidence Intervals Slides for Optional Sections Section 7.5 Finite Population Correction.
Chapter 5 STATISTICAL INFERENCE: ESTIMATION AND HYPOTHESES TESTING
ESTIMATION.
Lecture 3 Biostatistics in practice of health protection
ECO 173 Chapter 10: Introduction to Estimation Lecture 5a
Sampling Distributions and Estimation
Essential Statistics Inference about a Population Proportion
Confidence Intervals and p-values
1. Estimation ESTIMATION.
Introduction to estimation: 2 cases
CHAPTER 6 Statistical Inference & Hypothesis Testing
Review of Basic Statistical Concepts
Confidence Intervals: Sampling Distribution
SAMPLING (Zikmund, Chapter 12.
Confidence Intervals for a Population Mean,
ECO 173 Chapter 10: Introduction to Estimation Lecture 5a
Chapter 7 Survey research.
Review: What influences confidence intervals?
Producing Data Chapter 5.
Random sampling Carlo Azzarri IFPRI Datathon APSU, Dhaka
Geology Geomath Chapter 7 - Statistics tom.h.wilson
Arithmetic Mean This represents the most probable value of the measured variable. The more readings you take, the more accurate result you will get.
Section 7.7 Introduction to Inference
Samples and Populations
Week Three Review.
Lecture: statistical significance.
Confidence Intervals Accelerated Math 3.
SAMPLING (Zikmund, Chapter 12).
BUSINESS MATHEMATICS & STATISTICS.
Sampling Methods.
Chapter 8: Estimating With Confidence
Part 4: Margins of Error Frank Porell
Lecture Slides Elementary Statistics Twelfth Edition
Estimating Population Parameters Based on a Sample
What do Samples Tell Us Variability and Bias.
Essential Statistics Inference about a Population Proportion
Presentation transcript:

As a data user, it is imperative that you understand how the data has been generated and processed…

This will help you understand the limitations of the data and the uses to which it can be put (and the confidence with which you can put it to those uses).

Ipsos MORI’s technical note Available on the survey website Contents: Sampling methodology Data collection Data processing Data weighting Statistical reliability…

An intro to sampling Not feasible to question all of the people in the population we are interested in i.e. all residents in a local authority area (this would be a ‘census’) Population… Sampling is making an inference about a… …from a… …Sample

We use sampling in our everyday lives! Do you need to eat the whole pot to see if it is correctly seasoned? Or Will just tasting it be enough? TASTING IT - PROVIDED IT’S WELL STIRRED!

NHTS uses probability sampling… That is every sampling unit (an address) has a known and non-zero probability of selection we know the size of the population – the total no. of addresses – we are drawing from we know the chance of selection that applies to every individual unit – an address – within the population (“1 in n”). This does not apply to quota sample surveys which are the most commonly used approach in market research. Random probability sampling is theoretically purer it is less prone to non-response bias all statistical reliability tests assume random sampling

Statistical reliability A sample survey produces figures which are estimates of the ‘truth’: that is, we are drawing inferences about the entire population based on a sample (the ‘true’ figure would be based on a survey of the entire population i.e. a census) Statistical reliability is a statement about those estimates and the confidence we have about them in relation to the ‘truth’ Reliability is sometimes referred to as “confidence intervals”, “margins of error” or “sampling tolerances” It is determined by: the percentage the sample size on which the percentage is based the level of confidence we want to apply – it is usual to test at the 95% confidence level the effect of any weights applied

Confidence intervals and sample size Larger samples provide more accurate data But to achieve double the reliability we need to quadruple the sample size For example, the following figures are for a 50% finding: Sample size 50 100 200 400 Confidence interval + 14% 10% 7% 5%

Some examples For an NHTS survey based on 400 responses and in relation to a finding that 50% are fairly satisfied with a service: “We are 95% confident that our sample percentage is reliable to plus or minus 5%.” Put another way: “Out of every 100 surveys we conduct where we see these figures, in 95 of them we would be right and the true figure lies within that range, in 5 of them we would be wrong and the true figure would not lie in that range.” “The chances are 95 in 100 that this result would not vary by more than 5 percentage points from the ‘true’ result, that which would be found had the entire population responded i.e. the ‘true’ result would be between 45% and 55%.” Another example, based on 100 responses: “We are 95% confident that our sample percentage is reliable to plus or minus 10%. The ‘true’ result would be between 40% and 60%.”

Statistical tests should not be used for sample sizes less than 50 – in fact, any estimates derived from small samples should be considered, at best, indicative.

A full list of confidence intervals for a range of sample sizes and percentages is provided in Ipsos MORI’s Technical note.

Statistical significance between %s When we extend this to look at two figures (survey estimates), we are usually interested in the likelihood of the difference between the two figures being ‘real’: In other words, how confident are we that the difference reflects what we would have found if we had surveyed the entire population? The issue may apply either to two figures from one sample or to a comparison of figures in two different samples for example: comparing the views of men and women comparing the views of one authority’s residents with another's comparing attitudes among one authorities’ residents and how they have changed between 2009 and 2010 The answer is obtained using a test of statistical significance

Statistical significance: some examples For two samples based on 1,000 responses and with a difference of 5 percentages points (50% vs 55%) The confidence interval is +4. “We are 95% confident that this is a statistically significant finding.” With the same %s based on 500 vs 500 responses The confidence interval is +6. “We are 95% confident that this is not a statistically significant finding”. With the same %s based on 100 vs 100 responses The confidence interval is +14.

This means that there will be different confidence intervals involved in comparisons of survey %s for local authority A vs B, and comparisons of C vs D (assuming that A-D received different numbers of responses).

There can still be merit in reporting on, and using findings, which are not statistically significant but caution should be exercised the smaller the sample size. We cannot use these findings with the same degree of confidence.

A full list of confidence intervals for different sample size comparisons and percentages is provided in Ipsos MORI’s Technical note.