1. Sampling and Sample Size Calculation
Lazereto de Mahón, Menorca, Spain
September 2006
Sources:
-EPIET Introductory course,
Thomas Grein, Denis Coulombier, Philippe Sudre, Mike Catchpole, Denise Antona
-IDEA
Brigitte Helynck, Philippe Malfait, Institut de veille sanitaire
Modified: Viviane Bremer, EPIET 2004, Suzanne Cotter 2005,
Richard Pebody 2006

2. Objectives: sampling To understand: Why we use sampling
Definitions in sampling
Sampling errors
Main methods of sampling
Sample size calculation

3. Why do we use sampling? Get information from large populations with:
Reduced costs
Reduced field time
Increased accuracy
Enhanced methods

4. Definition of sampling
Procedure by which some members
of a given population are selected as representatives of the entire population
Sampling is the use of a subset of the population to represent the whole population

5. Definition of sampling terms
Sampling unit (element)
Subject under observation on which information is collected
Example: children <5 years, hospital discharges, health events…
Sampling fraction
Ratio between sample size and population size
Example: 100 out of 2000 (5%)

6. Definition of sampling terms
Sampling frame
List of all the sampling units from which sample is drawn
Lists: e.g. children < 5 years of age, households, health care units…
Sampling scheme
Method of selecting sampling units from sampling frame
Randomly, convenience sample…

7. Survey errors Systematic error (or bias) Sampling error (random error)
Sample not typical of population
Inaccurate response (information bias)
Selection bias
Sampling error (random error)

8. Representativeness (validity)
A sample should accurately reflect distribution of
relevant variable in population
Person e.g. age, sex
Place e.g. urban vs. rural
Time e.g. seasonality
Representativeness essential to generalise
Ensure representativeness before starting,
Confirm once completed

9. Sampling and representativeness
Population
Sample
Target Population
Target Population  Sampling Population  Sample

10. Sampling error
Random difference between sample and population from which sample drawn
Size of error can be measured in probability samples
Expressed as “standard error”
of mean, proportion…
Standard error (or precision) depends upon:
Size of the sample
Distribution of character of interest in population
an error is the amount by which an observation differs from its expected value.

11. Sampling error
When simple random sample of size ‘n’ is selected from population of size N, standard error (s) for population mean or proportion is:
σ p(1-p)
 n n
Used to calculate, 95% confidence intervals
an error is the amount by which an observation differs from its expected value.
Estimated 95% confidence interval

12. Quality of a sampling estimate
No precision
Random error
Precision but
no validity
Systematic error (bias)
Precision & validity
précision = Precision (also called reproducibility or repeatability) is the degree to which further measurements or calculations will show the same or similar results.
validité = capacité à estimer la vraie valeur du paramètre dans la population
an error is the amount by which an observation differs from its expected value.
the latter being based on the whole population from which the statistical unit was chosen randomly. The expected value, being the average of the entire population, is typically unobservable. If the average height of 21-year-old men is 5 feet 9 inches, and one randomly chosen man is 5 feet 11 inches tall, then the "error" is 2 inches; if the randomly chosen man is 5 feet 7 inches tall, then the "error" is −2 inches. The nomenclature arose from random measurement errors in astronomy. It is as if the measurement of the man's height were an attempt to measure the population average, so that any difference between the man's height and the average would be a measurement error.
A residual, on the other hand, is an observable estimate of the unobservable error. The simplest case involves a random sample of n men whose heights are measured. The sample average is used as an estimate of the population average. Then we have:
The difference between the height of each man in the sample and the unobservable population average is an error, and
The difference between the height of each man in the sample and the observable sample average is a residual.
Residuals are observable; errors are not.
Note that the sum of the residuals within a random sample is necessarily zero, and thus the residuals are necessarily not independent. The sum of the errors need not be zero; the errors are independent random variables if the individuals are chosen from the population independently.
In statistics, when analyzing collected data, the samples observed may not be a representative sample from all possible samples. This is sampling error and is controlled by ensuring that, as much as possible, the samples taken have no systematic characteristics and are a true random sample from all possible samples. If the observations are a true random sample, statistics can make probability estimates of the sampling error and allow the researcher to estimate what further experiments are necessary to minimize it.
A systematic error is any biasing effect, in the environment, methods of observation or instruments used, which introduces error into an experiment
Constant systematic errors are very difficult to deal with, because their effects are only observable if they can be removed. To remove systematic error is simply to make a better experiment 7

13. Survey errors: example
Measuring height:
Measuring tape held differently by different investigators
→ loss of precision
Large standard error
Tape shrunk/wrong
→ systematic error
Bias (cannot be corrected afterwards)
179
178
177
176
175
174
173

14. Types of sampling
Non-probability samples
Probability samples

15. Non probability samples
Convenience samples (ease of access)
Snowball sampling (friend of friend….etc.)
Purposive sampling (judgemental)
You chose who you think should be in the study
Normally sample should reflect the population structure. A convenience sample does not! In a convenience sample some persons have a higher chance of being sampled than others. For example, if we choose people on the main shopping street as a sample for a city ward, those who are frequently doing their shopping there are at higher risk of being chosen than those working during the day, bedridden
Nonprobability sampling techniques cannot be used to infer from the sample to the general population. Any generalizations obtained from a nonprobability sample must be filtered through one's knowledge of the topic being studied. Performing nonprobability sampling is considerably less expensive than doing probability sampling, but the results are of limited value.
Examples of nonprobability sampling include:
Convenience sampling - members of the population are chosen based on their relative ease of access. To sample friends, co-workers, or shoppers at a single mall, are all examples of convenience sampling.
Snowball sampling - The first respondent refers a friend. The friend also referes a friend, etc.
Judgmental sampling or Purposive sampling - The researcher chooses the sample based on who they think would be appropriate for the study. This is used primarily when there is a limited number of people that have expertise in the area being researched.
Case study - The research is limited to one group, often with a similar characteristic or of small size.
ad hoc quotas - A quota is established (say 65% women) and researchers are free to choose any respondent they wish as long as the quota is met
Even studies intended to be probability studies sometimes end up being non-probability studies due to unintentional or unavoidable characteristics of the sampling method. In public opinion polling by private companies (or organizations unable to require response), the sample can be self-selected rather than random. This often introduces an important type of error: self-selection error. This error sometimes makes it unlikely that the sample will accurately represent the broader population. Volunteering for the sample may be determined by characteristics such as submissiveness or availability. The samples in such surveys should be treated as non-probability samples of the population, and the validity of the estimates of parameters based on them unknown
Probability of being chosen is unknown
Cheaper- but unable to generalise, potential for bias

16. Probability samples Random sampling
Each subject has a known probability of being selected
Allows application of statistical sampling theory to results to:
Generalise
Test hypotheses

17. Methods used in probability samples
Simple random sampling
Systematic sampling
Stratified sampling
Multi-stage sampling
Cluster sampling

18. Simple random sampling
Principle
Equal chance/probability of drawing each unit
Procedure
Take sampling population
Need listing of all sampling units (“sampling frame”)
Number all units
Randomly draw units

19. Simple random sampling
Advantages
Simple
Sampling error easily measured
Disadvantages
Need complete list of units
Does not always achieve best representativeness
Units may be scattered and poorly accessible
e.g simple random sampling of telephone numbers- is everyone in the telephone book? Are there people without phones? Or non-functioning phones
Because units may be scattered may be more time consuming difficult to involve units

20. Simple random sampling
Example: evaluate the prevalence of tooth decay among 1200 children attending a school
List of children attending the school
Children numerated from 1 to 1200
Sample size = 100 children
Random sampling of 100 numbers between 1 and 1200
How to randomly select?

21. EPITABLE: random number listing

22. EPITABLE: random number listing
Also possible in Excel

23. Simple random sampling

24. Systematic sampling Principle Advantages Disadvantages
Select sample at regular intervals based on sampling fraction
Advantages
Simple
Sampling error easily measured
Disadvantages
Need complete list of units
Periodicity e

25. Systematic sampling  1st person selected = the 8th on the list
N = 1200, and n = 60
 sampling fraction = 1200/60 = 20
List persons from 1 to 1200
Randomly select a number between 1 and 20 (ex : 8)
 1st person selected = the 8th on the list
 2nd person = = the 28th etc ..... 8

26. Systematic sampling

27. Stratified sampling Principle :
Divide sampling frame into homogeneous subgroups (strata) e.g. age-group, occupation;
Draw random sample in each strata.

28. Stratified sampling Advantages Disadvantages
Can acquire information about whole population and individual strata
Precision increased if variability within strata is less (homogenous) than between strata
Disadvantages
Can be difficult to identify strata
Loss of precision if small numbers in individual strata
resolve by sampling proportionate to stratum population

29. Multiple stage sampling
Principle:
consecutive sampling
example : sampling unit = household
1st stage: draw neighborhoods
2nd stage: draw buildings
3rd stage: draw households 8

30. Cluster sampling Principle
Sample units not identified independently but in a group (or “cluster”)
Provides logistical advantage.

31. Cluster sampling Principle
Whole population divided into groups e.g. neighbourhoods
Random sample taken of these groups (“clusters”)
Within selected clusters, all units e.g. households included (or random sample of these units)

32. Example: Cluster sampling
Section 1
Section 2
Section 3
Section 5
Section 4

33. Cluster sampling Advantages
Simple as complete list of sampling units within population not required
Less travel/resources required
Disadvantages
Potential problem is that cluster members are more likely to be alike, than those in another cluster (homogenous)….
This “dependence” needs to be taken into account in the sample size….and the analysis (“design effect”)

34. Selecting a sampling method
Population to be studied
Size/geographical distribution
Heterogeneity with respect to variable
Availability of list of sampling units
Level of precision required
Resources available

35. Sample size estimation
Estimate number needed to
reliably measure factor of interest
detect significant association
Trade-off between study size and resources….
Sample size determined by various factors:
significance level (“alpha”)
power (“1-beta”)
expected prevalence of factor of interest

36. Type 1 error
The probability of finding a difference with our sample compared to population, and there really isn’t one….
Known as the α (or “type 1 error”)
Usually set at 5% (or 0.05)

37. Type 2 error
The probability of not finding a difference that actually exists between our sample compared to the population…
Known as the β (or “type 2 error”)
Power is (1- β) and is usually 80%

38. A question? Are the English more intelligent than the Dutch?
H0 Null hypothesis: The English and Dutch have the same mean IQ
Ha Alternative hypothesis: The mean IQ of the English is greater than the Dutch

39. Type 1 and 2 errors Truth Decision H0 true H0 false
Reject H0 Type I error Correct decision
Accept H Correct Type II error
decision

40. Power The easiest ways to increase power are to: increase sample size
increase desired difference (or effect size)
decrease significance level desired e.g. 10%

41. Steps in estimating sample size for descriptive survey
Identify major study variable
Determine type of estimate (%, mean, ratio,...)
Indicate expected frequency of factor of interest
Decide on desired precision of the estimate
Decide on acceptable risk that estimate will fall outside its real population value
Adjust for estimated design effect
Adjust for expected response rate

42. Sample size for descriptive survey
Simple random / systematic sampling
z² * p * q
1.96²*0.15*0.85
n =
= 544 d²
0.03²
Cluster sampling
z² * p * q
2*1.96²*0.15*0.85
n = g*
= 1088 d²
0.03²
z: alpha risk expressed in z-score
p: expected prevalence
q: 1 - p
d: absolute precision
g: design effect

43. Case-control sample size: issues to consider
Number of cases
Number of controls per case
Odds ratio worth detecting
Proportion of exposed persons in source population
Desired level of significance (α)
Power of the study (1-β)
to detect at a statistically significant level a particular odds ratio

44. Case-control: STATCALC Sample size
If we have too few cases we may improve the power of our study by selecting more controls, as we said almost no extra power is gained selecting more than 4 controls but

45. Case-control: STATCALC Sample size
Risk of alpha error %
Power %
Proportion of controls exposed 20%
OR to detect > 2
The power of a statistical test is the probability that the test will reject a false null hypothesis, or in other words that it will not make a Type II error. The higher the power, the greater the chance of obtaining a statistically significant result when the null hypothesis is false.
Statistical power depends on the significance criterion, the size of the difference or the strength of the similarity (that is, the effect size) in the population, and the sensitivity of the data.

46. Case-control: STATCALC Sample size
If we have too few cases we may improve the power of our study by selecting more controls, as we said almost no extra power is gained selecting more than 4 controls but

47. Statistical Power of a Case-Control Study for different control-to-case ratios and odds ratios (with 50 cases)
a probability is a number between 0 and 1;
the probability of an event or proposition and its complement must add up to 1; and
the joint probability of two events or propositions is the product of the probability of one of them and the probability of the second, conditional on the first.
Representation and interpretation of probability values
The probability of an event is generally represented as a real number between 0 and 1, inclusive. An impossible event has a probability of exactly 0, and a certain event has a probability of 1, but the converses are not always true: probability 0 events are not always impossible, nor probability 1 events certain. The rather subtle distinction between "certain" and "probability 1" is treated at greater length in the article on "almost surely".
Most probabilities that occur in practice are numbers between 0 and 1, indicating the event's position on the continuum between impossibility and certainty. The closer an event's probability is to 1, the more likely it is to occur.
For example, if two mutually exclusive events are assumed equally probable, such as a flipped coin landing heads-up or tails-up, we can express the probability of each event as "1 in 2", or, equivalently, "50%" or "1/2".
Probabilities are equivalently expressed as odds, which is the ratio of the probability of one event to the probability of all other events. The odds of heads-up, for the tossed coin, are (1/2)/(1 - 1/2), which is equal to 1/1. This is expressed as "1 to 1 odds" and often written "1:1".
Odds a:b for some event are equivalent to probability a/(a+b). For example, 1:1 odds are equivalent to probability 1/2, and 3:2 odds are equivalent to probability 3/5.
There remains the question of exactly what can be assigned probability, and how the numbers so assigned can be used; this is the question of probability interpretations.

48. Conclusions Probability samples are the best Ensure
Representativeness
Precision
…..within available constraints

49. Conclusions
If in doubt…
Call a statistician !!!!