The Logic of Sampling Week 2 Day 2 DIE 4564 Research Methods

Types of Research Populations At least four different types of populations must be considered when preparing to collect data: The results of the study should be applicable to the target population The source population is a well-defined subset of individuals from the target population The sample population is the individuals from the source population who are asked to participate The study population is the members of the sample population who actually participate in the study

Target Populations A well-defined study question identifies a target population to which the results of the study should apply. A target population might be quite narrow (like one wing of a long-term acute care hospital) or relatively large (like a whole country). Unless the target population is very small, measuring the entire target population or even randomly sampling from it may be impossible.

Source Populations A source population (sometimes called a sampling frame) consists of an enumerated list of population members. For example: All women with a breast cancer diagnosis in the past 2 years who are indexed in a particular cancer registry All members of a professional sports league All households within 2 miles of a particular nuclear power plant

Sample Populations A source population is often much larger than the sample size required for a study. In this situation, only a portion of the source population is selected to serve as a sample population. Sampling methods can be categorized as probability-based or non-probability.

Examples of Types of Probability Sampling

Example of a non-probability-based sample A non-probability-based convenience population can be selected based on the ease of access to those individuals, schools, or communities. Caution- Convenience samples may not be representative!

Study Populations The study population will consist of the members of the sample population who can be located, who consent to participation, and who meet all eligibility criteria. A 100% participation rate is extremely rare. A low response rate may result in nonresponse bias if the members of the sample population who agree to be in the study are systematically different from nonparticipants. What % response would you think is “representative” of a population?

Study Populations A less than 100% participation rate is usually not a problem as long as the researcher: Uses suitable and carefully explained sampling methods Takes appropriate steps to maximize the participation rate Recruits an adequately large sample size

Cross-Sectional Surveys The goal of most cross-sectional surveys is to describe a specific target population accurately. Convenience samples rarely result in a study population that is representative of the target population. Ideally, the researcher needs some way to confirm that the source population is similar to the target population and that the sample population is similar to the source population.

Case-Control Studies All cases must have the same disease, disability, or other health-related condition. The controls must be similar to the cases in every way except for their disease status, so cases and controls should be drawn from populations with similar demographics.

Cohort Studies Longitudinal cohort studies: the participants should be representative of the source and target populations  The requirements for longitudinal studies are similar to those for cross-sectional studies, since both study designs recruit population-based samples. Prospective / retrospective cohort studies: the exposed and unexposed should be drawn from similar populations  The recruitment of exposed and unexposed for cohort studies is like the recruitment of the cases and controls for case-control studies.

Experimental Studies Experimental studies require a source population that is reasonably representative of the target population. Safety is always the top priority in designing an experimental study. The risk of harm to participants can be reduced by selecting an appropriate source population and defining strict inclusion and exclusion criteria.

Vulnerable Populations Vulnerable populations in health research include some people with poor health, some people with limited decision-making capacity, and members of some socially marginalized groups, among others. Despite the potential risks of including members of these populations in research studies, including them is the only way to study health issues in these groups.  Example: The health of prisoners can only be studied by conducting research in prisons.

Vulnerable Populations Research conducted with members of vulnerable populations requires extra consideration of the potential risks of research to participants. The ability of every participant to provide informed consent free from coercion must be assured. Concerns about the increased risks of adverse effects from study participation must be addressed.

Community Involvement Some studies benefit from or require the participation and/or support of whole geographic, cultural, or social communities and their leaders. Community-based studies often work best when they use methods such as those developed for Community-Based Participatory Research.

Nonprobability Sampling* Nonprobability Sampling – any technique in which samples are selected in some way not suggested by probability theory.  Reliance on available subjects (convenience)  Purposive or judgmental sampling  Snowball sampling  Quota sampling

Nonprobability Sampling Reliance on Available Subjects  Convenience sampling  Does not allow for control over representativeness.  Only justified if less risky methods are unavailable.  Researchers must be very cautious about generalizing when this method is used.  When might this method be appropriate?

Nonprobability Sampling Purposive or Judgmental Sampling – a type of nonprobability sampling in which the units to be observed are selected on the basis of the researcher’s judgment about which ones will be the most useful or representative.  Small subsets of a population  Two-group comparison  Deviant cases  When might this method be appropriate?

Nonprobability Sampling Snowball Sampling – a nonprobability sampling method whereby each person interviewed may be asked to suggest additional people for interviewing.  Often used in field research, special populations  When might this method be appropriate?

Nonprobability Sampling Quota Sampling – a type of nonprobability sampling in which units are selected into a sample on the basis of pre- specified characteristics, so that the total sample will have the same distribution of characteristics assumed to exist in the population being studied.  Similar to probability sampling, but has problems: quota frame must be accurate, selection of sample elements may be biased  When might this method be appropriate?

Nonprobability Sampling Selecting Informants  Informant – someone who is well versed in the social phenomenon that you wish to study and who is willing to tell you what s/he knows about it.

Nonprobability Sampling Review Question  A researcher studying college success knows that a particular university’s student body is 40% first years, 25% second years, 20% third years, and 15% fourth years. The researcher selects cases to match this distribution. What kind of nonprobability sampling technique has the researcher used?

Nonprobability Sampling Review Question  Because the researcher is sampling in order to match the population distribution, the quota sampling technique is being used.

The Theory and Logic of Probability Sampling Probability Sampling – the general term for samples selected in accord with probability theory.  Often used for large-scale surveys.  If all members of a population were identical in all respects there would be no need for careful sampling procedures. However, this is rarely the same.  A sample of individuals from a population must contain the same variations that exist in the population.

The Theory and Logic of Probability Sampling Conscious and Subconscious Sampling Bias  Bias – those selected are not typical nor representative of the larger population.

The Theory and Logic of Probability Sampling Representativeness and Probability of Selection  Representativeness – the quality of a sample of having the same distribution of characteristics as the population from which it was selected.  Samples might not need not be representative in all respects, yet must be representative in all aspects relevant to the research question.

The Theory and Logic of Probability Sampling Representativeness and Probability of Selection  A sample will be representative of the population from which it is selected if all members of the population have an equal chance of being selected in the sample.  EPSEM (Equal Probability of Selection Method) - method to create a random sample

The Theory and Logic of Probability Sampling Representativeness and Probability of Selection  Advantages of Probability Sampling 1. Probability samples are typically more representative than other types of samples because biases are avoided. 2. Probability theory permits researchers to estimate the accuracy or representativeness of the sample.

The Theory and Logic of Probability Sampling Key terms:  Element – that unit of which a population is composed and which is selected in a sample.  Population – the theoretically specified aggregation of the elements in a study.  Study Population – a sampling method in which each element has an equal chance of selection independent of any other event in the selection process.

The Theory and Logic of Probability Sampling Key terms: Random Selection – each element has an equal chance of selection independent of any other event in the selection process. Sampling Unit – that element or set of elements considered for selection in some stage of sampling. Parameter – a summary description of a given variable in a population

Ex: Sampling Distribution of Ten Cases

The Theory and Logic of Probability Sampling Sampling Distribution and Estimates of Sampling Error  Statistic – the summary description of a variable in a sample, used to estimate a population parameter.

The Theory and Logic of Probability Sampling Sampling Distribution and Estimates of Sampling Error  Sampling Error – the degree of error to be expected of a given sample design.

Sample Size Sample size determination is the act of choosing the number of observations to include in a statistical sample. Sample size must be large enough to make inferences about a population from a sample.

Ways to determine sample size* Expedience based on what is doable or affordable (convenience) Use a target variance for an estimate to be derived from the sample eventually obtained. Use a target for the power of a statistical test to be applied once the sample is collected.

Importance of Sample Size An adequate number of study participants is required to achieve valid and significant results.

Importance of Sample Size The goal is to recruit just the right number of participants based on statistical estimations of how many people are required to answer the study question with a specified level of certainty. If more participants are recruited than are statistically required, resources are wasted. If too few participants are recruited, the whole study will be almost worthless because there will not be enough statistical power to answer the study question.

Bigger Samples Are Better Large samples from a population are usually better than small ones at yielding a sample mean close to the true population value.

Sample Size and Means

Bigger Samples Are Better When the sample size is small, the sample mean may be quite far from the mean in the total population from which the sample was drawn. This is represented by a wide confidence interval that reaches far from the sample mean. When the sample size is large, the sample mean is expected to be close to the population mean, and the confidence interval will be narrower.

Larger Samples from a Population Have a Narrower 95% Confidence Interval Than Smaller Samples

Sample Size Estimation A sample size calculator – more accurately called a sample size estimator – should be used to identify an appropriate sample size goal. Sample size estimators suggest an appropriate minimum sample size based on a series of “best guesses” the researcher makes about the expected characteristics of the sample population. When in doubt, err on the size of a larger sample!

FIGURE Examples of Sample Size Calculation

Power Estimation Another way to check for sample size requirements is to work backward from the number of participants likely to be recruited to see whether that sample size provides adequate statistical power for the study design. Statistical power is the ability of a statistical test to detect significant differences in a population when differences really do exist.

Power Estimation Sometimes a sample population does not capture the true experience of the population: Type 1 errors (α) occur when a study population yields a significant statistical test result when one does not exist in the source population. Type 2 errors (β) occur when a statistical test of data from the study population finds no significant result when one actually exists in the source population. Power = 1 – β

FIGURE Power and Errors

Examples of Power Calculation

Be prepared to rethink the study question, study approach, and/or target and source populations if the power for the estimated number of participants is not sufficient.

The Theory and Logic of Probability Sampling Confidence Levels and Confidence Intervals  Confidence Level – the estimated probability that a population parameter lies within a given confidence interval.  Confidence Interval – the range of values within which a population parameter is estimated to lie.

The Theory and Logic of Probability Sampling Review Question  True or False: Regardless of the sample size, the mean of the sampling distribution will equal the true population mean.

The Theory and Logic of Probability Sampling Review Question  True: Regardless of the sample size, the mean of the sampling distribution will equal the true population mean.

Populations and Sampling Frames Sampling Frame – a list of units that compose a population from which a sample is selected.  If the sample is to be representative of the population, it is essential that the sampling frame include all members of the population.

Populations and Sampling Frames Review of Populations and Sampling Frames 1. Findings based on a sample represent only the aggregation of elements that compose the sampling frame. 2. Sampling frames do not include all the elements their names might imply. Omissions are inevitable. 3. To be generalized, all elements must have equal representation in the frame.

Populations and Sampling Frames Review Question  To study college performance, a researcher obtains a list of all enrolled students at the university from the Registrar’s Office. This list is called what?

Populations and Sampling Frames Review Question  This list would be called the sampling frame. The researcher could then select cases from that list to comprise the sample.

Types of Sampling Designs Simple Random Sampling Systematic Sampling Stratified Sampling Implicit Stratification in Systematic Sampling

Types of Sampling Designs Simple Random Sampling – a type of probability sampling in which the units composing a population are assigned numbers. A set of random numbers is generated and the units having those numbers are included in the sample.  May be time consuming.  Used in experimental designs.

Types of Sampling Designs

Systematic Sampling – a type of probability sampling in which every k th unit in a list is selected for inclusion in the sample.  Easier than simple random sampling.  In Social Science may even be considered more accurate than simple random samples.

Types of Sampling Designs Systematic Sampling  Sampling Interval – the standard distance between elements selected from a population in the sample.

Types of Sampling Designs Systematic Sampling  Sampling Ratio – the proportion of elements in the population that are selected to be in a sample.

Types of Sampling Designs Stratified Sampling  Stratification – the grouping of units composing a population into homogenous groups (strata) before sampling.  Per Social Science, slightly more accurate than simple random sampling.  Stratification is a modification to simple random and systematic sample methods.

Types of Sampling Designs Stratified Sampling

Types of Sampling Designs Implicit Stratification in Systematic Sampling  Systematic sampling can, under certain conditions, be more accurate than simple random sampling.  Particularly when the arrangement of the list is implicitly stratified.

Types of Sampling Designs Illustration: Sampling University Students  Study population and sampling frame  Stratification  Sample selection  Sample modification

Types of Sampling Designs Review Question  When using systematic sampling, the first unit is selected by __________.

Types of Sampling Designs Review Question  When using systematic sampling, the first unit is selected by random choice.

Multistage Cluster Sampling Cluster Sampling – a multistage sampling in which natural groups are sampled initially with the members of each selected group being sub- sampled afterward. Used when it is not practical or possible to create a list of all elements that compose the target population. Highly efficient, but less accurate.

Multistage Cluster Sampling Multistage Designs and Sampling Error

Multistage Cluster Sampling Stratification in Multistage Cluster Sampling  Stratification can take place at each level of sampling.

Multistage Cluster Sampling Probability Proportionate to Size (PPS) Sampling – a type of multistage cluster sample in which clusters are selected not with equal probabilities but with probabilities proportionate to their sizes—as measured by the number of units to be sub-sampled.  A more sophisticated form of cluster sampling.

Multistage Cluster Sampling Disproportionate Sampling and Weighting  Weighting – assigning different weights to cases that were selected into a sample with different probabilities of selection.

Probability Sampling in Review Probability sampling remains the most effective method for the selection of study elements for two reasons.  Probability sampling avoids researchers’ conscious or subconscious biases in element selection.  Probability sampling permits estimates of sampling error.

The Ethics of Sampling Because probability sampling always carries a risk of error, the researcher must inform readers of any errors that might make results misleading.

The Ethics of Sampling Sometimes, nonprobability sampling methods are used to obtain the breadth of variations in a population. In this case, the researcher must ensure that readers do not confuse variations with what’s typical in the population.

Quick Quiz

Chapter 7 Quiz One of the most visible uses of survey sampling lies in _____. A. political polling B. probability sampling C. core sampling D. nonprobability sampling

Chapter 7 Quiz Answer: A. One of the most visible uses of survey sampling lies in political polling.

Chapter 7 Quiz _____ sampling occurs when units are selected on the basis of pre-specified characteristics. A. Snowball B. Quota C. Purposive D. Probability

Chapter 7 Quiz Answer: B. Quota sampling occurs when the units are selected on the basis of pre-specified characteristics.

Chapter 7 Quiz _____ describes a sample whose aggregate characteristics closely approximate the aggregate characteristics of the population. A. Exclusion B. Probability sampling C. EPSEM D. Representativeness

Chapter 7 Quiz Answer: D. Representativeness describes a sample whose aggregate characteristics closely approximate the aggregate characteristics of the population.

Chapter 7 Quiz A _____ is the list of elements from which a probability sample is selected. A. confidence level B. confidence interval C. sampling frame D. systematic sample

Chapter 7 Quiz Answer: C. A sampling frame is the list of elements from which a probability sample is selected.

Chapter 7 Quiz _____ is the general term for samples selected in accord with probability theory. A. Nonprobability analysis B. Correlation C. Probability sampling

Chapter 7 Quiz Answer: C. Probability sampling is the general term for samples selected in accord with probability theory.

Chapter 7 Quiz A _____ population is the aggregation of elements from which a sample if actually selected. A. theoretical B. small C. large D. concept E. study

Chapter 7 Quiz Answer: E. A study population is the aggregation of elements from which a sample if actually selected.