Sampling Lecture 10

Readings are good.for.us Going forward, most readings are in Freedman et al. textbook It is essential that you have read the material before class, otherwise I cannot tell what really needs emphasis and what is just obvious from the reading. Freedman text is probably best I have used; It is especially good and I don’t want to waste your time! More than we can talk about in one semester.

Non-probability and Probability Sampling Non-probability Sampling Probability Sampling Probability Distributions Info 271B

Why Sample? Drawing populations versus ‘samples’ Reducing Error E.g., survey of iSchool masters students Reducing Error Consider Bernard’s example of interviewing all doctors in a hospital Difference between a census and survey? Info 271B

A Few Key Concepts: Units of Analysis (elements, cases, participants) Selected Sample Actual Sample Sampling Design Selected Actual Selected: who we target Actual: who responds

Method Matters… Method of choosing sample and “planned introduction of chance” Literary Digest Poll revisited The problem of bias May not be detectible in the valid data If sampling method is biased, larger sample will not help

Design: Non-Probability Sampling Quotas Pick key groups of interest and find individuals to fill specific goals (i.e., 100 people in each key group). Quotas are fulfilled without using random sampling Purposive Sampling Find key groups and only study them TWO requirements of random sampling (and one or both are violated in non-probability sampling): Researcher has no discretion over who is included The procedure for selecting sample is definite and it involves planned use of chance Purposive Sampling: that guy at Safeway. But also in a lot of field research Info 271B

More Non-Probability Sampling Convenience Sampling Taking anyone you can get to participate Snowball Sampling Find starting point and use these individuals to get next participant…and so on Info 271B

“Judgment and choice usually show bias, while chance is impartial “Judgment and choice usually show bias, while chance is impartial. That is why probability methods work better than judgment” Freedman et al., p. 342 So why probability???

Design: Probability Sampling Sampling Frames A list of units of analysis from which you take a sample Directories, local census, registered users of an online system, etc But, often cannot get an adequate sampling frame Field research Question: how does sampling frame relate to generalizability? Info 271B

Randomized Samples Simple Random Sample Requires numbering all potential participants in a given sampling frame (N) Pull random numbers from any source, use these as the sample (n) Select n units out of N such that each NCn has and equal chance of being selected. Issues… True randomization? Replacement “in the field” Simple random sampling = drawing without replacement True Randomization -> How do you insure that you are truly being random? Replacement Issues-> Example of door-to-door sampling (homogeneity of replacement) Info 271B

Systematic Random Samples Random start and sampling interval (Sample x Interval = Population) Issues Periodicity Poor sampling frames Why Use Systematic Random Samples? - Do not have ability (or resources) to literally number the population Divide total sample frame (N) by the sample you want (n), which gives you the sampling interval PERIODICITY -Occurs when there is a systematic difference that occurs throughout the sample (i.e., the sample frame isnt random) (www.socialresearchmethods.net) Info 271B

Bias! From voter survey examples alone… Nonvoters…Undecided…Response bias…Non-response bias…Checking the data…Interviewer control The point is not that bias is everywhere in sampling. The point is that we have to anticipate it, find it even when we didn’t anticipate it, and deal with it.

Chance Error Estimate = parameter + bias + chance error Chance error = sampling error: we expect this! It happens in all samples Bias = non-sampling error: we try to do everything possible to avoid it, kill it, stamp it out.

How Large of a Sample? Sample Accuracy versus Sample Precision Key Issues for Determining Sample Size: Heterogeneity of Population Number of Subgroups Size of Subgroup Precision of sample statistics Sample Accuracy = Every element has an EQUAL chance of being selected Sample Precision = Increasing size of an unbiased sample Heterogeneity: How much do people differ on variables of interest? Number of subgroups and size of them: (SEE NEXT 2 SLIDES) Info 271B

Sampling (no subgroups): Attitudes About the President (1-5 Likert Scale) Suppose we want to know the mean about attitudes re: president…

SaDo men and women differ in their assessment of the US president? But what happens when we want to split into sub-groups?

Stratified Random Sampling Key Issue: Representation of salient sub-populations Maximize between-group variance while minimizing within-group variance Proportionate Samples Do you know the key independent variables? If not, may be better off avoiding stratification Disproportionate Samples Weighting Info 271B

Complex Sampling Designs Cluster Sampling No available frames Based on areas, institutions, or ‘clusters’ Randomly select clusters, then sample from those clusters (or, sample the entire cluster) Info 271B