366 6.5

Sampling Defined / The idea Making inference about a larger population What is the population Some particular value in the population estimating a parameter

Sampling

Sampling Population must be defined If interested in opinions of... All adults Registered voters Likely voters Actual voters These are all distinct populations

Sampling Population must be defined If interested in opinions of... People in Whatcom County Voters in Whatcom County People in Bellingham Voters in Bellingham Likely voters in Bellingham These are all distinct populations

Sampling Population must be defined If interested in opinions of... Students at WWU Seniors at WWU (xxx # of credits & up) Students in College of Arts & Sciences etc. These are all distinct populations; who should be included, excluded

Sampling Sampling unit A single member of the population a case if population = voters sampling unit = registered voter (?) If population = conflicts / wars sampling unit = nation of a certain size; conflict of particular duration

Sampling Sampling Frame Once clear about what population & units are, how do we find them? Frame = complete list of population Registered voters; Students at WWU In reality this may not exist e.g., all people living in the US

Sampling Sampling Frame US Census How get ‘the list?’ $3billion; 500,000 workers...

Sampling Sampling Frame Registered voters; Students at WWU Piece of cake? Accuracy of sample depends on comprehensiveness of frame

Sampling Sampling Frame Ahead of time, evaluate for problems Missing elements New residents, newly registered voters, ? Clusters Census tracts, city blocks, Zip code, Area code, prefix Take random draw of clusters, then random draw of households in cluster

Classic Sample Failure 1936 Literary Digest Survey Survey of 2.4 million Americans Predicted Alf Landon 57%, FDR 43% Actual result FDR 62%, Landon 38% Frame = 10 million people subscribers to Digest; phone directories; club memberships

Sampling Sampling Frame Ahead of time, evaluate for problems Blank elements Phone directories (address w/o #) Phone #s (unassigned prefixes; fax machine; pager) List of all residents when population = voters

Classic Sample Failure 1936 Literary Digest Survey What went wrong?

Classic Sample Failure 2000 & 2004 & 2012 (WI) US Exit polls Surveys of tens of thousands 2000 initially predicted Gore win FL Actually, Bush won 2004 initially predicted Kerry win OH Frame: Key precincts, people voting at polling places

2004 VNS Exit Polls, Ohio

2004 Exit Polls State Exit (Bush) Actual Diff FL 49 52.5 -3.5 PA 46.5 48.9 -2.4 OH 51.3 -2.3 MI 46.9 48.3 -1.3 NJ 44.9 -2 NH 49.3 -4.4 NY 36.7 41.2 -4.5

Exit Polls Exit poll difficulty: Identify representative precincts Sample throughout day Estimate non-polling place vote 'Weight' data to account for sample problems: o group difference in non-response o turnout differences o vote by mail All before 8pm After polls close, weight again...and again

“This can’t happen in America. Maybe in Ohio...”

Classic Sample Failure 2000 & 2004 US Exit polls What went (goes) wrong? also response bias that favors Democrats

2016 Exit polls A bit better https://www.electiontonight.com/ https://www.electiontonight.com/
 Many states too close to call; no networks called the election until Clinton conceeded (AP did).

Sample Designs Probability vs. Non probability sampling Probability sample We know the probability that each unit in the population has of being in the sample Non probability sample We don’t know if every unit has a fixed chance of being in sample

Probability Sample & Normal Distribution Normal distribution has “areas” under it percent of observations in sample in terms of distance from mean (in s.d units) We use this to estimate the probability an observation will occur.

Probability Sample & Normal Distribution Study: Population teen girls texting Sample: 200 teen girls Sample unit: number of texts per day mean 70 texts per day, SD = 10 (z = +1.00) What is probability of selecting a teen girl in sample who texts 70 – 80 times per day?

Sample Design Probability sample If 22% of population are white males over 21 years of age... a .22 probability that a white male over 21 would end up in sample

Sampling

Sample Design Probability sample If study repeated w/ different samples, high likelihood that results similar We can estimate likelihood that things observed in the sample are representative of the population

Sample Design Real world probability sample problems Population = likely voters Good sample frame? Voters yes, likely voters no Proper randomization You try it Missing elements Land line vs. cell phones

Probability Samples Simple random sampling Systematic samples Stratified samples Cluster samples

Probability Samples Simple random sampling List each unit (person) in population Give each a number (List from 1 to n) Use random # generator If 1207 comes up, select #1207 from list Repeat

Probability Samples Systematic sample Have list of population, 1 – nth Find random #, start there on list Pick each kth unit (person) on list Hope there is no structure to list Starting point random, increment random Easier Kind of how exit polls work at polling place

Probability Sample Stratified sample Use available information from the population Dived so elements w/ in groups (strata) are more alike than population A series of homogeneous groups Stratify by race/ethnicity; income randomly sample in each subgroup; maybe over-sample a (smaller) group Combine samples into one Cheaper

Probability Samples Cluster sample Identify clusters (groups) Select large groups by random Cities, congressional districts, states, neighborhoods Randomly sample within cluster

Probability Samples Simple random sampling Systematic samples Stratified samples Cluster samples Other types, some of these used together

Non-probability Samples Convenience sample (Biased) All students in this class Population = WWU students First 200 people walking down Railroad Ave. Population = Whatcom County voters No way to know representativeness of sample

Non-probability samples Purposive sample Units selected subjectively Chance of being selected depends on researcher’s judgment “Critical elections” Population = all US Presidential elections “Major wars” Population = all wars

Non-probability sample Quota sample Purposively select sample as representative as possible Use know characteristics of population Target quota based on know characteristics

Non-probability sample Quota sample WWU (Fake example) 57% female, 43% male 45% A&S; 25% CST; 10% CBE; 10% Huxley; 10% other Age Ethnicity

Non-probability sample Quota sample Whatcom Co. (Fake example) Gender Age Partisanship City resident vs. County resident Monitor demographics of respondents as you go

Non-probability sample Quota sample Poor person’s random sampling Can fail to predict 1948 3 surveys predicted Dewey to win None targeted partisanship

Internet Samples Opt-in Provide people computers Huge samples asked to do interviews “Weight” data after responses to represent population

Sample size If sample random (ish), precision of estimates depend on size Larger = more precise estimate, all else equal Very large doesn’t add much precision

Sample size Diminishing returns on size Depends on scale of population, subgroups Whatcom Co. State of WA USA

Sample size Diminishing returns on size Depends on scale of population, subgroups Whatcom Co. State of WA USA