Exam….

Sampling Basics Jeremy Kees, Ph.D.

Conceptually defined… Sampling provides a means of gaining information about the population without the need to examine the population in its entirety Sampling is far more important for descriptive/causal research than exploratory research

A quality sample allows us to use a sample statistic to tell us something about a population parameter In other words, we can draw confident conclusions about how the larger population will behave without surveying everyone in that population!

Why sample? Usually not practical to collect try and collect data from the entire population of interest –But sometimes it is! Workforce Villanova B2B customers

In context…. Sampling is one “piece of the puzzle” to getting valid data The sample could be great, but if your measures are bad, the data will be bad The composition of your sample directly impacts the robustness of your learnings –Sampling strategy relates to the internal or external validity of your findings??

External Validity The degree to which the conclusions in our study would hold for other persons in other places and at other times

Threats to External Validity The population you want to generalize to is different from your sample The setting/place you want to generalize to is different from the place you conducted your study The time period when you conducted your study was peculiar

How do we strengthen external validity?? Use random selection from the population of interest Replicate your study with different samples, in different settings, and over time

Sample Design Includes method of selection, sample structure, and analysis plans The goal is to achieve a balance between –Required precision –Available resources

Step 1: Define the population Specify the target population as clear and complete as possible –This is your “sampling frame” List of all units in your population Not always available –In fact, it’s rarely available for me

Procedure for Drawing a Sample Select the Sample Elements Define the Target Population Identify the Sampling Frame Select a Sampling Method Determine the Sample Size Collect the Data from the Designated Elements Step 1 Step 2 Step 4 Step 3 Step 5 Step 6

Defining the Population (Step 1) Who are we interested in generalizing to?

Define the Sample Frame In some cases, we can identify all possible cases and then randomly select our sample However, in social science research, this is not typically the case We do our best to chose a sampling frame that will give us our best shot and obtaining a sample that is representative of the population of interest

Select a Sampling Method So we have decided who we’re going to sample from, now we have to decide HOW to go about doing it

Sampling Methodologies: Probability Sampling Simple Random Sampling –Systematic Random Sampling Stratified Random Sampling Cluster Random Sampling

Sampling Methodologies: Nonprobability Sampling Convenience Sampling Modal Instance Sampling Expert Sampling Quota Sampling Heterogeneity Sampling Snowball Sampling

Simple Random Sampling Every member of the population has the same chance of being selected Advantages –Simple –Easy calculation of error Limitations –Need complete and accurate population listing

Cluster Sampling Clusters are formed and then a random sample is taken from each cluster –Clusters are heterogeneous –Geography Advantages –Quick/easy (relative to other forms of random sampling) –Don’t need complete population info

Stratified Sampling Strata are formed and then a random sample is taken from each strata –Stratas are homogenous segments Advantages –Ensures underrepresented groups in the population are represented in the sample

Quota Sampling Stratify the population an important variable and then sample each stratum Advantages –Quick/easy (relative to other forms of random sampling) –Ensures representation from minorities in the population Use of quotas to get an approximate representative sample (CWL example)

Convenience Sampling Using those who are willing to volunteer (or forced to participate) Advantages –Quick/easy (relative to other forms of random sampling) –Inexpensive Limitations –Generalization problems

Sample Size Sample size is a function of… –Effect size –Alpha level (precision) –Statistical power Sometimes difficult to estimate –Sample size calculatorSample size calculator

Just to get us thinking… TED Talk Analysis Issues to consider –Collect as much data as you can reasonably afford –It is often useful to look at segments

Panel Companies Qualtrics Research Now CriticalMix Others… Even professional marketing research firms need accountability Attention screeners are highly recommended

Data Costs Cost by Incidence Rate (IR) Gen Pop (100% IR) = $4-5 –Mturk = $0.50 or less Specialized Sample (<5% IR) = $ –Adolescent smokers Physicians = $150+

Sampling Challenges Adolescent Smokers –Challenges??

Sampling Challenges Veterinarians –Challenges??

Sampling Challenges Pet owners –65% female –Age –Urban/suburban –$75k+ HHI –Have visited the vet in the past year –Pet sleeps indoors –Must be a “conscientious” owner Eg, celebrate pets bday; purchase holiday gifts for pet; feel guilty about leaving the pet alone, etc. Challenges??

Sampling Challenges Any research that will inform very important decisions that will have major impact –Food/Tobacco policy decisions (warnings/disclosures) –Industry initiatives (FOP labeling) Challenges?? –As more $$ or human wellbeing is at stake, sampling becomes very important

Team Assignment #3 ** Don’t start on this assignment until you’ve read Fowler (CH 6-7) Based on your research design 1.Write a paragraph about what your measurement instrument is supposed to accomplish 2.Make a list of what should be measured to accomplish the goals of the study 3.Develop your measurement instrument

Team Assignment #3 Deliverables include: 1.A very clean, polished version that you could use to actually collect data This means you will need to carefully think through all of the issues we covered tonight (e.g., set-up, ordering, length, multi-item scales, etc.) 2.Intro paragraph and variable list (see previous slide) (Note: Don’t worry about defining your sample--you’ll have your chance to do that next week)

Assignment #4 Develop a sampling plan 1.Discuss your target market and define the sampling frame 2.Select a sampling method and explain why it is most appropriate for your study 3.Determine your sample size 4.Explain your data collection strategy (e.g., how you would go about getting your sample to participate)