JAMM 444: Public Opinion Survey methodology Comparing survey methods Planning your surveys

Tuesday, Sept. 19 Schedule update Survey methods Sources of error Meet in small groups: team project

Schedule update TODAY: Chapter 3, Survey methods Thursday: Chapter 9, Using public opinion data Sept. 26: Barbara Foltz & Stephanie Kane, UI Social Science Research Unit Sept. 28: Valerie Steffen, president, Strategic Intelligence (Boise)

Survey methodology Why do a survey? Measure opinion of a population too large to contact individually Save time, save money  “Surveys can be used in a scientific way to realize the great benefits of interviewing a representative sample instead of the whole population.” Salant & Dillman, 1994

Surveys work, if... Sample is large enough to yield desired precision. Everyone in population has equal chance of being selected. Questions enable respondents to give accurate answers. Sampled respondents have similar characteristics to non-respondents.

5 key elements of a survey 1. Population 2. Type of sample 3. Method(s) for gathering data 4. Questionnaire 5. Analyses and inferences

5 steps for a successful survey 1. Be specific about what new information you need and why. 2. Understand, avoid 4 kinds of error. 3. Choose the best survey method. 4. Write good questions. 5. Test questionnaire before beginning real survey.

Choosing a sample Identify population of interest:  University of Idaho students  Moscow residents  Idaho voters  ‘The American public’

Choosing a sample Non-probability sampling  1936, Literary Digest Probability sampling  Simple random sampling (rarely used)  Systematic random samples (every 30th name in student directory)  Stratified sample: divided into groups (strata)  Cluster sampling (spread over a wide area)

Cluster sampling Used for many national surveys:  National Election Survey (NES)  General Social Survey (GSS)  Gallup Organization 350 geographical segments Sample to reflect U.S. as a whole (geographic area, size of community) Random telephone numbers in each area

How big is a typical sample? UI or Moscow: 200 to 500 Idaho: 500 to 800 National: 800 to 1,200

Four sources of error 1. Coverage error: Not allowing every person in population equal chance of being sampled. 2. Sampling error: Only some members of study population are asked to respond. (Range of possible results)

Four sources of error, cont. 3. Measurement error: inaccurate answers due to improper question wording 4. Non-response: Some people in sample don’t respond; different from respondents

Sampling error The probable difference in results between interviewing everyone in the population vs. a scientific sample taken from the population. Expressed as “plus or minus X percentage points”

Sampling error See explanation, note, p. 79, PO Rules of thumb:  The closer the margin, the larger the sample that is needed to reduce error.  For very large populations, a relatively small sample can produce reasonably accurate results. (See overhead)

How large a sample do you need? The answer depends on: How much sampling error you can tolerate The size of your population, if the population is small How varied the population is The smallest sub-group that you are likely to analyze

Survey methods 1. Face to face 2. Telephone 3. Mail 4. Internet What are advantages & disadvantages of each method? Consider: cost, time, response rate, amount of information

For Thursday Read:  PO9: Public Opinion & Policy Making

Small Groups Team 1: UI issue or controversy Team 2: Moscow or Latah County issue or controversy Team 3: Political knowledge and/or civic engagement by UI students Team 4: Idaho state candidates or U.S. House of Representatives races Team 5: Idaho state ballot issues

Small Groups 1. Meet in groups for at least 15 minutes 2. Refine your topic 3. Brainstorm target population, survey methodology, timetable 4. Choose time and place to meet again before Sept. 26

Small Groups ONE-PAGE MEMO due Sept. 26: Choose a name for your group (e.g., Bird’s Eye Consulting) List names of all participants. Rationale for survey (why?) & target population (who?) Topics or candidates (what? who?) Methods (how?)