1. Survey Research. Sampling and Inference June 9, 2008 Ivan Katchanovski, Ph.D. POL 242Y-Y

2. 2 Survey Research Surveys: One of the main methods of data collection and research in political science and sociology Polls: Used in mass media and business Advantages Reliable Involves relatively small numbers of respondents Relatively cheap and fast – Compare with census Disadvantages – Sampling error – Validity

3. Example of Polls: Bush Approval 3

4. Observation and Sampling Polls and surveys are based on observations Population: universe of subjects the researcher wants to sample – Adult population in Canada – Voters in Ontario Sample: cases or observations drawn from a population 4

5. Random Sample Random selection: Each element has an equal chance of selection independent of any other event in the selection process – Are internet polls random? – Are telephone polls random? Weighting: Giving some cases in a survey dataset more weight than others – Weights examples 5

6. Parameters and Sample Statistics Parameters: Characteristics of a population – Proportion of the vote for political parties in national or local elections – Median personal income in Canada Sample Statistics: Estimates of population parameters, based on a sample drawn from a population – Exit poll estimate of proportion of the vote for political parties in national or local elections – Survey estimate of median personal income in Canada Inference about the characteristics (parameters) of a population based on samples 6

7. Confidence Level Surveys and polls: margin of error Random sampling error: – Decreases with increase in sample size Confidence Level – The estimated probability that a population parameter lies within a given confidence interval Example: we might be 95% confident that between 48 and 52% of all voters favor Candidate A. 7

8. Election Eve Polls - U.S. Presidential Candidates, 2004 Date Begun AgencyBushKerry 10/29Gallup/CNN/USA4951 10/29NBC/WSJ5149 10/29Harris5248 10/29CBS5149 10/30Fox/OpinDynamics4952 10/30TIPP5149 10/31GWU Battleground5248 11/2Actual Vote5248 8

9. Types of Survey Questions Open-ended questions – Respondent is asked to provide his or her own answer to the question Closed-ended questions – Respondent is asked to select an answer from among a list provided by the researcher Example: different % correct responses in open- ended and closed-ended questions 9

10. Example: Two Types of Questions in a 2007 Pew Poll of Americans Open-Ended Question % Can you tell me the name of the president of Russia? Yes, Vladimir Putin 36 Yes, other incorrect 3 No, don’t know 61 Total, 100 Closed-Ended Question, % Who is the President of Russia? Is it… (A) Vladimir Putin 60 (B) Boris Yeltsin 7 (C) Mikhail Gorbachev 7 (D) or is it someone else 8 Don’t know (Vol.) 18 Total 100 10

11. Contingency Questions and Biased Questions Contingency questions: Questions which are contingent on response to previous questions Example of a contingency question: Q14.1a. When it comes to political matters, do you ever think of yourself in terms of Left and Right? (1)Yes (2)No (9)DK, etc. (IF YES TO Q.14.1a, ASK Q.14.1b) 14.1b) How would you place your views on this scale, generally speaking – where 1 means “very Left” and 10 means “very Right”? Biased questions: Questions that encourage respondents to answer in a particular way 11

12. Response Rate Number of people participating in a survey divided by the number selected in the sample Acceptable response rates – 50% - adequate for analysis and reporting – 60% - good – 70% - very good Lower response rates – Can be used if non-respondents are similar to respondents 12