1. 1 Exam 1 Review

2. 2 COMM 420.8 Fall 2007 Nan Yu Survey Research

3. 3 Picking a Research Design Quantitative methods  survey  experiment  content analysis Each one of the above-mentioned designs have advantages and disadvantages, and based on what type of question a researcher is trying to answer, one is more appropriate than the others.

4. 4 Survey “Audience research”  Questioning people about their attitudes, emotions, beliefs, intentions, and behaviors.  How people evaluate or perceive issue, event or message.

5. 5 Characteristics of Survey A set of questions to measure variables Researchers tend to find the patterns among variables. Answers “ why people differ in their perceptions of an event. ”

6. 6 Difference between poll and survey PollSurvey shortlong, in-depth what participants think why people have different thoughts determine the frequency distribution of the values of one variable determine the relationship between two or more variables.

7. 7 Poll vs. Survey Poll Survey 384 240 Dem Attitude toward abortion Rep Pro-Choice Pro-Life 95.2% of Republicans are pro-life. 90.5% of Democrats are pro-choice. The data suggests that there is a relationship between political affiliation and attitude toward abortion. Pro-life: 51% Pro-choice: 49%

8. 8 Sampling (a review) Census: if all people in the population were surveyed Sampling  Non-probability  Probability Sampling error  Sampling process (sometimes random)  biases

9. 9 Sampling error (standard error) Sampling error gives us some ideas of the estimated amount of errors you are willing to tolerate in making your selection. Calculate sampling error is based on probability theory.  Simple random, systematic, and stratified samples  Sometimes researchers report sampling error if the sampling is close to probability sample.

10. 10 How to calculate sampling error  SE = Sampling error  P= Sample proportion (population parameters of interests)  The proportion of people in the population being surveyed who are expected to answer a certain way on the key measure in the survey. If you are unsure what the proportion might be, use 50% because this produces the maximum possible variation.  N=Sample size

11. 11 Example of sampling-error calculation A poll conducted before the last presidential elections showed that 48% voters wanted to vote for Bush, 52% voters wanted to vote for Gore. The poll sampled 100 people from State College. 5% sampling error suggests: In reality, the % of votes in the population can be anywhere between (43%, 53%) for Gore and between (47%, 57%) for Bush.

12. 12 Example (cont.) Imagine that we now take a sample of 1000 respondents, and we find the same percentage of votes for Bush (48%) and Gore (52%). 1.4% sampling error suggests: Range between which the true population values would fall would then be (46.6%; 49.4%) votes for Gore, and (50.6%; 53.4%) % votes for Bush.

13. 13 Sampling Error (cont.) The larger the sample size, the smaller the sampling error. How would you know then what sample size would be sufficient? (p. 224)  Depends on how accurate you want to be.

14. 14 Sampling error and sample size Sample size The greater the sample size, the less the sampling error.

15. 15 Sampling error Census  If you take a sample that consists of the entire population you actually have no sampling error because you don't have a sample, you have the entire population.

16. 16 Confidence level The confidence level tells you how sure you can be. The degree to which that you are confident about the sample can represent the true population. Most researchers use the 95% confidence level.

17. 17 Confidence level (p.224) If the population proportion is determined (50-50%, 30-70%, etc…) Sample size Sampling error (confidence interval) Confidence level

18. 18 Survey design Based on the point in time when the questionnaire is administered, survey studies can be divided into:  Cross-sectional studies observation at one point of time  Longitudinal studies observation over time establish patterns and shifts over time

19. 19 Types Longitudinal Study TrendCohortPanel different samples Samples are subgroups from a same population. same samples A cohort is a group of people who share a common characteristic or experience within a defined time period. (e.g. class of 1997, people that were born between 1990 to 1995. ) Sample size might be different over time (people moved, died…). Require probability sampling from the same population, otherwise you can not compare the data collected at different times.

20. 20 Types of surveys Watt, J. H., & van den Berg, S. A. (2002). Research methods for communication science. Needham Heights, MA: Allyn and Bacon, p.278, p. 290

21. 21 Constructing questionnaires Make a list of the variables that you want to measure. Construct questions  Types of questions Open-ended, closed questions Likert-type, semantic differential  Levels of measurement Nominal, ordinal, interval, or ratio

22. 22 Constructing questionnaires Principles: questions need to be direct, clear, and unambiguous. Avoid double-barreled questions. Does your company offer pension and health insurance benefits? Are you a republican or a democrat? Avoid double negations. They are grammatically incorrect and usually very confusing.

23. 23 Principles of constructing questionnaires  Avoid leading questions (i.e., implying the answers in the questions) Examples : Isn ’ t true that PSU tuition is too high? (implies that it is too high) What are the flaws in the nation ’ s health-care system? (implies that the system has flaws)

24. 24 Principles of constructing questionnaires  PRETEST before using it on a large scale (this will show you if there are any ambiguities, etc.)  Keep it short, so that you avoid respondents ’ fatigue (assure that the last questions are answered as well as the first ones)  Avoid jargon, slang, and abbreviations

25. 25 Problematic questions  What are problems with the following questions? “ What is your income? ” “ Do you jog regularly? ” “ Does your college offer master or doctoral programs? ” What is your total family’s household income before taxes during last year? How many times do you job per week? 123 45 67 Double-barreled questions

26. 26 Sensitive questions Researchers ask sometimes about sensitive/threatening issues  sexual behavior  drug or alcohol abuse  deviant behavior  mental health  illegal activity  controversial public issues  demographic questions (age, gender, ethnicity, income) Researchers who ask such questions have to do so with extra care.

27. 27 Techniques of asking sensitive questions Goal: to increase truthful answers to sensitive questions:  “Foot in the door” The researcher builds some trust and respondents are more comfortable towards the end with the researcher and the procedures. Place sensitive questions (e.g. demographic questions) at the end.

28. 28 Techniques of asking sensitive questions Assuring the anonymity of the respondent. Be careful with wording (e.g., forbidden versus not allowed) Self-administered questionnaires (mail or internet surveys.

29. 29 Closed Questions: Easy to response (answers are provided) If participants don’t have an opinion… Good for sensitive questions If the provided answers are not exhaustive… Fewer irrelevant or confused answers You never know if one is misunderstand the question… Easy to replicate Simplistic answers to complex issues Advantages Disadvantages

30. 30 Open Questions Advantages Disadvantages Unlimited answersHard to code/comparison More detailed answers to complex issues Respondents may be intimidated by the question Unanticipated answers (important to new measurements) Different respondents give different degrees of detail in answers Reveal participants’ thinking process Require more time and thoughts

31. 31 Surveys Advantages Disadvantages Surveys show correlational evidence (i.e., patterns of variables in a large population). Survey instruments rely solely on self-reported measures. Low reliability. Surveys are very good at discerning attitudes, beliefs and perceptions They are not good for studying behavior, emotions, etc. Have high external validity (real-world situation) They can hardly prove causation. Random error is high.

32. 32 In-class demo Researchers want to design a study to see PSU students ’ thoughts on campus bus service. Researchers assume that that 50% respondents think the campus bus service needs improvement, 50% students are satisfied with the service. Researchers decide that they can tolerate if the sampling error is 3%. The confidence level in this study is 95%. According to the table on p. 224, how many people should the researchers sample?

33. 33 Answer to in-class demo According to the table on p. 224, researchers have to sample 1,067 people.