Survey research CAS 204 Furness

Advantages & Disadvantages Advantages of using surveys: Respondents can answer large numbers of quesitons rapidly Large numbers of respondents With appropriate sampling, you can make broader generalizaitons about much larger populations Disadvantages: Problems with question formats Most survey designs do not allow us to assess casual relationships (as in cause & effect) Unwillingness of people to participate Having to decide whether results are valid

Types of Surveys Cross-sectional Trend Panels Cohorts Cross-lagged Capture what’s going on at one point in time, i.e. an opinion poll. Trend Measure the same items over time but draw different samples from the population each time. Panels Group of individuals is sampled and recruited, and the same folks answer questions over time. Cohorts Groups of people defined, most typically, by having an event in common, i.e. surveying all female corporate executives born in 1980. Cross-lagged Measure a dependent variable and an independent variable at two points in time, i.e. asking different campus groups questions about the same controversy.

Issues with sampling Researchers are tasked with the responsibility of making sure that the groups being surveyed are representative, if their aim is to make claims about larger populations. If sampling is being done on a smaller scale, the issue of accurate representation is still a factor, but it stems more from sampling size. For ex. if you’re polling PSUGA students on an issue and only get 35 respondents to participate, it would be irresponsible to utilize that data as representing the entire student body’s position.

On Writing Questions The function of question formats is to clarify both the question and response options as far as possible for respondents while giving researchers relevant categories of answers that will help them analyze results. Some questions are easier than others for respondents to answer and so researchers need to spend a lot of time crafting questions that are both intelligible and appropriate to their needs.

Question Formats Open-Ended – allow respondents to answer in their own words. Advantage is insights not gleaned with highly structure questions. Disadvantage is that they are extremely time consuming to code Dichotomous – force respondents to choose one of two possible answers. Advantage is simplifying data coding and analysis. Disadvantage is over-simplification. Multiple-Choice – provide several possible answers Likert Scale – statements (not questions) with which respondents are asked to locate their level of agreement, e.g. between “strongly agree” and “strongly disagree.” Semantic Differential Scale – present a topic or concept followed by scales anchored at each end by words that have opposite meaning, e.g. “strong” and “weak.” Researchers need to make sure that the words used best capture opposite sentiments

Common Problems with Wording Leading Questions – force the respondent into an assumption that may not be true. Example: “Why do you think the campus administration is unethical?” This could be clarified and improved by asking question with likert scale or semantic differential scale. Double-Barreled Questions – ask two questions simultaneously but allow for only one answer. Example: “Will you vote against the bill, or aren’t you going to vote?” Negative Wording – may be simple but still misunderstood. Ex: “A course in statistics should not be required for the major” Double negative – two instances of negation. Example: “Does it seem possible or impossible that the moon landing never took place?”

Does this home have an Internet connection? Yes ____ No _____ Guiding Respondents Funnel design – starts with broad questions that a respondent will be comfortable with and progresses to more specific questions. Inverted funnel – starts specific and moves to broader questions Mail surveys need to begin with relevant questions that hold interest. Phone surveys should begin with easy questions to establish comfortability with someone he or she does not know. Generally, questions related to the same theme should be grouped together. But sometimes a question may be “sandwiched” between unrelated questions to make sure all questions on the same topic are being answered consistently. Sensitive questions may be placed among more comfortable/easy questions. Filter questions need to be appropriate, for example: Does this home have an Internet connection? Yes ____ No _____

Evaluating Responses There are all kinds of ways in which surveys responses can yield inaccurate conclusions that can be misused. Consequently, researchers need to adjust for certain kinds of bias when assessing their data. It varies for each kind of survey with respect to the ways in which a survey can be administered as well as the group being surveyed. We can get into more detail about this if/when you decide to utilize surveys in your research projects.