Survey Research

What is survey? The survey is a non-experimental, descriptive research method Survey can be useful when a researcher wants to collect data on phenomena that cannot be directly observed (such as opinions and behaviors)

Surveys are Everywhere Pop-up quick polls Evaluations On-line Product Registrations Email Initiated Surveys Product Downloads requiring survey Election Exit Polls

Types of surveys Surveys can be divided into two broad categories: the questionnaire and the interview. Questionnaires are usually paper-and-pencil instruments that the respondent completes Interviews remind conversations Sometimes, it's hard to tell the difference between a questionnaire and an interview because you will see questionnaires with open-ended questions (although they do tend to be shorter than in interviews) and there will often be a series of closed-ended questions asked in an interview

Questionnaires: mail survey When most people think of questionnaires, they think of the mail survey There are many advantages to mail surveys They are relatively inexpensive to administer You can send the exact same instrument to a wide number of people They allow the respondent to fill it out at their own convenience

Mail survey But there are some disadvantages as well Response rates from mail surveys are often very low And, mail questionnaires are not the best vehicles for asking for detailed written responses

Group administered questionnaire A sample of respondents is brought together and asked to respond to a structured sequence of questions Traditionally, questionnaires were administered in group settings for convenience High response rate If the respondents have questions about the meaning of a question they can ask for clarification

Focus group… What's the difference between a group administered questionnaire and a group interview or focus group? In the group interview or focus group, the interviewer facilitates the session People work as a group, listening to each other's comments and answering the questions Someone takes notes for the entire group -- people don't complete an interview individually.

Household drop-off survey A researcher goes to the respondent's home and hands the respondent the instrument In some cases, the respondent is asked to mail it back or the interview returns to pick it up Like the mail survey, the respondent can work on the instrument in private, when it's convenient Like the group administered questionnaire, the interviewer makes personal contact with the respondent -- they don't just send an impersonal survey instrument And, the respondent can ask questions about the study and get clarification on what is to be done Generally, this would be expected to increase the percent of people who are willing to respond.

Interviews Interviews are a far more personal form of research than questionnaires In the personal interview, the interviewer works directly with the respondent Unlike with mail surveys, the interviewer has the opportunity to probe or ask follow-up questions And, interviews are generally easier for the respondent, especially if what is sought is opinions or impressions

Telephone interview Enable a researcher to gather information rapidly Allow for some personal contact between the interviewer and the respondent Allow the interviewer to ask follow-up questions Many people don't have publicly-listed telephone numbers. Some don't have telephones People often don't like the intrusion of a call to their homes Telephone interviews have to be relatively short or people will feel imposed upon.

Survey Survey is a series of questions asked of a number of people Face-to-face (interview) Self-administered (mail, web) Phone Provide different answers

Reasons for differences Social desirability and sensitive questions Acquiescence (culturally-based tendency to agree)

Social desirability During fall semester, how often did you use drugs other than alcohol or marijuana, for example cocaine, speed, LSD, etc.? No Response Never Less than once a month but at least once in the past year One to three times a month One to two times a week More than twice a week

Sensitive question Since the beginning of the school year, have you been forced or coerced to engage in unwanted sexual activity by...someone you someone you know very well Never Once or twice Once a month Once a week 2-3 times a week Every day

Acquiescence Please indicate the extent to which you agree that lying is okay if keeps friends out of trouble. Strongly agree Agree Neither agree nor disagree Disagree Strongly disagree

Surveys The goal of writing questions for self-administered questionnaire is to develop a query that every potential respondent will interpret in the same way, be able to respond accurately, and be willing to answer

Types Of Questions Dichotomous Questions -When a question has two possible responses, we consider it dichotomous Surveys often use dichotomous questions that ask for a Yes/No, True/False or Agree/Disagree response There are a variety of ways to lay these questions out on a questionnaire:

Nominal question

Ordinal question

Likert scale

Cumulative or Guttman scale

Filter or Contingency Questions

Response Format The response format is how you collect the answer from the respondent: unstructured response formats structured response formats

Structured formats Structured formats help the respondent to respond more easily and help the researcher to accumulate and summarize responses more efficiently But, they can also constrain the respondent and limit the researcher's ability to understand what the respondent really means There are many different structured response formats, each with its own strengths and weaknesses. We'll review the major ones here.

Fill-In-The-Blank One of the simplest response formats is a blank line. A blank line can be used for a number of different response types. For instance: Please enter your gender: _____

Fill-In-The-Blank NAME: ________________________ NAME:

Check The Answer The respondent places a check next to the response(s). Sometimes, we supply a box that the person can fill in with an 'X' (which is sort of a variation on the check mark.

Check The Answer Please check if you have the following item on the computer you use most: modem Printer CD-ROM Drive Joystick scanner

Circle The Answer Sometimes the respondent is asked to circle an item to indicate their response. Usually we are asking them to circle a number. For instance, we might have the following:

Unstructured Response Formats While there is a wide variety of structured response formats, there are relatively few unstructured ones What is an unstructured response format? Generally, it's written text. These can vary from short comment boxes to the transcript of an interview.

Unstructured Response Formats In almost every short questionnaire, there's one or more short text field questions. One of the most frequent goes something like this: Please add any other comments:

Writing Questions One methodologist described the question-writing task as similar to a driving in freeway traffic while drinking a cup of hot coffee and answering an emergency call on his cell phone

How answer categories might impact results of the study How many hours per day do you typically study? Less than 0.5 hours 0.6 -1 hour 1.1- 2 hours 2.1 – 2.5 hours More than 2.5 hours How many hours per day do you typically study? Less than 2.5 hours 2.6 – 3 hours 3.1 – 4 hours 4.1 – 4.5 hours More than 4.5 hours

Reported Hours of studying Mail Telephone Reported Hours/Day 2.5 hours or more Less than 2.5 hours Low version High version 23% 77% 58% 42% 69% 70% 30% 31%

Rules for writing questions Rule 1: Each respondent has to have a chance to answer a question

What is wrong with the following question? If you fixed dinner at home last night, did you eat meat as part of that meal? Yes No

Rule 2.1 Do not use ambiguous questions

What is wrong with the following question? “Have you had an opportunity to make use of the suggestions for handling obscene calls from the phone company?”

Revision “Has the phone company offered suggestions for handling obscene phone calls? Have you made use of them?”

Is anything wrong with this question? “Do you agree that pornographic videos are harmful to your children and that you must get rid of them?”

Revision “Do you agree or disagree that pornographic videos should be banned?”

Rule 2.1 Do not use ambiguous answer categories

What is wrong with the following question? How often did you attend religious services during the past year? Never Rarely Occasionally Regularly

Revision How often did you attend religious services during the past year? Not at all A few times About once a month Two to three times per month About once a week More than once a week

Rule 3: Use equal numbers of positive and negative categories for scalar questions How satisfied were you with the service you received when you bought air conditioner? Completely satisfied Mostly satisfied Somewhat satisfied Neither satisfied nor dissatisfied Dissatisfied How satisfied were you with the service you received when you bought air conditioner? Completely satisfied Somewhat satisfied Neither satisfied nor dissatisfied Somewhat dissatisfied Dissatisfied

Rule 3.1: avoid overlapping categories What is your age ? Less than 10 10-15 15-20 20-30 30-40 40 and older

Rule 4: Avoid double-barreled questions Should the city build a new swimming pool that includes lanes for swimming laps that is not enclosed for winter use? Yes No

Revision Should the city build a new swimming pool that includes lanes for swimming laps? Yes No Should the city build a new swimming pool that is not enclosed for winter use?

Rule 5 Avoid biased or leading questions: “Do you still beat your spouse?” Yes No

Rule 6:Use appropriate language Avoid complex phrases ( do not show off your vocabulary) “Occupants of this household” vs “people who live here” “Subnational region” vs “area of the country

Rule 7: Avoid jargon and slang “Do you agree or disagree that out county prosecutors regularly kink their cases?”

Rule 8: Write questions in full sentences Sex______________ Revision: What is your sex?_____________

Rule 9: Avoid negative questions “Do you agree or disagree that taverns ought not to be prohibited from opening on non-weekends-not including holidays?”

Sensitive Questions Barton (1958, p. 67) provides humorous examples of many of the techniques that have been used as means of asking threatening questions The Causal Approach: “Do you happen to have murdered your wife?” The “Other People” approach: “Do you know any people who have murdered their wives? How about yourself?”

Sensitive Questions The Numbered Card: “Would you please read off the number on this card which corresponds to what became of your wife?” (Hand card to respondent). Natural Death I killed her Other (What?) (Get the card back from respondent before proceeding)

Elements of visual design In self-administered questionnaires, information is communicated through four distinct languages, each of which may independently influence question meaning Verbal language (the words) Numeric language (numbers) Symbolic language (symbols defined by one’s culture, used to convey meaning independent of any words) Graphical language (variations in the display of words, numbers, and/or symbols)

Eye Movement Patterns Under conditions of visually “equal” information, we tend to follow culturally prescribed patterns

Eye Movement Pattern Equal Information

Eye Movement Patterns When information is visually “unequal” what we see and comprehend is often influenced by visual qualities of the information

Eye Movement Pattern; Unequal Information

Eye Movement Pattern Unequal Information

Gestalt Psychology: Principles of Visual Perception Principle of Proximity: We tend to group together things that are close to one another and more distant from other (similar) objects.

Principle of Proximity: Similar figures in closer proximity will be seen as a group)

Principle of Proximity

Principle of Proximity

Principle of Similarity: Similar figures will be seen as a group Do you see groups?

Principle of Similarity: Similar figures will be seen as a group Do you see groups?

Figure/Ground Variations Can Also Facilitate Grouping It is difficult to switch from one grouping to the other To what extent to you agree or disagree that this sentence is difficult to read? Would you say that you strongly agree, somewhat agree, have no opinion, somewhat disagree or strongly disagree that this sentence is difficult to read. extent to you agree or strongly agree, no opinion, somewhat to read.

Figure and Ground