SURVEY RESEARCH BUSN 364 – Week 12 Özge Can

Survey Research  Asks a large number of people (respondents) the same questions about their beliefs, opinions, characteristics and past and present behaviors  Standard questionnaire forms  Self-reported beliefs and behaviors  Produce information that is statistical in nature => quantitative  Asks many questions at once, thereby measuring many variables => one can test multiple hypotheses  Alternative explanations are statistically controlled

What Is Asked in a Survey?  Behavior  How frequently do you brush your teeth?  Attitudes/beliefs/opinions  What is the biggest problem facing the youth today?  Characteristics  What is your age? Are you married or not?

What Is Asked in a Survey?  Expectations  Do you plan to buy a car in the next 12 months?  Self-Classifications  Which social class would you put your family into?  Knowledge  Who was elected as the city mayor in the last election?

Steps in Conducting a Survey (1):

Steps in Conducting a Survey (2):

Principles of Good Question Writing  Two key principles guide writing good survey questions:  Avoid any possible confusion  Keep the respondent’s perspective in mind  We want a survey to provide a valid and reliable measure:  Respondents should easily understand the question’s meaning as you intended (clear, relevant and meaningful) and answer completely and honestly

Principles of Good Question Writing 1. Avoid jargon, slang, abbreviations or acronyms  When surveying general public, use the language of popular culture 2. Avoid ambiguity, confusion and vagueness  “What is your income?” “Do you jog regularly?” 3. Avoid emotional language and prestige bias  “What do you think about paying murderous terrorists who threaten to steal the freedom of peace-loving people?”

Principles of Good Question Writing 4. Avoid double-barreled questions  “Does your employer offer pension and health insurance benefits?” 5. Avoid leading questions  “You don’t smoke, do you?” 6. Avoid questions beyond respondents’ capabilities  “How did you feel about your brother when you were 6 years-old?”  “How much was your electricty bill last year?”

Principles of Good Question Writing 7. Avoid asking about distant future intentions  “Suppose 8-years later you become a manager, how would you behave to your employees?” 8. Avoid double negatives  “I ain’t got no job”  “Do you agree or disagree that students should not be required to take a comprehensive exam to graduate?” 9. Avoid overlapping or unbalanced response categories  Make response choices mutually exclusive, exhaustive and balanced.

Getting Honest Answers  Sensitive Topics => Illness and disability (mental health problems, cancer), illegal or deviant behavior (drug or alcohol use, law violations), financial status (income, debts, occupation)  Threatens people’s presentation of themselves; their positive image. We should particularly cautious about these questions and the results. To increase honest answering: 1. Create comfort and trust 2. Use enhanced phrasing 3. Establish a desensitizing context 4. Use anonymous questioning methods

Getting Honest Answers  Social Desirability Bias => occurs when respondents distort answers to conform to popular social norms. They give a socially acceptable answer rather than an honest answer  Ex: people tend to overstate being highly cultured, being outgoing and fun, giving money to charity, having a good marriage, loving their children  One way to reduce it: phrase questions in ways that make norm violation appear less objectifiable and give respondents “face-saving” alternatives

Open-Ended vs. Close-Ended Questions I. Open-ended question:  A type of survey inquiry that allows respondents freedom to offer any answer they wish to the question II. Close-ended question:  A type of survey inquiry in which respondents must choose from a fixed set of answers  Partially open questions => a set of fixed choices with a final open choice of “other”

Open-Ended Questions

Close-Ended Questions

Valid Responses  Three types of answers from respondents yield invalid responses:  Swayed opinion => falsely overstating a position  False positive => selecting an attitude position but lacking any knowledge on the issue  False negative => respondent refuses to answer the question when he/she actually has information or an opinion  Neutral positions: Should we offer respondents who lack knowledge or have no position a neutral or “no opinion” choice?

Valid Responses  Response Set (Reponse Bias) => tendency of respondents to agree with every question in a series rather than carefully thinking through one’s answer to each  To avoid it: include different question types; change the sequence of response categories

Response Set Bias :

Valid Responses  To get valid responses:  Choose the appropriate response style for the question: agreement scales, rankings or ratings?  Present the alternatives fairly  Attach numbers to a response scale to assist respondents and give them a clue for understanding  Use visual presentations, colors, symbols and pictures  Appropriate question format and questionnaire design

Valid Responses

Questionnaire Design Issues  Length of survey or questionnaire  No absolute proper length; it depends on the survey format (e.g. mail, internet) and respondent characteristics  Short questionnaires are appropriate for the general population  Question order and sequence  You should sequence questions to minimize respondent discomfort and confusion  After an introduction explaining the survey, make opening questions easy to answer and pleasent.  Order effect => answers to earlier questions can influence later ones

Questionnaire Design Issues  Layout and format  Questionnaire layout => Appearence of the questionnaire form should be clear, neat and easy to follow; sheets, fonts, envelope, cover letter...  Question format => Should respondents circle responses, check boxes, fill in dots or write in a blank?  Nonresponse  The failure to get a valid response from every sampled respondent = nonresponse bias  Can be a major problem if a high proportion of the sample does not respond; lowers generalizability

Ways to Increase Response Rate Address to specific individual Include a cover letter – Request cooperation – Guarantee confidentiality – Explain purpose – Provide researcher contact information Include postage-paid, addressed return envelope Easy to follow questionnaire Send follow-up reminders Avoid conducting study during busy holidays Back page for general comments Advertise legitimate sponsors (university, gov’t agency, etc.) Small monetary incentives

Types of Surveys  Mail surveys  Telephone surveys  Face-to-face surveys  Internet/web surveys

Types of Surveys  Features to consider when choosing a particular survey type:  Administrative issues => response rate, cost, speed, length  Researcher control on questions => explaining the questions, flexible formats  Success with different questions => open-ended questions, complex or sensitive questions  Sources of bias => social desirability, interviwer bias, respondent’s reading skills

Constructing a Survey – Some Resources Many links for carrying out survey research: Information on designing surveys: eas/Soc_survey.shtml Online survey websites:  SurveyMonkey (  PsychSurveys (