Survey Research and Other Ways of Asking Questions 1 Chapter 7: Survey Research and Other Ways of Asking Questions

Guidelines for Asking Questions 2 Guidelines for Asking Questions Open-ended – respondent is asked to provide his or her own answer Closed-ended – respondent selects an answer from a list Choices should be exhaustive and mutually exclusive Questions and Statements – (Likert scale)

3 More Guidelines… Make items clear – avoid ambiguous questions; do not ask “double-barreled” questions Short items are best – respondents like to read and answer a question quickly Avoid negative items – leads to misinterpretation Avoid biased items and terms – do not ask questions that encourage a certain answer

Questionnaire Construction 4 Questionnaire Construction General questionnaire format – critical, must be laid out properly Contingency questions – relevant only to some respondents – answered only based on their previous response Matrix questions – same set of answer categories used by multiple questions

Ordering Questions in a Questionnaire 5 Ordering Questions in a Questionnaire Ordering may affect the answers given Estimate the effect of question order Perhaps devise more than one version Begin with most interesting questions End with duller, demographic data This is opposite for in-person interview surveys

Self-Administered Questionnaires 6 Self-Administered Questionnaires Can be home-delivered Researcher delivers questionnaire to home of sample respondent, explains the study, and then comes back later Mailed (sent and returned) survey is most common Researchers must reduce the trouble it takes to return a questionnaire

Warning Mailings, Cover Letters 7 Warning Mailings, Cover Letters Used to increase response rates Warning mailings – “address correction requested” card sent out to determine incorrect addresses and to “warn” residents to expect questionnaire in mail Cover letters – detail why survey is being conducted, why respondent was selected, why is it important to complete questionnaire Include institutional affiliation or sponsorship

Other Aspects of Self-Administered Questionnaires 8 Other Aspects of Self-Administered Questionnaires Monitoring returns – pay close attention to the response rate, assign #s serially Follow-up mailings – nonrespondents can be sent a letter, or a letter and another questionnaire; timing Acceptable response rates – 50%? 60%? 70%? We would rather have a lack of response bias than a high response rate

Computer-Based Self-Administration 9 Computer-Based Self-Administration Via fax, email, web site/page Issues representativeness mixed in with, or mistaken for, spam requires access to Web sampling frame?

In-Person Interview Surveys 10 In-Person Interview Surveys Typically achieve higher response rates than mail surveys (80-85% is considered good) Demeanor and appearance of interviewer should be appropriate; interviewer should be familiar with questionnaire and ask questions precisely When more than one interviewer administers, efforts must be coordinated and controlled Practice interviewing

Telephone Surveys 94% of all households now have telephones 11 Telephone Surveys 94% of all households now have telephones Random-Digit Dialing Obviates unlisted number problem Often results in business, pay phones, fax lines Saves money and time, provides safety to interviewers, more convenient (pajamas!) may be interpreted as bogus sales calls; or the ease of hanging up

Comparison of the Three Methods (mailed, phone, interviews) 12 Comparison of the Three Methods (mailed, phone, interviews) Self-administered questionnaires are generally cheaper, better for sensitive issues than interview surveys Using mail – local and national surveys are same cost Interviews – more appropriate when respondent literacy may be a problem, produce fewer incompletes, achieve higher completion rates Validity low in survey research; reliability high Surveys are also inflexible, superficial in coverage

Specialized Interviewing 13 Specialized Interviewing Two variations: General interview guide – less structured, lists issues, topics, questions you wish to cover; no standardized order Standardized open-ended interview – more structured, specific questions in specific order; useful in case studies, retrieves rich detail in responses

14 Focus Groups 12-15 people brought together to engage in guided group discussion of some topic Members are selected to represent a target population, but cannot make statistical estimates about population Most useful when precise generalization to larger group is not necessary May be used to guide interpretation of questionnaires following survey administration

Should You Do It Yourself? 15 Should You Do It Yourself? Consider start-up costs Finding, training, paying interviewers is time consuming and not cheap, and requires some expertise Mail surveys are less expensive, and can be conducted by 1-2 persons well The method you use depends on your research question