Survey Research Pierre-Auguste Renoir: Luncheon of the Boating Party, 1881 [D]D

Topics Appropriate for Survey Research 1.Data Collection from Large Numbers Survey research is a relatively inexpensive procedure for collecting information from a large number of elements. They are efficient means of learning about attitudes, intentions to act. Unfortunately, they are overused by marketing and political organizations, to the point where many persons refuse to respond to any type of survey.

Issues Related to Survey Research 1.Survey Response Rate When survey response rates are low then questions arise about how well the results can be generalized to the population. What were the opinions of those elements who did not respond? Various types of extrapolation procedures can be used to estimate the responses of those who did not complete the questionnaire, but no procedure is widely accepted by the community of scholars.

Issues Related to Survey Research 1.Survey Response Rate (Continued) A large volume of research addresses the issue of how to improve survey response rates. Key features that affect response rates: The topic of the survey, sponsorship, incentives, survey mode, time/day of contact, question wording, question ordering, formatting of the questionnaire, follow-ups, interview style, interviewer style, envelope style, cover letter, time to complete.

Issues Related to Survey Research 1.Survey Response Rate (Continued) The survey response rate is calculated as: Number of completed questionnaires (Number of attempted contacts - Unavailable for contact) For university-sponsored research, survey response rates of 50% are acceptable, 60% is preferred, 70% is very good.

Issues Related to Survey Research 2.Item Response Rate The item response rate is the percentage of responses to an item among all returned and completed questionnaires. Questionnaires might be returned and otherwise complete except for missing responses to certain questions. Example: Responses to questions about income can receive low response rates. Key features that affect item response rates: Topic, question wording, question ordering, questionnaire formatting, interviewer style.

Guidelines for Asking Questions 1.Choose Appropriate Question Forms Open Ended Questions: Respondents are asked to give information in their own words. Closed-Ended Questions: Respondents are asked to state which response most closely represents their answers to questions. 2.Respondents Must be Competent to Answer Typically, surveys are administered to adults. Sometimes, screening questions are needed to insure that the respondent is qualified to answer (i.e., “Do you live in the city limits of Sappville?”).

Guidelines for Asking Questions 3.Respondents Must Be Willing to Answer Respondents must consent to complete the survey and must be competent and legally able to give their consent. 4.Questions Should be Relevant Questions should apply to most respondents. Questions should address the main topic of the survey. 5.Short Items are Best Persons are more likely to understand and respond to simple, short questions.

Guidelines for Asking Questions 6.The First Question Should directly address the main topic of the survey, even if the survey covers sensitive topics (e.g., sex, religiosity, employment). Attempts to “warm up” the subject with background questions or ones not directly related to the topic will give the impression of being evasive, coy, misleading in purpose. Should apply to all potential respondents. Should be short and easy to answer.

Guidelines for Asking Questions 7.Question Wording See the: Question Wording website for detailed directions and examples of how to avoid common mistakes in writing survey questions.Question Wording

General Questionnaire Format 1.Formats for Respondents The questionnaire should present an easy to understand format for respondents. It should be clear about how to respond. The questionnaire should have a professional appearance. Money spent on professional formatting and graphics is money well spent. The response options should tend to flow down the page to give the sense of completing the questionnaire quickly.

General Questionnaire Format 2.Contingency Questions These questions screen respondents for characteristics and then direct them to the next appropriate question on the survey. 3.Matrix Questions A lead in question is followed by a series of statements that complete the question. Example: “How much do you trust... 1.Federal legislators. 2.State legislators. 3.Local legislators.

General Questionnaire Format 4.Ordering Items in a Questionnaire The ordering should flow logically from one topic to the next. Question ordering affects responses. It is difficult to avoid this bias altogether. 5.Questionnaire Instructions A self-administered questionnaire will need to be especially easy to understand. Short introductions help the respondent grasp the intent of the questions that follow. Providing opportunities to give additional information helps gain validity and responses.

General Questionnaire Format 6.Pretesting Questionnaire Pretesting helps uncover many flaws types of flaws that can occur when writing a questionnaire. Pretesting will reveal the approximate time it takes to complete the questionnaire.

The Mailed Survey 1.Introduction Questionnaires are mailed to potential respondents and returned by mail. Business reply envelopes can be used for the return of the questionnaire. The questionnaire is accompanied by a cover letter. Sometimes, incentives are used to increase the response rate.

The Mailed Survey 2.Comparison with Other Modes  Least expensive of traditional surveys.  Best mode for collecting sensitive, extensive, or complex information.  Must have strong content validity of items.  Most time to gather data.  Requires entry of data after collection.  Does not allow for clarification or probing.  Does not allow for editing questions.  Must have very clear directions.  No control over who completes the questionnaire.

The Mailed Survey 3.The Total Design Method The Total Design Method, developed by Don Dillman, has been very effective in improving survey and item response rates. The TDM includes a wide range of guidelines for questionnaire development and survey administration. One key feature of the TDM is the suggested guideline for administering follow-up questionnaires in a mailed survey.

The Mailed Survey 3.The Total Design Method (Continued) The TDM approach to follow-ups: Day 1: Mail questionnaire and cover letter. Day 7: Mail a reminder postcard to all elements. Day 21: Mail a replacement questionnaire and cover letter to those elements who have not yet replied. Day 49:Mail a replacement questionnaire by certified mail and cover letter to those elements who have not yet replied.

The Mailed Survey 3.The Total Design Method (Continued) The TDM approach to follow-ups increases the cost of conducting a survey. Be prepared to hear complaints from elements, especially after the fourth mailing (See: Angry Letter). Instructor’s Note: I feel justified in using the TDM approach to follow-ups when I am conducting research with taxpayer’s money because I think that elements have a responsibility to reply on behalf of other members of the society.

The Telephone Survey 1.Introduction The preferred method for marketing research and political polling. The direct contact and rapid data acquisition of a telephone survey are well suited to collection of information on a few, easy to understand items. Telephone surveying is not restricted by the “Do Not Call” registry.

The Telephone Survey 2.Comparison with Other Modes  Best mode for collecting a few, easy to answer questions in a short time frame.  Does not require entry of data after collection (Computer Assisted Telephone Interviewing).  Allows for clarification or probing.  Allows for editing questions.  More expensive than a mailed survey.  Questions cannot be overly complicated and must be answered with knowledge at hand.  Requires trained interviewers.

The Personal Interview 1.Introduction There is no substitute for a personal interview when the research requires much in-depth information from elements. Personal interviewing is an excellent mode for exploratory research. Personal interviewing allows the researcher to collect additional information not included in the interview itself: body language, setting, condition of the home, nuances of meaning, personal contact with hard-to-reach elements.

The Personal Interview 2.Comparison with Other Modes  Best mode for collecting in-depth responses.  Can collect information other than what is asked on the questionnaire.  Allows for clarification or probing.  Allows for editing questions.  Allows for unstandardized interviewing.  The only reasonable procedure for collecting data from some elements.  Most expensive type of survey.  Requires trained interviewers.  Data entry after collection might be required.

The Internet Survey 1.Introduction This method is gaining in popularity as more persons have home computers and access to the internet. Internet surveys can combine some of the advantages of mailed and telephone surveys. The quality and affordability of software packages is improving rapidly.

The Internet Survey 2.Comparison with Other Modes  Even less expensive than mailed surveys.  Can ask many questions.  Can ask detailed information.  Respondents can complete the survey at their own pace and at a time convenient to them.  Allows for randomization of questions by respondent.  Immediate data entry.  Still unfamiliar to many respondents.  Not a good way to contact older or less affluent adults.

New Technologies 1.Introduction CAPI: Computer assisted personal interviews. Computer is used to collect data in person. CASI: Computer assisted self-interviews. Researcher brings the computer to the person’s location. The person completes the questionnaire on the computer. CASQ: Computer self-administered questionnaire. The person completes the questionnaire online.

New Technologies 1.Introduction (Continued) TDE: Touchtone data entry. The person calls telephone number and responds to a computerized interviewer with entries on the telephone key pad. VR: Voice recognition. The person calls a telephone number and responds by voice. WWW Surveys: Person enters or links to a URL and responds to questions on a web-based questionnaire.

New Technologies 2.Comparison with Other Modes  Can be very inexpensive.  Subjects respond when convenient.  Data seem as valid as traditional modes.  Do not need to train interviewers.  Becoming more accepted by the public.  Ideal for some audiences.  No need for data entry after collection.  Concerns about representativeness.  Requires specialized software.

Secondary Analysis 1.Introduction Analysis of data from existing surveys. Typically, this term refers to analysis of large- scale, government sponsored surveys. Various censuses of the population. National Longitudinal Survey National Survey of Families and Households General Social Survey National Health and Nutrition Examination Survey Many others....

Secondary Analysis 2.Comparison with Other Modes  Large number of subjects.  Longitudinal studies.  Data available to many researchers.  Expert data collection procedures.  Take questions as they come.  Questions over time do not always match.  Can be slow to adopt new theories and concepts.