1. RMK Chapter 4: Survey Research l Types of Surveys (Q & I) Types of Surveys (Q & I) Types of Surveys (Q & I) l Types of Questions Types of Questions Types of Questions l Decisions about Question Content Decisions about Question Content Decisions about Question Content l Decisions about Response Format Decisions about Response Format Decisions about Response Format l Decisions about Question Wording Decisions about Question Wording Decisions about Question Wording l Decisions about Question Placement Decisions about Question Placement Decisions about Question Placement l Conducting Interviews Conducting Interviews Conducting Interviews l Advantages / Disadvantages of Methods Advantages / Disadvantages of Methods Advantages / Disadvantages of Methods

3. Types of Surveys Questionnaires and Interviews

4. Questionnaires Group administration Mail administration Household drop off Web-based

5. Interviews Personal Telephone

7. Types of Questions

8. Type of Questions l Unstructured »Open ended / short comment box / transcript of interview / narrative l Structured »Details in following slides

9. Structured Questions Dichotomous ____Yes ____No MaleFemale

10. Nominal Occupational Class: 1 = Truck driver 2 = Lawyer 3 = etc.

11. Structured Questions Ordinal Rank the candidates in order of preference from best to worst... ___ Bob Dole ___ Bill Clinton ___ Newt Gingrich ___ Al Gore

12. Structured Questions Interval Likert Response Scale 12345 stronglydisagreeneutralagreestrongly disagreeagree

13. Structured Questions Interval Semantic Differential verysome-neithersome-very muchwhatwhatmuch interestingboring simplecomplex uncaringcaring usefuluseless etc.

14. Structured Questions Interval Guttman (Cumulative) Scale ___Are you willing to permit immigrants to live in your country? ___Are you willing to permit immigrants to live in your community? ___Are you willing to permit immigrants to live in your neighborhood? ___Would you be willing to have an immigrant live next door to you? ___Would you let your child marry an immigrant?

15. Filter or Contingency Questions Have you ever smoked marijuana? yesno

16. Filter or Contingency Questions Have you ever smoked marijuana? yesno If yes, about how many times have you smoked marijuana? once 2 to 5 times 6 to 10 times 11 to 20 times more than 20 times

17. Filter or Contingency Questions l Try to avoid having more than three levels (two jumps) for any question. l If only two levels, use graphic to jump (for example, arrow and box). l If all responses can’t fit on the page, then jump to a new page. l Electronic advantages (web pages) –Can be programs to move forward in a specific way based on response.

19. Decisions About Question Content

20. l Example –Do you need the age of each child or just the number of children under 16? l Example –Do you need to ask income or can you estimate? Is the question necessary / useful?

21. Decisions About Question Content l Examples: Bad Question –What are your feelings towards African- Americans and Hispanic-Americans? –What do you think of proposed changes in benefits and hours? Are several questions needed? Don’t combine two issues in one...

22. Decisions About Question Content l If you ask about earnings, might not mention all income l If ask if they’re in favor of public TV, ask for whom (might not mention it’s because of Sesame Street for the kids) Are several questions needed? Did you get all the info desired...

23. Decisions About Question Content l If you ask research participants about attitudes towards a specific religion, can you interpret the results without finding out about their attitudes towards religion? Are several questions needed? Do you need more context to interpret answer...

24. Decisions About Question Content l If they say they support public TV … –do they watch it? –would they be willing to have their tax dollars spent on it? Are several questions needed? Need more info to determine the intensity...

25. Decisions About Question Content l Examples –‘Do you think Dean Rusk acted correctly in the Bay of Pigs crisis?’ -- can’t say if they don’t know who he is. –In TV surveys, ‘regular’ watchers can be asked about program content, ‘never watch’ should not be asked. Do respondents have the needed info?

26. Decisions About Question Content l How well did you like the book? Versus... l Did you recommend the book to others? l Did you look for other books by that author? Does question need to be more specific?

27. Decisions About Question Content l For instance, seasonal specificity. –If you ask what they usually watch and it’s summer, you’ll get a different answer than in the winter. Is question sufficiently general?

28. Decisions About Question Content l Ask about the benefits of a tax cut without asking about the disadvantages l Ask about the disadvantages of eliminating welfare without asking about the benefits Is question biased or loaded?

29. Decisions About Question Content l To find out age or income, give brackets. –[18-22], [23-27], etc. l To find out about charitable contributions, ask how much “people you know” typically give. l To find out what magazines they read, offer to buy used ones. Will respondent answer truthfully?

31. Decisions About the Response Format

32. Should the Response Format be...  Single option vs. multi-option  Structured  Dichotomous (two possibilities)  Multiple choice  Fill-in-the-blank  Scale - interval (e.g., Likert response format, semantic differential, Guttmann scale)  Checklist – see next slide…  Unstructured  Open ended / free answer / narrative

33. If Checklist...  are all alternatives covered?  is it of reasonable length?  is the wording impartial?  is the form of the response easy, uniform?  If its possible that there may be some other alternatives, provide “other” option and allow open ended response.

35. Decisions About Question Wording

36. l Questions about ‘nationality’ or ‘marital status are too vague. l Questions about ‘mass media’ are too general. l What kind of headache remedy do you use? –Want brand name? –Want type? (Aspirin, pill, or capsule?) Can the question be misunderstood?

37. Decisions About Question Wording l If you ask what social class someone’s in, you assume that they know what social class is and that they think of themselves as being in one. l Check assumption in a previous question. What assumptions does question make?

38. Decisions About Question Wording l Do you think Congress will cut taxes? l Do you think Congress will successfully resist tax cuts? l Neither of these specifies a time frame. Is the time frame specified?

39. Decisions About Question Wording l Are working conditions satisfactory or not satisfactory in the plant where you work? »Objective l Do you feel that working conditions are satisfactory or not satisfactory in the plant where you work? »Feeling l Are you personally satisfied with working conditions in the plant where you work? »Personal satisfaction How personal is the wording? Ensure Appropriate Context for the study

40. Decisions About Question Wording l How did you feel about being in the war? - May be too direct l How well did the equipment hold up in the field? l How well were new recruits trained? Is the wording too direct?

41. Decisions About Question Wording l Does question contain difficult or unclear terminology? l Does the question make each alternative explicit? l Is the wording objectionable? l Is the wording loaded or slanted? Some Additional Issues...

43. Question Placement and Sequence

44. Decisions About Placement l Is the answer influenced by prior questions? l Does question come too early or too late to arouse interest? l Does the question receive sufficient attention?

45. The Opening Questions l These are particularly true in the case of an interview. –Should be easy to answer –Should not be sensitive material –Should get the respondent “rolling” l In a survey, consider –putting basic demographic questions towards the end of a survey because these are easy to complete and thus it may overcome “survey fatigue.”

46. Sensitive Questions l Only after trust is developed l Should make sense in that section of the survey (not “out of left field”) l Precede with warm-up questions

47. A Checklist of Considerations (slide 1) 4 Start with easy, nonthreatening questions. 4 Put more difficult, threatening questions near end. 4 Never start mail survey with an open- ended question. 4 Put demographics at end (unless needed to screen).

48. A Checklist of Considerations (slide 2) 4 Avoid demographics at beginning. 4 For historical demographics, follow chronological order. 4 Ask about one topic at a time. 4 When switching topics, use a transition. 4 Reduce response set. 4 For filter or contingency questions, make a flowchart.

49. Important Rule Keep questionnaire as short as possible!

51. Interviews

52. The Role of the Interviewer l Locate and enlist cooperation of respondents. l Motivate respondents to do good job. l Clarify any confusion/concerns. l Observe quality of responses. l Conduct a good interview.

53. Training the Interviewers (slide 1) Ô Describe the entire study. Ô State who is sponsor of research. Ô Teach enough about survey research (should respect method and be motivated). Ô Explain the sampling logic and process. Ô Explain interviewer bias. Ô “Walk through” interview.

54. Training the Interviewers (slide 2) Ô Explain selection procedures, including –Reading maps –Identifying households –Identify respondents Ô Rehearse interview Ô Explain supervision Ô Explain scheduling

55. The Interviewer’s Kit l Official-looking 3-ring notebook l Maps l Sufficient copies of survey l Official identification l A cover letter l A phone number to call

56. The Interview â Opening remarks â Asking the questions â Obtaining adequate answers â Recording the response â Concluding the interview More Details on Following Slides

57. Opening Remarks l Gaining entry -- appearance, confidence, non-threatening l Doorstep technique -- brief, suggest don’t ask l Introduction -- name, organization, credentials l Explaining the study -- general, stress confidentiality

58. Asking the Questions l Use questionnaire carefully, but informally. l Ask questions exactly as written l Follow the order given. l Ask every question. l Don’t finish sentences. l Don’t leave any questions blank.

59. Obtaining Adequate Responses - The Probe l Silent probe - a pause l Overt encouragement l Elaboration –How do you mean? Anything else? l Ask for clarification l Repetition

60. Recording the Response l Record responses immediately l Include all probes (write them in) l Use abbreviations where possible (for example –R = respondent; DK = don’t know)

61. Concluding the Interview l Thank respondent. l Don’t be brusque or hasty. l Immediately after leaving -- write down any notes about how the interview went.

63. Advantages and Disadvantages of Survey Methods (general assessment)

64. Visual Presentations Possible? Questionnaire GroupMailDrop-Off Interview PersonalPhone

65. Questionnaire GroupMailDrop-Off YesYesYesNoYes Interview PersonalPhone

66. Long Response Categories Possible? Questionnaire GroupMailDrop-Off Interview PersonalPhone

67. Questionnaire GroupMailDrop-Off YesYes?NoYes Interview PersonalPhone

68. Privacy? Questionnaire GroupMailDrop-Off Interview PersonalPhone

69. Questionnaire GroupMailDrop-Off YesNoNo?No Interview PersonalPhone

70. Flexible? Questionnaire GroupMailDrop-Off Interview PersonalPhone

71. Questionnaire GroupMailDrop-Off NoNoYesYesNo Interview PersonalPhone

72. Open-ended Questions Feasible? Questionnaire GroupMailDrop-Off Interview PersonalPhone

73. Questionnaire GroupMailDrop-Off NoNoYesYesNo Interview PersonalPhone

74. Reading and Writing Needed? Questionnaire GroupMailDrop-Off Interview PersonalPhone

75. Reading & Writing Needed? Questionnaire GroupMailDrop-Off YesYesNoNo? Interview PersonalPhone

76. Can You Judge Quality of Response? Questionnaire GroupMailDrop-Off Interview PersonalPhone

77. Questionnaire GroupMailDrop-Off No?Yes?Yes Interview PersonalPhone

78. High Response Rates? Questionnaire GroupMailDrop-Off Interview PersonalPhone

79. Questionnaire GroupMailDrop-Off NoYesYesNoYes Interview PersonalPhone

80. Explain Study in Person? Questionnaire GroupMailDrop-Off Interview PersonalPhone

81. Questionnaire GroupMailDrop-Off NoYesYes?Yes Interview PersonalPhone

82. Low Cost? Questionnaire GroupMailDrop-Off Interview PersonalPhone

83. Questionnaire GroupMailDrop-Off YesNoNoNoYes Interview PersonalPhone

84. Staff and Facilities Low? Questionnaire GroupMailDrop-Off Interview PersonalPhone

85. Questionnaire GroupMailDrop-Off YesNoNoNoYes Interview PersonalPhone

86. Access to Dispersed Samples? Questionnaire GroupMailDrop-Off Interview PersonalPhone

87. Questionnaire GroupMailDrop-Off YesNoNoNoNo Interview PersonalPhone

88. Respondent Has Time to Formulate Answers? Questionnaire GroupMailDrop-Off Interview PersonalPhone

89. Questionnaire GroupMailDrop-Off YesYesNoNoNo Interview PersonalPhone

90. Personal Contact? Questionnaire GroupMailDrop-Off Interview PersonalPhone

91. Questionnaire GroupMailDrop-Off NoYesYesNoYes Interview PersonalPhone

92. Long Survey Feasible? Questionnaire GroupMailDrop-Off Interview PersonalPhone

93. Questionnaire GroupMailDrop-Off NoNoYesNoNo Interview PersonalPhone

94. Quick Turnaround? Questionnaire GroupMailDrop-Off Interview PersonalPhone

95. Questionnaire GroupMailDrop-Off YesNoNoYesNo Interview PersonalPhone