© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. Designing Surveys and Data Collection Instruments CHAPTER 6 © 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.

Q. 1. Define Questionnaire.

© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. Function and Importance of Questionnaires A questionnaire is a formalized method for collecting data from respondents that measures:  past purchase and usage behavior  attitudes and opinions, intentions, awareness and knowledge, ownership  variety of respondent characteristics A structured technique for data collection that consists of a series of questions, written or verbal, that a respondent answers. To avoid measurement error, questions must pose realistic alternatives or trade-offs.

© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. Q. 2. What constitutes the five sections of a questionnaire?

© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. Questionnaire Components Components of a questionnaire:  identification data  respondent’s name  address  phone number  request for cooperation  instructions  information sought – (longest portion of the questionnaire)  classification data – ( characteristics of the respondent, primarily “geodemographic” data)

© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. Q. 3. What are the broad, interrelated guidelines researchers follow when constructing questionnaires?

© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. Questionnaire Design Best practices in questionnaire design:  review preliminary considerations  decide question content  decide response format  decide question wording  decide question sequence  decide physical characteristics  pre-test and revise  make final draft

© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. Preliminary Considerations  review issues already decided by research design:  detailed listing of information needs  type of research design  sources of data  definition of the target population  detailed sampling plan  specification of scales and communication media  visualization of the research findings  link between data-collection and information needs  Each question should relate to a specific information need. Questionnaire Design (continued)

© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. Q. 4. What influences the content of the questions in the questionnaire?

© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. Consider ways question will not be answered accurately or at all: 1.respondent unable to provide the data 2.respondent uninformed 3.respondent misremembers 4.respondent forgetful – can attempt to stimulate memory through:  unaided recall  aided recall  recognition 1. Ability to answer accurately

© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. Ways question will not be answered accurately or at all (cont.): respondent unwilling to respond accurately – can compensate with:  counterbiasing statement  indirect statement – refer to “other people”  labeled response categories  randomized response technique 2. Willingness to respond accurately

© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. Q. 5. What types of questions are used in the format of the questionnaire?

© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. Decide on Response Format 1.open-ended questions – free-response, usually in respondent's own words 2.multiple-response questions – require choice from an explicit list of options 3.dichotomous questions – offer choice of two responses and often a third neutral alternative

© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. Q. 6. What are the advantages and disadvantages of open-ended questions?

© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. Open-ended questions Advantages:  allow general attitudes to be expressed, which can aid in interpreting later questions  establish rapport and gain respondent’s cooperation  exert minimal influence on subsequent responses  can provide researcher with insights and explanations  quotations from open-ended questions can add a sense of realism to research findings  useful for exploratory research purposes

© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. Open-ended questions Disadvantages of open-ended questions:  high potential for interviewer bias  time and cost to code the responses  implicit extra weight given to respondents who are more articulate  higher effort and time commitment required of respondents

© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. Q. 7. What are the advantages and disadvantages of multiple-response questions?

© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. Multiple-response questions Advantages:  reduce interviewer bias (in interpreting verbal responses)  reduce effort respondents must put into replying  reduce cost and time associated with data processing  easy and quick to administer  can limit responses to a set of interest Disadvantages:  design of questions requires considerable time and cost

© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. Decide on Response Format (continued) Issues in multiple-response question design:  number of alternatives  collectively exhaustive  usually mutually exclusive  the alternatives included  position bias – bias toward the central position of a number range or toward first idea on a list. Compensate with:  alternate order of alternatives  split design, where half of respondents see one scale, and the other half a different one

© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. Q. 8. What are the advantages and disadvantages of dichotomous questions?

© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. Decide on Response Format (continued) Advantages:  all the advantages of multiple-response questions  quick and easy to administer  respondents understand  little chance of interviewer bias  responses easy to code, process, analyze and report  binary responses can be analyzed using powerful statistical methods

© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. Decide on Response Format (continued) Disadvantages of dichotomous questions:  may miss many grades of feeling  can lead to substantial measurement error  especially susceptible to error resulting from positive or negative posture of the question Main issue in dichotomous question design: whether to include a neutral response alternative

© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. Coding Verbal Data from Open-Ended Questions  Step 1: Two people independently read through all verbal responses.  Step 2: Each reader forms a list of key words useful for categorization.  Step 3: Together with third person who has not taken part thus far, readers merge their lists.  Step 4: A fourth and fifth person sort the responses into the given categories.  Step 5: Together with a sixth person, sorters agree to final categorization scheme.

© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. Q. 9. What are guidelines for researchers in constructing appropriate question wordings?

© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. Decide on Question Wording Guidelines for question wording:  use simple language  use unambiguous words  does the word truly convey what the researchers intended?  can respondents extrapolate any alternative meaning?  if so, does context help make the intended meaning clear?  is there any word with similar pronunciation or spelling that could be confused with it?  could we use a simpler word or phrase instead?

© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. Decide on Question Wording (continued) Guidelines for question wording (cont.):  avoid leading questions  avoid biasing questions  avoid implicit alternatives  avoid implicit assumptions  avoid estimates  avoid double-barreled questions  consider frame of reference

© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. Q. 10. What guidelines are needed in sequencing questions in a questionnaire?

© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. Decide on Question Sequence Guidelines for question sequence:  sequence questions to retain respondent interest without introducing bias  use intriguing, readily understood opening question  ask general questions first  place uninteresting and difficult questions late in the sequence  arrange questions in logical order

© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. Computer-Aided Questionnaire Design Special design programs for questionnaires  provide pre-defined question formats for:  attitude scales  paired comparisons  demographics  allow user to:  specify question switching and skipping based on previous answers  randomize the order of options or questions  reverse positive and negative scale directions  custom-tailor standard question formats

© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. Observational Forms Observational forms  make explicit the types of observations to be made  make explicit how observations are to be measured  are designed logically, based on the listing of information needs:  who is to be observed?  what is to be observed?  when is observation to be made?  where should observations be made?

© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. International Questionnaire Design  translate language or regional variation  back-translate and test translation  give special attention to answer categories to compensate for cultural preferences  if necessary, ask questions different ways  if necessary, change format to match a different interviewing mode  more processing time (typically several months)  higher costs (typically 2-5 times)

© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. Self-Reports: How the Questions Shape the Answers Issues with question design in practice:  question comprehension  response alternatives clarify what the question is asking  meaningful answers require inferences about question  frequency scales  range of frequency options affect inference made  numerical rating scales  respondents infer different meanings from bipolar (-5 to +5) and unipolar (0 to 10) scales  behavioral reports  autobiographical memory is poor, especially for frequent behaviors and experiences

© 2013 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part. Self-Reports: How the Questions Shape the Answers (continued)  estimation strategies  estimation strategies are highly dependent on context, particularly of scale ranges  subsequent judgments  earlier questions can affect judgments during later questions  attitude reports  when asked about attitudes, respondents often form judgments on the spot, strongly affected by earlier questions Standard pre-tests won’t catch cognitive ambiguities. Cognitive pre-test is needed.