1. 1 3. Drafting Each Module

2. What we typically think of as “Designing a Survey Instrument” Going question by question to ensure it meets the research objective Deciding question format (A) Specify wording (B) Detailing (C) Translation (D) 2

3. 3. Drafting Each Module Questions should flow logically from one to the next. Within each module, questions should flow: from the more general to the more specific; from the factual and behavioral to the attitudinal; from the least sensitive to the most sensitive. 3

4. 3. Drafting Each Module A. Deciding Question Formats 1.Open vs. Closed 2.Single vs. Multiple Response 3.Rating 4.Ranking 4

5. 3. Drafting Each Module A. Deciding Question Formats 1. Open vs. Closed Questions “Why don’t you save in a formal bank?” gives precise judgments But….if you are concerned, as most surveys are, in summarizing replies to produce picture of population, better approach is probably a closed ended question: Do you not save in a formal bank because? (prompt each response) 5

6. 3. Drafting Each Module A. Deciding Question Formats 2. Single vs. Multiple Response Do you want respondent to give single or multiple answer? Example of Multiple Response: 6

7. 3. Drafting Each Module A. Deciding Question Formats 2. Single vs. Multiple Response Example of single response: 7

8. 3. Drafting Each Module A. Deciding Question Formats 3. Likert (“Rating”) Scales: Another Popular Approach 8

9. 3. Drafting Each Module A. Deciding Question Formats 3. Likert (“Rating”) Scales: Another Popular Approach Response categories such as strongly agree, agree, neutral, disagree, and strongly disagree. Advantage: good for opinions and attitudes. Disadvantage: Not good for factual information. AVOID asking descriptive terms for facts. Ask directly to provide numerical estimate. Example: Inaccurate: How often do you deposit savings in your account? Never, hardly ever, sometimes, constantly Accurate: In the last 1 month, how many times did you deposit savings? Number of times_______________. 9

10. 3. Drafting Each Module A. Deciding Question Formats 4. Ranked Responses Sometimes useful for respondent to rank a set of options by numbering. Trouble spots: Assumes respondent feels differently about each item. Can be difficult to compute. Do not add too many categories. 10

11. 3. Drafting Each Module B. Specifying Wording – Coding Designing good response codes requires strong knowledge of the phenomena being studied.  Background research and possible qualitative research are handy here. See similar questionnaires.  Ideally responses to almost all questions should be given code numbers only code numbers, as opposed to words or phrases, are recorded on the questionnaire 11

12. 3. Drafting Each Module B. Specifying Wording – Precoding Precoding requires that codes be clear, simple, and 1.mutually exclusive; 2.that they exhaust all likely answers, 3.that respondents will not all provide the same response, 4.and that none of the codes apply to only a handful of respondents. 12

13. 3. Drafting Each Module B. Specifying Wording – Precoding Useful tricks: To ensure codes are mutually exclusive, add a qualifier where more than one answer could apply: –What is most recent….” –“What is biggest…,” –“What is closest…” –“What was first (or last, or principal) reason for…?” To ensure codes encompass all possible answers, add “other (specify______)” code to questions where precoding each possible response is impossible/ inconvenient. 13

14. 3. Drafting Each Module B. Specifying Wording – Recall periods Trade off potential recall (memory) bias from long period reporting against potential variance from short periods Example, consumption: Frequently purchased items (e.g. food), recall period between one week and one month When practical: diaries for a week or two Durable goods: annual recall basis  Reference periods can be different across questions, and sections  Different recall periods can present a problem when combining or comparing data across modules. 14

15. 3. Drafting Each Module B. Specifying Wording: be concise, clear No Jargon! Asking a rural household: “Do you believe the government should ban higher interest rates?” Will get better response than: “Do you believe the Microfinance Bill / RBI should take action to…” Be clear: Asking “Have you borrowed any money lately?” is more ambiguous than “Have you borrowed any money from friends, employers, groups, etc in the last 30 days?” 15

16. 3. Drafting Each Module B. Specifying Wording: Avoid Double Questions Sometimes questions hide a dual question, for example: Instead ask: 16

17. 3. Drafting Each Module B. Specifying Wording: stick to observables It’s tempting to take shortcuts Stated preference and revealed preference Questions about future intentions are unreliable Can infer preferences from observed behavior 17

18. 3. Drafting Each Module B. Specifying Wording: Be Precise “What is your date of Birth?” Is better than: 18

19. 3. Drafting Each Module B. Specifying Wording: Why Be Precise? Asking exact age will also suffice and have the added advantages that: Exact ages may very simply be recoded into groups Can alter these groups while keeping the same ages. Not only could you inspect different age groups than first envisaged, but you could backdate your age groups. Imagine that a government program makes 20 year old women eligible for a credit and savings program… Can obtain continuous descriptive statistics, e.g., mean and standard deviation. 19

20. 3. Drafting Each Module B. Specifying Wording: final words Avoid leading questions Example: “do you believe that the clutches of the rural moneylenders is a good reason for the government to make a ban on high interest rates?” Avoid “would”, “could”, “should” Ask short questions with simple answers Increases size of questionnaire, but decreases time to administer Make use of skip patterns and filter questions Answers should be read out exactly as written Error rate is up to 20% higher when questions are not read verbatim 20

21. 3. Drafting Each Module C. Detailing Definitions –who is a household member? –what is a business? –who is a migrant? –what is a health event? etc. Don’t bias your respondent –What did you use this loan for? instead of where did you invest this loan? (investment) or what did you buy with this loan? (consumption) –Avoid questions that are open to interpretation How often do you deposit savings in your account? –Never, Hardly ever, Sometimes, Constantly In the last 1 month, how many times did you deposit savings? –Number of times 21

22. 3. Drafting Each Module C. Detailing Cross-check important questions –Sources of income; assets; land ownership; health events and health expenditure; household consumption – at different points in the survey Unit of data and components –food consumption in Rs or Kg. –“bhiga” for reporting land in rural areas –loans, asset ownership is usually recorded at household level Self-reporting v.s. other means of evaluating data –business income – you could capture inputs and sales –perceived resale value of assets instead of cost price (actual expenditure) 22

23. 3. Drafting Each Module D. Translation The local language is probably not English, which makes things tricky as to the wording of certain questions Translate –Ensures that every surveyor knows the exact wording of the questions, instead of having to translate on the fly Always back-translate! –Outsource to 2 people: to translate from English to local language and another to back translate to English. Compare Contextualize – refine during pilots. Find local terms for quantities (units – acre, bhigas, etc.); Adapt local terms/language – keep it informal; meaning should be clear (so avoid using language experts!) 23