1 Non-Response Non-response is the failure to obtain survey measures on a sample unit Non-response increases the potential for non-response bias. It occurs after the sampling step of a survey Non-response rates can help us assess the ACCURACY of our survey statistics

2 Non-Response Measurement The survey community has not developed a standard approach in measuring non- response Statisticians are working together to develop a core set of unit non-response measures AAPOR - American Association for Public Opinion Research IHSNG - Interagency Household Survey Non- Response Group

3 Types of Non-Response in Household Censuses or Surveys No one contacted Refusal No one home Temporarily Absent Language Problem Other Reason

4 Core Set of Non-Response Measures (IHSNG) Initial non-response rates The initial non-interview rate The initial non-interview language problem rate The initial non-interview temporary absence rate The initial non-interview not at home rate The initial non-interview refusal rate The initial non-interview other reasons rate

5 Variables Needed to Compute Core Non-Response Rates NEU: Number of Eligible units NINT: Number of Interviewed Units NENIU: Number of Eligible Non- Interviewed Units NEUE: Number of Eligible non- interviewed Units Excluding refusals, language problems, no one at home, temporarily absent NREF: Number of Eligible Units Refusing to be Interviewed. NEUNH: Number of Eligible Units not interviewed due to No One Home NEUTA: Number of eligible Units never interviewed because occupants are away during the field interviewing period. NEUL: Number of eligible Units not interviewed due to language problems

6 Non-Response Rates All rates include the NEU (Number of Eligible Units) as the denominator INR: Initial Non-interview Rate Definition: Combination of eligible interviewing units that were not interviewed due to language problems, refusal, no one home, temporarily absent or other reasons divided by the total number of eligible interviewing units. Computation: (NENIU/ NEU)*100

7 Field Data Requirements Master address list, field updates Also good to measure characteristics of interview and non-interview by looking at enumerator field notes, enumerator checklist End goal would be to have a coded list of characteristics for non-interview and maybe interviewed units Having the enumerator track his/ her experience can be a good way to find out ways to make contact and conduct an interview

8 Post Enumeration Survey (PES) Post enumeration surveys (PES) are an accepted census self-evaluation tool. The PES should be a independent survey that replicates a census, and occurs three weeks after census enumeration. The PES and census results are then compared during the matching phase of the PES. The PES Evaluates: Coverage of persons, that is the extent of the undercount of individuals Coverage of dwellings, that is where a separate sample of dwellings was used to compare dwellings listed by PES interviewers with those compiled by census collectors, and Accuracy of responses to particular census questions. PES results can be used to adjust Census results Under or over-counts can be converted into adjustment factors and the census population increased or decreased accordingly by these factors. PES does not allow for intervention during field operations. An advantage of taking good field records, updating master address database is that you can calculate a unit non-response measure and assess areas that need intervention.

9 Importance of the Enumerator The enumerator is the link between the survey organization and the householder Enumerators contact the respondent, communicate the purpose of the survey, the importance of participation, the confidentiality policy Enumerator communicates the observations to the survey organization

10 Enumerator Characteristics Socio-demographic characteristics: In areas where linguistic or cultural barriers might limit interaction between enumerators and respondents, select enumerators that are “like” the respondents. Confidence: Enumerators confidence can have a significant effect on survey participation Enumerator Experience: Studies indicate that enumerator experience can have a significant effect on survey participation

11 Improving Contact and Cooperation Develop a contact and cooperation strategy: Field workers should study at-home patterns to determine times of day when sample households are occupied What are the social environment predictors to contact and cooperation? Arm enumerator with communication skills to remain flexible and positive during introduction and in refusal cases Provide enumerator with a short checklist to complete after each interview Determine best way to leave message with potential respondent Written notice, visit to neighbor?

12 Enumerator Messages Survey or Census Institution There have been several studies to determine if the organization sponsor affects the respondents decision In the US, the overwhelming majority of these studies have clearly demonstrated that government or university sponsorship is the most effective in capturing respondents. Standardization Studies indicate that enumerator behavior should be standardized during the questionnaire HOWEVER, standardization of interviewer behavior during the introductory interaction with sample householders is ineffective

13 Enumerator Messages Topics of Study Differential response effects are found for different topics studied, regardless of the conditions of the interview. KNOW THE SURVEY Anonymity and Confidentiality Some studies have shown that response rate is affected by the anonymity/ confidentiality policy of a study. Others have reported that responses became more distorted when subjects felt threatened that their identities would become known. Others have found that anonymity and confidentiality issues do not affect response rates or responses.

14 Publicity and Community Liaison Strategy Increasing public understanding by advance notice of survey or census will help improve response rates, and higher quality data will be achieved. Develop content of key Census/ Survey information explaining the dates, general process, confidentiality, topic, importance, sponsorship The NSO (field operations or media office) will coordinate with partner organizations to deliver the message in the most appropriate way for the community they are targeting Media, publicity, community organizations contact data base should be created to facilitate communication with potential publicity partners. Regular, scheduled information kits should be delivered to contacts so they are aware of operational changes

15 Hard to Reach Populations Partnerships with community groups Field Operations should have a data base/ list of community groups that work with hard to reach populations (nomads, homeless, orphans). Recruitment of enumerators from these hard to reach populations Enumeration in a single night

16 Practical implications for Census and Survey Implementation For National Statistical Offices: Non-response measurement is possible with minimal data and can help improve the methodologies used Focus on developing training curriculum that emphasizes contact, communication and cooperation training. What are the budget implications? Importance of budgeting for high quality, pre-tested training instruments For Field Operations Enumerator training should develop communications skills, strategies for tailoring and maintaining interaction Use of a checklist for enumerators Importance of a Master Address Database, and the listing operation so you can have the number of eligible units Importance of recording case characteristics, interview and non- interview alike