Field procedures and non-sampling errors Datainnsamling Field procedures and non-sampling errors Litteratur: Churchill kap. 12 Troye & Grønhaug kap. 5

Sampling Given a group of objects, the total group is called the population. For the purpose of analysis and inference to the population, we often look at a sample, or subset of the population. When conducting research studies, two types of errors arise: sampling errors and non-sampling errors.

Random Errors versus Nonrandom Errors Random errors tend to average out so, all else being equal, they don’t cause a bias. Nonrandom errors tend to affect results in one direction, thus causing a bias (i.e. problem: we hate nonrandom errors).

Nonrandom Errors May increase as sample size increases. We may not know the direction or magnitude of the problem. Do we overestimate or underestimate the parameter? They distort reliability.

Non-observation Errors versus Observation Errors Non-observation errors = failure to obtain data from parts of the survey population. Either missed in collection or did not respond. Observation errors = inaccurate information or data processing problems. Can be more problematic because we may not recognize that a problem exists.

Overview of Non-Sampling Errors coverage Not-at- homes Non- observation Non- response Non-sampling Biases Refusals Data collection errors Observation Office processing errors

Identifying Data Collection Errors Use bias-free information as validity check. Use different forms of same questions to check for consistency of responses. Measure consistency over time from same respondent.

Non-Coverage Errors Not a deliberate exclusion. e.g. women only. How do you determine your sampling frame (i.e. list of potential respondents)? Telephone book, membership list, etc. Is your sampling frame a designated area of the city? e.g. interviewer bias for/against ethnic groups.

Approaching households. Lower income are often avoided. Typically select most accessible person in household, e.g. housewife. Mall or street interviews. Is it random? Ski Storsenter (low-med.) Vinterbru (med.-high). How often do they shop? Who stops? Interviewer cheating, e.g. filling quotas.

Over-coverage Error Duplication Multiple telephones. Multiple addresses

Two Questions 1. How pervasive is the non-coverage bias? You have to find some outside criterion (e.g. another accurate study) to compare to. Is it accurate? Can you compare the two? E.g. what was the unit of analysis (people, households, etc.)?

2. What can be done to reduce non-coverage bias? Use accurate sampling frame. Weight sub-samples, like low-income households.

Non-Response Error: Failure to obtain information from some elements of the population that were selected and designated for the sample. Not contacted. Why? Not home, not available, too busy, etc. Contacted, but no response. Why? Rejected, not eligible, not complete, not usable, etc.

# of complete responses # of eligible respondents Response rate = Are those who responded different in some important and systematic way from those who did not respond? Married women. Low income families. Rural families.

Interviewer Effectiveness # of eligible units contacted # of eligible units approached Contact rate = This measures the persistence of the interviewer.

Refusals depend on: Nature of respondent/interviewer. Looks, sex, race, etc. Female, non-whites, and low education are more likely to refuse. The type of research. Is it a sensitive subject. Is it an important subject. The context of the contact. Public (in the classroom).

Remedies Make an appointment. Call back. Re-send questionnaire. Offer reward. Questionnaire length. Questionnaire wording. Type of questions, e.g. open versus closed.

Adjusting results: Sending method. Confidentiality. Identify non-respondents, analyze how they differ from respondents, then weight the results. Compare early versus late responses, then extrapolate the results.

Item Non-Response: Specific questions are not answered. Is it random? How to deal with it: Delete incomplete cases. Substitute answers: Use the mean (gjennomsnitt) from completed cases. Base the answer on completed questions. Use regression to estimate an answer.

Field Errors: The respondent refuses to answer specific questions, answers incorrectly, or lies. The respondent must: Understand the question. Think about the question. Evaluate the response for accuracy. Evaluate the response in terms of goals. Give the response.

Interviewer-Interviewee Interaction Model Background Characteristics age, education, race, sex religion socioeconomic status Background Characteristics age, education, race, sex religion socioeconomic status Psychological Factors Perceptions Attitudes Expectations Motives Psychological Factors Perceptions Attitudes Expectations Motives Behavioral Factors Errors in asking questions Errors in probing Errors in motivating Errors in recording responses Behavioral Factors Response to questions -adequate-inadequate -accurate-inaccurate