Dr. Michael R. Hyman, NMSU Survey Error: Focus on Systematic Error

2

3 Total error Systematic error (bias) Random sampling error Total Survey Error: Components

4 Random Sampling Error A statistical fluctuation that occurs because of chance variation in the elements selected for the sample.

5 Systematic Error Systematic error results from some imperfect aspect of the research design or from a mistake in the execution of the research

6

7 Respondent Error A classification of sample bias resulting from some respondent action or inaction Nonresponse bias Response bias

8 Respondent error Nonresponse error Response bias Respondent Error: Components

9 Non-response Error Non-respondents –Refuse to cooperate –Not-at-homes Self-selection bias –Over-represents extreme positions –Under-represents indifference Varies by type of interview

10 Why People Cooperate To be supportive/helpful Have a social interaction Curiosity Remuneration

11 Factors Contributing to Refusal Fear/anxiety Invasion of privacy Hostility toward –Survey sponsor –Topic of interview –Interviewer

12 Response Bias A bias that occurs when respondents tend to answer questions with a certain slant that consciously or unconsciously misrepresents the truth

13 Response bias Unconscious misrepresentation Deliberate falsification Response Bias: Components

14 Unwillingness to Respond Accurately Invasion of privacy Time pressure and fatigue Physical/social environment Questionnaire-specific issue –Question/response wording –Layout Mischievous respondents

15

16 Inability to Respond Accurately Inarticulate Ignorant –Asked about other people –Asked to predict own behavior –Asked question meant for other people

17

18

19 Inability to Respond Accurately Faulty memory –Only ask about recent events –Problem with questions about behavior over time (e.g., frequency) –Telescoping Everything happened yesterday

20 Acquiescence bias Extremity bias Interviewer bias Auspices bias Social desirability bias Unconscious Misrepresentation: Components

21 Acquiescence Bias A type of response bias due to some people tending to agree with all questions or to concur with a particular position.

22 Extremity Bias A type of response bias due to response styles that vary from person to person; some people tend to use extremes when responding to questions.

23 Interviewer Bias A response bias that occurs because the presence of the interviewer influences answers.

24

25 Auspices Bias Bias in respondents’ answers due to being influenced by the organization conducting the study.

26 Social Desirability Bias Bias in responses caused by respondents’ desire, either conscious or unconscious, to gain prestige or appear in a different social role.

27

28

29 Systematic error (bias) Administrative error Respondent error Systematic Error: Components

30 Administrative Error Improper administration of the research task Blunders Confusion Neglect Omission

31 Administrative Error Interviewer cheating - filling in fake answers or falsifying interviewers Data processing error - incorrect data entry, computer programming, or other procedural errors during the analysis stage. Sample selection error -improper sample design or sampling procedure execution. Interviewer error - field mistakes

32 Interviewer Error Subtle source of cues –Can inadvertently reinforce some responses Appearance of incompetence

33

34

35 Interviewer Error Insufficient or poor probing

36

37

38

39 Interviewer Error Fraud Why commercial houses validate roughly 10-15% of completed interviews Selecting unqualified respondents

40 Checklist for Unacceptable Questionnaires Major portions of questionnaire or key questions left unanswered Evidence that respondent did not understand instructions or did not take task seriously Missing pages Respondent not qualified for target population Questionnaire returned after cutoff date