2.3 Bias in Surveys

The results of a survey are only accurate if the sample is representative of the population, and the measurements are objective Statistical Bias - any factor that favours certain outcomes or responses, and skews the results

1. Sampling Bias The sampling frame does not reflect the characteristics of the population Results from inappropriate sampling technique or data collection

Examples - Identify the bias, suggest how it could be avoided A) A survey asks students at a high-school football game whether a fund for extra curricular activities should be used to buy new equipment for the football team or instruments for the band

BIAS - only asking football fans - not representative of the entire population –SAMPLING TECHNIQUE BIAS AVOID by conducting a Random Sample of the entire school

Examples - Continued B) An aid agency in a developing country wants to know what proportion of households have at least one personal computer. One of the agency’s staff members conducts a survey by calling households randomly selected from the telephone dierctory

BIAS - in a developing country - not everyone will have a phone, and they likely don’t have a computer (  overestimate the number of computers) –DATA COLLECTION BIAS AVOID by visiting randomly selected households

2. Non-Response Bias Particular groups are under represented in a survey because they choose not to participate Form of sampling bias

Example A science class asks every fifth student entering the cafeteria to answer a survey on environmental issues. Less than half agree to complete the questionnaire. The completed questionnaires show that a high proportion of the respondents are concerned about the environment and are well informed about envirmonmental issues

BIAS - the students who choose not to respond are likely not interested in the issues (survey misrepresents the veiws of the students) *Sample is not representative of the population –SAMPLING BIAS (NON-RESPONSE)

** To avoid non-response bias - ensure that the sampling process is random. Include questions that identify members of a particular group to ensure they are properly represented in the population

3. Measurement Bias Data collection method consistently under or overestimates a characteristic of the population

Examples A) A highway engineer suggests that a way to survey traffic speeds on an expressway would be to have the police officers who patrol the highway record the speed of the traffic around them every half hour

BIAS - drivers slow down when they see cop car  underestimate the average speed of traffic) –DATA COLLECTION BIAS AVOID by using an unmarked car, hidden radar

Examples B) As part of a survey of the “Greatest Hits of All Time” a radio station asks its listeners: Which was the best song by the Beatles? a) Help b) Nowhere Man c) In My Life d) Other

BIAS - Only lists early Beatles songs –LEADING QUESTION - A question that prompts a particular answer Usually by providing choices for a particular question, instead of leaving it open

Examples C) A poll by a tabloid newspaper includes the question: “Do you favour the proposed bylaw in which the government will dictate whether you have the right to smoke in a restaurant?”

BIAS - “…Government dictates…” –LOADED QUESTION - A question containing information or language intended to influence the respondents answers

4. Response Bias Participants deliberately give false r misleading answers. Respondents intentionally want to influence the results, or are too afraid or embarrassed to answer honestly

Example Expose survey on student’s use of tobacco: asked questions such as “have you ever smoked before”, “when did you start”, etc. –Some students may be afraid to answer truthfully –Some students may intentionally answer incorrectly