TEXTBOOK There are now copies of the textbook in the campus bookshop

Chapter 3 How to ask a question (plus some statistical terms). n We saw in the previous chapter that deciding exactly what to measure and what questions to ask is extremely important. Remember the 4th component n 4. The exact nature of the measurements made or the questions asked. n In this chapter we will examine this component in detail.

Section 3.1 Questions n A study was conducted in the US in 1974 where two researchers showed college students a film of a car accident. n After viewing the film the students were asked one of two questions. n Group 1 was asked the question: “About how fast were the cars going when they contacted each other” “About how fast were the cars going when they contacted each other” u The average of the responses for Group 1 was 31.8 Miles per Hour

n Group 2 was asked the question: “About how fast were the cars going when they collided with each other” “About how fast were the cars going when they collided with each other” u The average of the responses for Group 2 was 40.8 Miles per Hour Both groups had seen exactly the same film. The only difference was the use of the word collided instead of contacted. n Simply using the word collided increased peoples estimates of the speed of the accident by 9 mph or 28%.

There are many problems associated with asking questions we will examine seven of them n Deliberate Bias n Unintentional Bias n Desire to please n Asking the uninformed n Unnecessary complexity n Ordering of questions n Confidentiality and anonymity

n Deliberate Bias Sometimes when a survey is conducted, the questions are worded in a leading manner to illicit a favourable response. Recall the questions on refugees in Ireland. n The responses to questions that begin “Do you agree that…..” should be treated with caution. n “Asked whether they felt New Improved Persil was better at cleaning clothes than ordinary Persil, 90% of people said yes.” u Who wouldn’t say yes to such a leading question. n “Do you agree with the continued destruction of trees on this campus for the construction of new buildings?” n “Do you agree that during the construction of new buildings to alleviate the overcrowding on Belfield campus that it is okay to knock down a few trees?”

n Unintentional Bias Besides the deliberate bias caused by leading questions, sometimes the questions are worded badly unintentionally and are misinterpreted by many respondents. n “What was the most important date in your life?” People may respond differently to this question. n Some may interpret the word date as calendar date and may reply for instance - “The day I passed my finals” n Some may interpret the word date as “dinner and a movie”. n And some may think that a shrivelled fruit is being referred to.

n Desire to please Many respondents like to please the questioner. Recall the sketch from “The Fast Show” Respondents do not like to admit to certain socially undesirable habits n Surveys on the prevalence of cigarette smoking based on surveys of individuals disagree with data from cigarette sales.

n Asking the uninformed u Nobody likes appearing ignorant when asked a question. n The day that Articles 2 & 3 of our constitution were changed TV3 sent a reporter out to Grafton Street to ask Dubliners: u Do you know what important thing happened today in the North?” u Most people replied yes. n But the reporter then asked the people: u “OK, so what happened?” u Many people got embarrassed and said that they didn’t know after all.

n Unnecessary Complexity Questions should be kept simple, otherwise people may get confused. n “Shouldn’t students not be allowed to repeat their exams if they fail at the first attempt.” n This sentence actually contains a double negative.Is it therefore equivalent to the question: u “Should students be allowed to repeat their exams if they fail at the first attempt.”

n Ordering of Questions. If two questions are asked of a respondent but one question causes the respondent to think about something they may not have thought of otherwise then the order of the questions will be important. n Example u Name the five most popular types of television programme. u Do you watch hospital dramas on TV such as ER?

n Confidentiality and Anonymity n Anonymity Some questions may only be answered if the respondent feels that they are anonymous. n Confidentiality If a follow up study is necessary then respondents cannot remain anonymous and so confidentiality of responses must be ensured. n Questions on sexual behaviour and financial dealings are usually only responded to if either Anonymity or Confidentiality can be ensured.

Section 3.2 Choices n When asking a question should we present the respondent with a choice of possible answers. n Should we ask open questions or closed questions? n Most opinion polls are conducted using closed questions i.e. the respondent is asked to chose between a group of answers. This allows easy compilation of the results of the survey compared to an open question format.

n Closed Questions. We’ve already mentioned that opinion polls often use the closed question format, in which the respondent is presented with a choice of answers. This form of question can often lead to some very strange results. n The textbook refers to a study conducted in 1987 in the US to examine the difference between Open Questions and Closed Questions. The study asked the following Question: n “What do you think is the most important problem facing this country today?”

n Half of the sample were given this as an open question, the top four responses were: u 17% Unemployment u 17% General Economic Problems u 12% Threat of Nuclear War u 10% Foreign Affairs n The other half of the sample were given this as a closed Question to pick between the following choices: u The Energy Shortage u The Quality of Public Schools u Legalised Abortion u Pollution u If you prefer you may name a different problem as most important.

n The responses to this closed question were: u 5.6% The Energy Shortage u 32% The Quality of Public Schools u 8.4% Legalised Abortion u 14% Pollution n So even though the respondents were allowed to choose an alternative to these 4 choices, 60% saw these four as being the most important problems. n But in the open question format these problems were only listed by 2.4% of respondents. n Something is wrong here! n WHAT IS HAPPENING?

n Open Questions. n We mentioned one problem with the open question format, that it is hard to compile results from possibly thousands of different responses. n There is another major problem with the open question format, this was highlighted in the same 1987 study referred to earlier. n A group of respondents were asked to “name one or two of the most important national or world events or changes during the past 50 years”

n Half of this sample were given this as an open question, the top responses were: u 14.1% World War II u 6.9% Space Exploration u 4.6% JFK Assassination u 10.1% Vietnam War u 10.6% Don’t know u 53.7% All Other Responses n These responses were then given as a closed Question together with another choice “The invention of the computer” - this had been mentioned by only 1.4% of respondents in the Open Question format.

n The responses to this closed question were: u 22.9% World War II u 15.8% Space Exploration u 11.6% JFK Assassination u 14.1% Vietnam War u 29.9% The Invention of the Computer u 0.3% Don’t know u 5.4% All Other Responses n The problem here was the wording of the question, people concentrated on the word events rather than changes. When it was shown to them they realised that the invention of the computer was indeed one of the most important changes during the past 50 years.

n To summarise: Perhaps the best way to ask a question is to conduct a small trial Open Question format survey. Then use the responses from this trial survey as the choices in a Closed Question survey together with any other answers that may not immediately spring to mind.

Section 3.3 Defining what’s being measured n Before we use the results of a survey we should be fully aware of what was actually measured by the survey. n Example: Unemployment data in Ireland. n Example: Teenage Sex in America u See example in textbook n Newspaper carried two reports u “..sexual activity among adolescents is on the rise.” u “.. adolescents seem to be having sex less often, with fewer girls and at a later age than.. a decade ago.”

n So we seem to have two completely contradictory n The problem was the studies were actually measuring different things. n The first study was looking at the age of first intercourse: “Average age.. is 17.2 for females and 16.5 for males” n The second study was looking at frequency among boys 15 to 19: “average of 6 sex partners, compared to 7 a decade earlier. They.. had sex an average of 3 times during the previous month, compared with almost 5 times in the earlier survey.”

Section 3.4 Some Statistical Terms n n Measurement/Numerical Data: Data we measure in the form of numbers. n n Examples: u Percentage you will get on the summer exam for this course. u Number of lectures that you will skip. u Frequency of radio station you listen to when studying. n n Categorical Data: Data which can be placed in a category, cannot add/subtract this kind of data. n n Examples: u Grade you will receive on the summer exam for this course. u Name of radio station you listen to. u Brand of shoes you are wearing.

Numerical/Measurement data is further distinguished as to whether it is Discrete or Continuous. n n Discrete variables take only isolated whole number values (integers) on the number line. u Example: Number of Nike runners in this class. n n Continuous variables have values comprising entire intervals of the number line. Decimals and Fractions are allowed. u Example: The duration of this class. remember seconds are not the smallest unit of time measurement. This class could possibly last Minutes.

n Validity A valid measurement is one which actually measures what it claims to measure. u Example: Unemployment figures are validly measured using the Labour Force Survey not the Live Register

n Reliability A reliable measurement is one which will give approximately the same result time after time, when taken on the same individual or object. u Example: Most physical measurements are reliable, for example measuring your weight using a bathroom scales. u Some measurements may be reliable but not necessarily valid. n Are exams reliable measuring devices? n Are exams results valid measurements of intelligence?

n Bias Sometimes when measurements are made a systematic error is made which underestimates or overestimates the true value. Such a measurement is called a biased measurement. n Example: Suppose your bathroom scales always overestimated your weight. n Example: Car Speedometers are deliberately biased to overestimate a car’s real speed.

n Variability u If we try to measure a certain characteristic for many different objects or people we will most likely not get the same answer each time. The fact that the observations vary is referred to as the variability in the dataset. u Some datasets are more variable than others: u Example: A dataset consisting of the ages of 100 students in UCD will be less variable than a dataset consisting of the ages 100 randomly chosen Irish people.

Homework n Design two surveys to look at some of the concepts in this chapter. n Chose a random sample of 20 people and divide the sample in to two groups of 10. n 1. Examine bias caused by changing words in one question. n 2. Examine the effects of using Open Questions vs Closed Questions n The Topics of the questions are up to you!