Can the study be replicated? Would it produce the same results? Reliability Can the study be replicated? Would it produce the same results?

High reliability A study can be said to have high reliability if it would be possible and feasible to replicate that study. It needs to have a clear procedure that anybody could follow. In general, quantitative measures such as questionnaires and experiments have high reliability. Also showed this decrease was mostly due to 21% decrease in theft from a person and 22% theft of other property. Domestic burglary and vehicle theft remained the same. Showed that overall crimes totalled 6.8 million incidents. This was a 7% annual decrease. It was also the lowest figure since 1981. BCS2015

High reliability The BCS selects around 47,000 adults and 4,000 10 – 15 year olds each year. They are randomly selected from the Postal Address Frame. One researcher goes to each post code zone to selected post codes and asks a standardised questionnaire from which quantitative data can be taken.

High reliability The BCS has high reliability because it has a standardised procedure that technically anyone with a little training could carry out. It is easily replicable. Also if it was re-run it is highly likely to produce the same results.

Examples of low reliability If there is only one or a few researchers it is possible that their biases or inconsistencies may influence the results. Eg. Environmental campaigners conducting interviews about an airport expansion. Solutions Use an independent research team as they should be less biased. This can be expensive. Employ training tactics to ensure consistency in the interview process. This can be costly and timely.

Examples of low reliability If there are lots of researchers it is possible that there results will vary significantly from one researcher to another. Eg. Exam markers Solutions Inter-rater method. Get two or more markers to mark the same exam paper (or analyse the same interview transcript or perform the same observation). If all researchers get similar results we say with more certainty that the data is reliable. This is called ‘moderation’ when it is done by exam boards.

Examples of low reliability If a new and untested research criteria is being used. If there are reasonable doubts about the accuracy of the data. Eg. A questionnaire reveals higher than expected political interest after an election. Solutions Test re-test method. Simply run the experiment or research again. If the data was accurate the results will be repeated. The same test run six months and then twelve months later shows lower levels of political interest, this data is now more reliable.

Examples of low reliability Temporal issues Has the time and place of my research influenced my results? Eg. A questionnaire about attitudes to sport given on Aylestone Road on a Saturday. Is my research tied to some unrepeatable event? Eg. Conducting interviews after a terrorist attack. Solutions Consider the effect of time and place on the research. (Consider collecting data from multiple locations at a variety of times). There is little to be done here. Some very important research is by its very nature unreliable.

Low reliability? American writer Bill Buford spent eight years involving himself with 1980’s Manchester United hooligan gang called the intercity jibbers. Fully immersed, he participated in fights, riots and was beaten up by police. He relates both first and second hand reports of hooligan violence, ranging from beatings to stabbings to a supporter biting out the eye of a police officer.

“This is, if you like, the answer to the hundred-dollar question: why do young males riot every Saturday? They do it for the same reason that another generation drank too much, or smoked dope, or took hallucinogenic drugs, or behaved badly or rebelliously. Violence is their antisocial kick, their mind-altering experience, an adrenaline-induced euphoria that might be all the more powerful because it is generated by the body itself, with, I was convinced, many of the same addictive qualities that characterize synthetically-produced drugs.” Buford, B. 1992. Among the Thugs: The experience, and the seduction, of crowd violence. pg. 217

Low reliability But we can’t take it all to be true. Firstly, he was a journalist hoping to make an entertaining book. But also it is not very reliable. There was only one researcher. How much was Buford’s opinion? Would another researcher have produced different conclusions? Could his research be repeated?

Just because something is not reliable doesn’t mean it is not useful or valid. However, the more reliable the data is, the more validity the research can be said to have.