Reliability and Validity

* How consistent the test is within itself - this might be affected by different people collecting the data differently in an observation or two measures in the same test or tests which do not match (IQ tests) * How consistent the test is over time (for example would Milgram’s test still be reliable today? – the evidence, by the way, is that it is!)

Test Retest Method – Use the same participants (pps) and repeat the same test after a period of time and correlate the results. If there is a strong positive correlation, it is reliable. Tighten your controls – are there any extraneous or confounding variables that were not considered, that might have affected the data – if there were then control for these. An example might be personality in the Milgram experiment. He did not control for personality and so it is possible his results showed how certain personality types behave and not how everyone would behave. Other sample controls that can be tightened is age of participants, gender, culture, background and past experiment of the participants etc. If the study is observation then have more than one observer, collate the data and if there is a strong positive correlation between observers the data should be reliable. This is inter rater reliability

Internal – How properly the experiment was conducted to produce truthful and accurate results and therefore how likely it is to be measuring your aim or hypothesis External – Whether the results can be generalised beyond the research situation to everyday life situations 2. Ecological Validity… …The extent to which the study’s results would occur in a natural setting.

Internal Validity – check how the dependent variable is operationalised and ensure this really does measure the hypothesis. For example does gambling more than once a week on a fruit machine really make you a regular gambler? Does a personality test really measure your personality? Or – reduce demand characteristics by ensuring the participants do not know the hypothesis and the aim of the researchers (Milgram) Or -- reduce experimenter bias by ensuring the person who collects the data does not know which group he is measuring (experimental or control) – Maguire and Bandura did this! Concurrent Validity – Correlate the findings of your experiment with another measured. If there is a strong positive correlation, it is valid. For example Baron Cohen did a measure of facial recognition to ensure it correlated with the results of the eye test. Ecological validity – ensure you research is carried out in a real or at least realistic environment. Piliavin ensured high ecological validity and Reicher and Haslam tried to recreate it in a an experimental setting.

 Well Milgram was pretty reliable – he had a lot of controls - he standardised his instructions, the situation and the procedure! He has also repeated his experiment (so have others) and each time very similar results have been found!  One thing Milgram did not ‘control’ for was the personality of his participants. This leads us to the conclusion that by using a volunteer sample it could be that his data was biased due to the personality characteristics of his volunteers? Perhaps if he measured their personalities first and then chose a selection of different personalities or only measured one personality type he may have obtained different results! For example perhaps extroverts or very confident people are more likely to refuse to obey and people who are more shy and timid are more likely to obey?

 There are two validity problems with Milgram:  1 st internal validity – it could be said that demand characteristics affected the data even though he lied to the participants about the nature of the hypothesis.  2 nd ecological validity - the setting of a university was not the type of setting where you would expect to have to be thinking about being obedient to an authority figure where potential harm might be caused!

 Point 1 – internal validity:  Milgram would have to reduce demand characteristics and the only way he could do this would be not to let people know they were taking part in an experiment - the only option therefore would be to change his method entirely.  One thing he did do to see how important the university setting was on demand characteristics. He moved his experiment down town to an office. Once people did not know the experiment was for a prestigious university department they were indeed less obedient!

 Point 2 –ecological validity  In order to make the Milgram experiment more valid you would have to carry out the same or similar experiment in a real life setting. Hoffman did this in hospitals. He pretended to be a doctor on the phone to nurses and ordered them to give dosages of medication which was dangerous to patients without coming to the ward and giving written authority. He found without exception all the nurses were obedient to the doctors (they were stopped before the medication was given) even when they had suspicions that the medication was dangerous.