1. Designing psychological investigations
Issues of reliability and validity

2. Reliability and validity
When we design a study and collect data we want the data to be both reliable and valid.
If it is not reliable and valid we cannot draw conclusions from it.
Reliability is concerned with the consistency of data.
Do we get the same results time after time?
Validity is concerned with the truthfulness of the data.
Are we measuring what we intended to?
AQA Psychology A A2 Level © Nelson Thornes 2009

3. Tests can be reliable but not valid.
If I weigh something three times and each time the weight is 3 kg then the measure is reliable. However, if the true weight is 2 kg then the reliable measure is not valid.
Tests cannot be valid but not reliable.
If I get a 2 kg measurement for my 2 kg object then the measure is valid. Any other measurement (i.e. any unreliable measures) cannot be valid.
AQA Psychology A A2 Level © Nelson Thornes 2009

4. reliability
Reliability
There are two main types of reliability: external and internal
External = between tests
Do we get the same results time after time using the same test or doing the same investigation?
Internal = within one test
Are the results consistent within a test? If we split the test in half, do both halves show the same thing?
AQA Psychology A A2 Level © Nelson Thornes 2009

5. Improving reliability
Reliability can be improved by greater accuracy of measurement. This can be achieved by:
pilot studies – checking that measurements work
taking more than one measurement
standardising measurement.
In observations reliability can be improved by checking the effectiveness of observation categories.
AQA Psychology A A2 Level © Nelson Thornes 2009

6. Validity There are many types of validity.
reliability
Validity
There are many types of validity.
The two main types to consider when designing psychological investigations are:
internal validity
external validity.
AQA Psychology A A2 Level © Nelson Thornes 2009

7. reliability
Internal validity
Internal validity = whether we are measuring what we wanted to measure.
If a DV in a study is ‘level of anxiety’, does the measure we take actually reflect the participant’s anxiety?
There are a number of factors that can affect internal validity including:
operationalisation of variables
confounding variables
participant bias (demand characteristics)
experimenter bias.
AQA Psychology A A2 Level © Nelson Thornes 2009

8. Operationalisation of variables:
Most concepts in psychology are difficult to measure directly. We have to find some way of operationalising them. If we choose a poor way of doing so the internal validity will be poor.
For example, measuring speed of running up stairs may be reliable but it is not a valid measure of IQ.
Confounding variables:
In experiments sometimes the DV measurements may be consistently affected by variables other than the IV.
For example, we might think an increase in heart rate is due to anxiety about study, but really it has been caused by the participants drinking coffee.
AQA Psychology A A2 Level © Nelson Thornes 2009

9. Participant bias (demand characteristics):
If participants are aware of the purpose of a study, it may change their behaviour. Furthermore the situation of the study and the instructions can lead to demand characteristics – the participant starts to play a role rather than behaving normally.
For example, Orne (1962).
Experimenter bias:
There is a tendency to find what you ‘expect’ to find. This can be very subtle and investigators are usually unaware of it.
For example, Rosenthal and Fode (1963).
AQA Psychology A A2 Level © Nelson Thornes 2009

10. Improving internal validity
reliability
Improving internal validity
Confounding variables:
Confounding variables need to be removed to improve internal validity. This is the purpose of control.
Participant bias:
The single-blind technique.
Experimenter bias:
The double-blind technique.
AQA Psychology A A2 Level © Nelson Thornes 2009

11. reliability
External validity
External validity is concerned with how well the result of the study can be generalised outside the study.
Does the study have any meaning or relevance outside of the study?
There are two main elements:
population validity
ecological validity.
AQA Psychology A A2 Level © Nelson Thornes 2009

12. Population validity:
Psychology studies usually study a small sample of the target population they are interested in.
Can the findings of the sample in the study be generalised to other people?
Ecological validity:
Can the results of the study be generalised to other settings (including ‘real-world’ settings)?
AQA Psychology A A2 Level © Nelson Thornes 2009

13. Improving external validity
reliability
Improving external validity
Population validity is increased by having a random, representative sample.
The desire for a random, representative sample raises an ethical conflict. We cannot force participants to take part so will always have a biased sample.
Ecological validity can be increased by making the task ‘realistic’.
There is often a balance to be struck between external and internal validity – between realism and control.
AQA Psychology A A2 Level © Nelson Thornes 2009