Designing psychological investigations Issues of reliability and validity

Reliability and validity When we design a study and collect data we want the data to be both reliable and valid. If it is not reliable and valid we cannot draw conclusions from it. Reliability is concerned with the consistency of data. Do we get the same results time after time? Validity is concerned with the truthfulness of the data. Are we measuring what we intended to? AQA Psychology A A2 Level © Nelson Thornes 2009

Tests can be reliable but not valid. If I weigh something three times and each time the weight is 3 kg then the measure is reliable. However, if the true weight is 2 kg then the reliable measure is not valid. Tests cannot be valid but not reliable. If I get a 2 kg measurement for my 2 kg object then the measure is valid. Any other measurement (i.e. any unreliable measures) cannot be valid. AQA Psychology A A2 Level © Nelson Thornes 2009

reliability Reliability There are two main types of reliability: external and internal External = between tests Do we get the same results time after time using the same test or doing the same investigation? Internal = within one test Are the results consistent within a test? If we split the test in half, do both halves show the same thing? AQA Psychology A A2 Level © Nelson Thornes 2009

Improving reliability Reliability can be improved by greater accuracy of measurement. This can be achieved by: pilot studies – checking that measurements work taking more than one measurement standardising measurement. In observations reliability can be improved by checking the effectiveness of observation categories. AQA Psychology A A2 Level © Nelson Thornes 2009

Validity There are many types of validity. reliability Validity There are many types of validity. The two main types to consider when designing psychological investigations are: internal validity external validity. AQA Psychology A A2 Level © Nelson Thornes 2009

reliability Internal validity Internal validity = whether we are measuring what we wanted to measure. If a DV in a study is ‘level of anxiety’, does the measure we take actually reflect the participant’s anxiety? There are a number of factors that can affect internal validity including: operationalisation of variables confounding variables participant bias (demand characteristics) experimenter bias. AQA Psychology A A2 Level © Nelson Thornes 2009

Operationalisation of variables: Most concepts in psychology are difficult to measure directly. We have to find some way of operationalising them. If we choose a poor way of doing so the internal validity will be poor. For example, measuring speed of running up stairs may be reliable but it is not a valid measure of IQ. Confounding variables: In experiments sometimes the DV measurements may be consistently affected by variables other than the IV. For example, we might think an increase in heart rate is due to anxiety about study, but really it has been caused by the participants drinking coffee. AQA Psychology A A2 Level © Nelson Thornes 2009

Participant bias (demand characteristics): If participants are aware of the purpose of a study, it may change their behaviour. Furthermore the situation of the study and the instructions can lead to demand characteristics – the participant starts to play a role rather than behaving normally. For example, Orne (1962). Experimenter bias: There is a tendency to find what you ‘expect’ to find. This can be very subtle and investigators are usually unaware of it. For example, Rosenthal and Fode (1963). AQA Psychology A A2 Level © Nelson Thornes 2009

Improving internal validity reliability Improving internal validity Confounding variables: Confounding variables need to be removed to improve internal validity. This is the purpose of control. Participant bias: The single-blind technique. Experimenter bias: The double-blind technique. AQA Psychology A A2 Level © Nelson Thornes 2009

reliability External validity External validity is concerned with how well the result of the study can be generalised outside the study. Does the study have any meaning or relevance outside of the study? There are two main elements: population validity ecological validity. AQA Psychology A A2 Level © Nelson Thornes 2009

Population validity: Psychology studies usually study a small sample of the target population they are interested in. Can the findings of the sample in the study be generalised to other people? Ecological validity: Can the results of the study be generalised to other settings (including ‘real-world’ settings)? AQA Psychology A A2 Level © Nelson Thornes 2009

Improving external validity reliability Improving external validity Population validity is increased by having a random, representative sample. The desire for a random, representative sample raises an ethical conflict. We cannot force participants to take part so will always have a biased sample. Ecological validity can be increased by making the task ‘realistic’. There is often a balance to be struck between external and internal validity – between realism and control. AQA Psychology A A2 Level © Nelson Thornes 2009