1. Reliability

2. What does it mean?
In pairs, think of how we use the words reliability and validity in everyday life.
What do these words mean? Is there a difference between them or do they mean the same thing?

3. Objectives
To review Reliability and to define Internal and external reliability.
To use this understanding to discuss issues of reliability connected with different research methods.
To complete a table to help us revise.

4. Reliability
When assessing the reliability of a study, we generally need to ask two questions
Can the study be replicated?
If so, will the results be consistent?
High vs low reliability

5. Reliability
How consistently a test produces the same measurements when repeated

6. Measuring Reliability
Match the method of estimating reliability to the description
Test-Retest reliability
If the measure depends upon interpretation of behaviour, we can compare the results from two or more raters.
If the results in the two halves are similar, we can assume the test is reliable
Split Half Reliability
Splitting a test into two halves, and comparing the scores in both halves
If the results on the two tests are similar, we can assume the test is reliable
Inter-Rater reliability
The measure is administered to the same group of people twice
If there is high agreement between the raters, the measure is reliable

7. Types of Reliability Internal Dealing with this issue
Is a study consistent WITHIN ITSELF?
Eg do all the questions on an IQ test measure intelligence
Dealing with this issue
SPLIT-HALF method
Scores on both halves should be similar

8. Types of Reliability External Dealing with this issue
Consistent over several occasions?
Eg would a questionnaire get the same results if repeated
Dealing with this issue
TEST-RETEST method
Scores on both times should be similar

9. Objectives
To review Reliability and to define Internal and external reliability.
To use this understanding to discuss issues of reliability connected with different research methods.
To complete a table to help us revise.

10. Reliability of Observations
Inter-observer reliability
Two observers watching the same event should produce the same record
Calculated by total agreements/total observations
+0.8 or higher = good inter-observer reliability
Dealing with this issue
Training of observers in using coding systems and behaviour checklists.

11. Reliability of Field Experiments
What questions do you need to ask?
How could you try to ensure reliability? Can you?

12. Reliability of natural Experiments
What questions do you need to ask?
How could you try to ensure reliability? Can you?

13. Reliability of case studies
What questions do you need to ask?
How could you try to ensure reliability? Can you?

14. Reliability of correlations
What questions do you need to ask?
How could you try to ensure reliability? Can you?

15. Reliability of interviews
What questions do you need to ask?
How could you try to ensure reliability? Can you?

16. Reliability of questionnaires
What questions do you need to ask?
How could you try to ensure reliability? Can you?

17. Reliability of content analysis
What questions do you need to ask?
How could you try to ensure reliability? Can you?

18. Objectives
To review Reliability and to define Internal and external reliability.
To use this understanding to discuss issues of reliability connected with different research methods.
To complete a table to help us revise.

19. Objectives
To review Validity and to define Internal and external validity.
To use this understanding to discuss issues of validity connected with different research methods.
To complete a table to help us revise.

20. Types of Validity Internal validity External Validity
What happens inside the study.
External Validity
How our results can be generalised beyond the study
Validity of psychological measures
How well our methods of measurement actually measure what we intend

21. Internal Validity Mundane realism Extraneous variables Task A and B
Situational variables
Participant variables
Investigator effects
Demand characteristics
Participant effects
Task A and B
Name the variables
What are the EVs and how could they have been controlled?

22. External Validity
Fill in the gaps

23. Validity of psychological measures
Operationalisation of variables
Can result in loss of validity

24. Validity of psychological measures
Content validity
Does the method used actually seem to measure what you intended?
Concurrent validity
How well does the measure agree with existing measures?
Construct validity
Is the method actually measuring all parts of what we are aiming to test?
Predictive validity
Is our measure associated with future behaviour?

25. Validity of psychological measures
Complete tasks C, D and E.