1. Up to 33% of total A level from Research Methods and Statistics
Lesson 5
Research Methods and Statistics
- Unit 2 –
Also questions in Unit 1 & 3
Up to 33% of total A level from Research Methods and Statistics
10% of total A level from applying maths skills

2. Starter 1 on BWBs – Sampling techniques
Have a discussion in your groups about which two sample techniques would have to be thrown out in a very heavily overladen hot air balloon. Put these on the BWBs with full justification

3. Starter 2 on BWBs – Reliability
A) In pairs, discuss…
Reliability and why we need it in psychological research
Replicability and the role of standardisation
Inter-rater reliability and how to test for it
B) Write on a mini whiteboard, between 1-5 (1 = not at all, 5 = very well) how much you understand the concept of reliability. Don’t worry if you put a low number down. It takes time to understand things in psychology.

4. Starter 2a on BWBs – Reliability
If you put 4 or 5, complete task X. If you put 3, 2, or 1, complete task Y
Task X
Pair up with other 4 and 5’s
consider how you would answer this question
Researchers found that participants’ cognitive performance was better after the consumption of energy drinks. How would they test whether this was a reliable finding? (3 marks)
Write an answer on a BWB
Task Y
Studies into cognitive performance at a sixth form college were replicated after an initial finding that people performed better after consuming energy drinks and after a good night’s sleep
After 7 replicated studies, which findings were reliable?
So what conclusions ae you going to make about the consumption of energy drinks?
What conclusions are you going to make about the benefits of a good night’s sleep?
Studies showing cognitive performance before and after energy drink consumption
Studies showing cognitive performance before and after over 8 hours continual sleep
Study 1
Better
Study 2
No difference
Study 3
Study 4
Worse
Study 5
Study 6
Study 7

5. Starter 2b - Consolidate: Those who did task Y read the answers from task X. Those who completed task X complete the questions from task Y
Task X
Pair up with other 4 and 5’s
consider how you would answer this question
Researchers found that participants cognitive performance was better after the consumption of energy drinks. How would they test whether this was a reliable finding? (3 marks)
Write an answer on a BWB
Task Y
Studies into cognitive performance at a sixth form college were replicated after an initial finding that people performed better after consuming energy drinks and after a good night’s sleep
After 7 replicated studies, which findings were reliable?
So what conclusions ae you going to make about the consumption of energy drinks?
What conclusions are you going to make about the benefits of a good night’s sleep?
Studies showing cognitive performance before and after energy drink consumption
Studies showing cognitive performance before and after over 8 hours continual sleep
Study 1
Better
Study 2
No difference
Study 3
Study 4
Worse
Study 5
Study 6
Study 7

6. Model answer:
Researchers found that participants’ cognitive performance was better after the consumption of energy drinks. How would they test whether this was a reliable finding? (3 marks) Researchers would have standardised the procedures, apparatus and instructions to carry out a replication (1 mark). After replication, they would compare the results from the previous study to see if they were strongly correlated (1 mark) If the strong correlation existed, the researchers would confidently claim that their findings were reliable, with confidence increasing after repeated replications showing similar outcomes. (1 mark)

7. Do not get reliability and validity mixed up
Do not get reliability and validity mixed up! They are different concepts

8. Still unsure? More on validity next
Which is which?
Validity Reliability
The concept in research that the researchers are testing what they set out to test. Internal refers to the concept that it is the IV that is causing changes in the DV (no confounding variables). External refers to how the findings relate to the rest of the world, other populations and other times.
The concept that there is consistency in the findings. If the research is repeated under similar conditions, the results should be the same as previously. Additionally, If more than one researcher is recording behaviour, there must be consistency between them.
Still unsure? More on validity next

9. Validity: A very important concept for experimentation
Internal validity
If the IV has caused the change in the DV, then an experiment has internal validity. If something other than the IV (a confounding variable) has caused a change in the DV, then the experiment is said to ‘lack internal validity’
Example of a ‘lack of internal validity’ from social psychology
Zimbardo asked participants to give electric shocks (fake) to participants when they answered a question incorrectly. Participants who had to set the level of shock, gave a higher level if they were dressed in white robes with a hood (anonymous), or if they were dressed in normal clothes with a name tag (identified). Those in the white robes gave higher shocks, and the psychologist concluded that anonymity caused people to act more violently. However, critics pointed out that the participants had unwittingly been dressed in what looked like KKK uniforms, and so the participants were actually role playing, rather than influenced by their anonymity.

10. Zimbardo’s flawed experiment lacked internal validity
What was Zimbardo’s IV?
The anonymity or identity made public of the participant
What didn’t he control?
the power of conformity to roles (people felt more hostile because they were in a KKK uniform) or the power of demand characteristics (people felt they should give higher shocks as they thought that was what the experimenter wanted them to do, dressed in a KKK uniform). Either way, this means that the IV did not cause changes in the DV so it lacked internal validity
Another researcher showed this. The participant who was anonymous was dressed as a nurse with a hygienic face mask to hide their identity.
Guess what happened?
The participants, although anonymous gave lower electric shock levels than those identifiable dressed in civilian clothes. Confirming criticism of Zimbardo’s original experiment that it lacked internal validity

11. Our Experiment
Can you think of anything that compromises the internal validity of our study? Think first, then hints will appear to help
Demand Characteristics (characteristics in the study that demanded a specific answer, other than the IV)
Investigator effects (was there something that you did that could have impacted on the participants’ behaviour?)
Participant variables (was the fact that we used different participants in our group an issue?)
Any thing else?

12. More about Validity
External validity This questions the validity of the experiment outside the research situation and whether participants would behave in a similar way. Ecological validity is a specific type of external validity, and relates to whether findings can be generalised to naturally occurring settings
Example of a ‘lack of external validity’ from cognitive psychology
Loftus asked participants to wait in a room before an experiment was due to take place. During this period, participants either heard an argument, and saw a person run past with a knife in their hands, or they heard an argument, and saw a person run past with a pen in their hand. The results suggested that the anxiety levels of the participants lead them to focus on the weapon, and thus not give an accurate identification of the suspect. However, in real life, participants who are reportedly very anxious in real life crime situations, tend to be much more able to give valid details about the crime that took place. Thus her laboratory study lacks ecological validity.

13. Validity continued
External validity: Temporal (time) and population validity are also types of external validity
Many psychological studies are no longer thought to represent society because they were conducted many years ago. For example, a lot of research into conformity and obedience was conducted the 1950s and 1960s. People’s attitudes to groups and authority have changed since this research, so the temporal validity of such research can be questioned.
Likewise, the target population and sample used is likely to be narrow. Therefore, legitimate questions are raised about the population validity of the studies. For example, In a study by Machery, 2010, it was found that between 2003 and 2007, 96 per cent of psychological samples came from countries with a total of only 12 per cent of the world's populations. The U.S. alone provided nearly 70 per cent of these samples. We must not conclude that just because people in the USA (or anywhere) behave in a certain way, then others will think and do the same.

14. Our Experiment
With the examples on the BWBs, or anything else, can you think of anything that compromises the External validity of our study? Think first, then hints will appear to help
Target population
Sample technique
Mundane realism
Anything else?

15. Not all bad though?
What strengths did our experiment have? Put as many as you can on the BWBs
Answers:
Standardised procedures
Random allocation to conditions
Standardised instructions
Clear materials
Didn’t use psychology students (or did you)?
Ethically sound? Consent form and debriefing
Anything else?
Consider whether these are issues related to validity, reliability, both or neither.

16. POLL: Do you think that our study was valid enough to stand by our earlier conclusions or not? Definitely Probably Unsure Unlikely Not at all

17. Experimental design
Common error by students They think this means what type of experiment was used. Wrong! or They think this refers to the procedure of the experiment. Wrong!

18. Experimental design
This refers to how the participants are used in experiments. There are three types of experimental design
From your prep work, can you name them?
Independent groups design
Repeated measures design
Matched pairs design

19. Which is which? (Between repeated measures and independent groups design)
A different group of participants do each condition
The same group of participants do each condition
Independent groups design
Repeated measures design

20. Matched-pairs design Common error by students
“ a matched pairs design is when there are different participants in each group, but the groups have the same characteristics”
Noooooooooooooo!

21. Matched-pairs design
Define what is meant by a matched pairs design on MWBs
Each participant in one condition is paired with another participant in the other condition on specific, relevant characteristics.
For example, if one participant in a control condition is male, from the socio-economic band of ‘lower supervisory and technical occupations’, single, with an IQ in the top 15% of the population. Then a participant in the experimental condition should have the same (or very similar) characteristics.
Identify a practical problem with this?
How many characteristics do we match the participants? How do we know these are the correct characteristics to match? How matched are they really?

22. Experimental design: In pairs quickly prepare… if not enough pairs, those who are given the ‘define’ tasks could work alone (nos. 1, 5 and 9)
Independent groups design: define
Independent groups design: explain advantage (reduces demand characteristics)
Independent groups design: explain a disadvantage (participant variables)
Independent groups design: explain random allocation of participants to conditions as a way of dealing with participant variables
Repeated measures design: define
Repeated measures design: explain advantage (reduces participant variables)
Repeated measures design: explain a disadvantage (increases risk of DV influenced by demand characteristics)
Repeated measures design: explain counterbalancing as a way of dealing with demand characteristics
Matched pairs design: define
Matched pairs design: explain an advantage
Matched pairs design: explain disadvantage
Matched-pairs design: using identical twins as a way of dealing with disadvantage

23. Now collate your work on the BWBs under three headings
Independent groups design (groups 1 – 4) Repeated measures design (groups 5 - 8) Matched-pairs design (groups 9 – 12)

24. Our experiment
Q1) Which experimental design did you use? Q2) What was an advantage of this in relation to your study? Q3) What was a disadvantage of this in relation to your study? Q4) How did you deal with this disadvantage? (what and how) Q1) Independent groups design Q2) reduction in the effects of demand characteristics as participants were naive about the IV. They did not know that the other condition was either reflecting alone or were given information about the answers on a sheet Q3) participant variables. It maybe that a higher proportion of people in the experimental group had personality characteristics making them more likely to conform Q4) random allocation. Researchers tossed a coin to determine who was in the control condition, and who was in the experimental condition

25. Next lesson is the mini-mock
Next lesson is the mini-mock. Revise all the research methods and statistics that we have covered in the first two weeks The test will be out of 24. There will be some text about an experiment and questions about this text. Be extremely precise in our answers, and read both the text and the question very carefully Also begin preparing the week’s prep work for the first unit 1 topic. Social influence. See Unit 1 Social Influence (on home page) Prep 1