Problems in Research Poor design decisions Researcher bias Researcher effects on participants Things participants may do regardless psychlotron.org.uk

Poor Design Problems that stem from a badly designed study Poor operationalisation of variables Inadequate control of variables Threaten the internal validity of the study Cause reliability problems psychlotron.org.uk

Poor Design Poor operationalisation Insufficient care when translating unobservable psychological processes into observable behavioural ones Can end up measuring the wrong thing Can generate false positives or false negatives psychlotron.org.uk

Poor Design Reliability – the extent to which a study or measuring tool produces consistent results Test-retest reliability Inter-observer reliability Reliability is maximised by (1) careful operationalisation; (2) adequate control of variables NB: a reliable study can still be invalid psychlotron.org.uk

Poor Design Inadequate controls Extraneous variables should be held constant Most are not a problem (random effect) Those that have a systematic effect (e.g. affect one condition more than another) become confounding variables psychlotron.org.uk

Researcher Bias Emphasising some things at the expense of others Self-confirmatory bias – tendency for researchers to pay more attention to data that confirm their expectations Reduce the need for subjective judgements by (1) careful operationalisation; (2) use of double blind psychlotron.org.uk

Researchers & Participants Researcher effects - behaviour of researcher can affect that of participants: Intimidate (anxiety) Insult (motivation) Cue PPs to behave in particular ways Can be reduced through standardisation of scripts and procedures psychlotron.org.uk

Researchers & Participants Interviewer effects – characteristics of the researcher can affect how PP relates to them; hence their responses Gender Social class Age Ethnicity… Difficult to predict, impossible to eliminate psychlotron.org.uk

Participants… …are people too; people try to make sense of their situation…how they do this is not always predictable. Attempts to please (or displease) the researcher Management of self-presentation psychlotron.org.uk

Participants Respond to the fact that they are being studied Evaluation anxiety Feelings about the study & the researcher Effects on motivation – please the researcher or confound the results; depends on how researcher is perceived psychlotron.org.uk

Participants May respond to demand characteristics (cues); these can enter the research: Through poor design e.g. allowing PPs to be aware of both conditions in an expt Through researcher behaviour e.g. unwittingly indicating desired behaviour Can be reduced through careful choice of design; standardisation; double blind psychlotron.org.uk

Participants Generally try to manage how they are perceived by others (we all do); can lead to: Socially desirable responses Other distortions of truth in order to manage impressions (how depends on who and why) Can be reduced through anonymity, confidentiality but rarely eliminated psychlotron.org.uk

General Precautions Start with the aim, not the procedure Pay attention to choice of design, controls, operationalisation etc. Pilot all procedures and materials Use double blind procedures where appropriate & possible psychlotron.org.uk