The impact of discredited evidence David Lagnado Nigel Harvey Evidence project, UCL

Discredited evidence How do people revise their beliefs once an item of evidence is discredited?  For example, in a murder trial, when the testimony of a key witness is shown to be fabricated, how does this affect juror’s beliefs about the testimony of other witnesses, or even other forensic evidence?  OJ Simpson trial

Normative Models Bayesian network models  Normative model for combining probabilistic evidence  E.g., forensic DNA evidence; paternity cases (Dawid)  Formal modelling of ‘manipulated evidence’ (Baio)  Starting to be applied to crime cases But a lot depends on network construction No ‘normative’ method for this?

Crime case (Leucari, 2005)

Explaining away (Pearl, 1988) S S = Suspect commits crime P(S|C) > P(S) Finding out C raises probability of S C C = Suspect confesses F F = Police force confession P(S|C&F) < P(S|C) Finding out F too lowers the probability of S Despite its simplicity and ubiquity, this pattern of inference is hard to capture on most psychological models of inference (e.g., associative models, connectionist, mental models, mental logic)

Psychological models Belief-adjustment model  Hogarth & Einhorn (1992) Story model  Pennington & Hastie (1981, 1988, 1992, 1993)

Belief adjustment model For Evaluation tasks Evidence encoded as +ve or –ve relative to hypothesis Adding model S k = S k-1 + w k s (x k ) S k = degree of belief in hypothesis given k items of evidence S k-1 = prior opinion S (x k ) = subjective evaluation of kth item (-1 ≤ s (x k ) ≤ +1) w k = adjustment weight (0 ≤ w k ≤1)

Online vs. Global Processing Two processing modes Online (step-by-step)  Belief adjusted incrementally with each item of evidence Global (end-of-sequence)  Belief adjusted by aggregate impact of all items S k = S 0 + w k [s (x 1,…, x k )]

When is each process used? OnlineGlobal OnlineAll tasks Complex items and/or long series GlobalImpossible Simple items and short series PROCESS RESPONSE MODE Processing load - Aggregation can be costly in terms of mental resources whereas step-by-step integration makes minimal demands

Evidence for Belief Adjustment Order effects in online processing  Evaluation mode (adding not weighted average)  None with consistent evidence (e.g, ++ or --)  Recency with mixed evidence (-+ > +-) over-weight last item  Supported in Exps 1-5 Model is quite flexible – designed to account for rich patterns of primacy and recency evidence But does not address relations between evidence items (assumes independence?)

Story model Evidence evaluated through story construction Stories involve network of causal relations between events Causal narratives not arguments  People represent events in the world, not inference process Stories constructed prior to judgment or decision Stories determine verdicts, and are not post hoc justifications

Evidence for story model Verbal protocols  85% of events causally linked Verdicts covaried with story models Recognition memory tests  More likely to falsely remember items consistent with story model Story vs witness order  More likely to convict when prosecution evidence in causal order, defence in witness order, and vice-versa

Current experiments Investigate effect of discredited evidence Look at relations between items of evidence Do these modulate how people revise their beliefs?  Once an item of evidence is discredited, do people simply return to their prior level of belief?  Or does this change permeate their belief network? What factors affect this?  Relations between evidence  Order of presentation of evidence

HYPOTHESIS: Suspect S did it Scenario: House burglary, local suspect S apprehended EVIDENCE 1 Neighbour 1 says that S often loiters in area EVIDENCE 2 Neighbour 2 says S was outside house on night of crime Neighbour 2 is lying because he dislikes S ? P(S) Under-discounting Over-discounting

Generalisation When do people generalize from the discrediting of one item to other items? Dependent on relatedness of generating mechanisms? SAME  E.g. two statements from same neighbour SIMILAR  E.g. two statements from two different neighbours DIFFERENT  E.g., one statement and one blood test

HYPOTHESIS: Suspect S did it DIFFERENT Scenario: House burglary, local suspect S apprehended EVIDENCE 1 Footprints outside house match suspect’s EVIDENCE 2 Neighbour says S was outside house on night of crime Neighbour is lying because he dislikes S ? P(S) Under-discounting Over-discounting

Experiment 1 Each subject completes 12 problems (4 scenario types x 3 levels of relatedness)  ‘Relatedness’: SAME, SIMILAR, DIFFERENT Four probability judgments (of guilt) 1.Background information 2.Evidence 1 (E1) 3.Evidence 2 (E2) 4.Discredit evidence 2 (D2) Compare 2 and 4 (E2 vs. D2)  If D2 > E2 then under-discounting  If D2 < E2 then over-discounting

Example of SAME condition Background You are a juror on a murder case. The victim is a middle-aged woman and the suspect is her ex-husband. You will need to judge whether or not the suspect is guilty on the basis of several pieces of evidence. The woman was found stabbed at her home. She was fully clothed and the murder weapon, a knife, was present at the crime scene Evidence 1 The police have a statement from the current wife of the suspect, confessing that the suspect had previously revealed a desire for the victim to be dead Evidence 2 The same police station has a confession from the suspect, admitting that he killed the victim Discredit 2 The confession from the suspect was made under extremely pressured circumstances at the police station

Example of DIFF condition Background You are a juror on a murder case. The victim is a middle-aged woman and the suspect is her ex-husband. You will need to judge whether or not the suspect is guilty on the basis of several pieces of evidence. The woman was found stabbed at her home. She was fully clothed and the murder weapon, a knife, was present at the crime scene Evidence 1 Laboratory tests revealed that blood found at the crime scene matched the blood type of the suspect. Evidence 2 The police station has a confession from the suspect, admitting that he killed the victim Discredit 2 The confession from the suspect was made under extremely pressured circumstances at the police station

Exp 1: Results Significant ‘over’-discounting (D2 < E1) in all conditions  SAME: t(23)=3.74, p<0.05; SIM: t(23)=2.71, p<0.05; DIFF: t(23)=3.13, p<0.05 Amount of over-discounting greater in SAME vs. SIM, t(23)=1.91, p=0.07; no differences with SAME vs. DIFF, or SIM vs. DIFF Main effect of DIFF due to physical test as E1

Individual analysis Over-discount (D2 < E1) None (D2 = E1) Under-discount (D2 > E1)

Conclusions Difficult to interpret on either BA or Story model BA model  Does not predict effect of relatedness  Predicts recency effect with mixed evidence  ++- (overweight last item)  Asymmetric rebound effect? Story model  Predicts story construction only with global judgment  Does not predict over-discounting

Experiment 2 Better test of two models Order of evidence  LATE - discrediting info presented after both items B E1 E2 D  EARLY – discrediting info presented after first item B E2 D E1 Relatedness  SAME, DIFFERENT

HYPOTHESIS: Suspect S did it DIFFERENT & EARLY EVIDENCE 2 Neighbour says S was outside house on night of crime Neighbour is lying because he dislikes S EVIDENCE 1 Footprints outside house match suspect’s ? P(S)

Model predictions BA predicts recency  Final judgment for early > late  Because +-+ > ++- (overweight last item) Story model  predicts recency with online but not global  SAME ≠ DIFF for global condition (take account of relatedness)

Example of SAME and EARLY Background You are a juror on a murder case. The victim is a middle-aged woman and the suspect is her ex-husband. You will need to judge whether or not the suspect is guilty on the basis of several pieces of evidence. The woman was found stabbed at her home. She was fully clothed and the murder weapon, a knife, was present at the crime scene Evidence 2 The police station has a confession from the suspect, admitting that he killed the victim Discredit 2 The confession from the suspect was made under extremely pressured circumstances at the police station Evidence 1 The police have a statement from the current wife of the suspect, confessing that the suspect had previously revealed a desire for the victim to be dead

Example of DIFF and EARLY condition Background You are a juror on a murder case. The victim is a middle-aged woman and the suspect is her ex-husband. You will need to judge whether or not the suspect is guilty on the basis of several pieces of evidence. The woman was found stabbed at her home. She was fully clothed and the murder weapon, a knife, was present at the crime scene Evidence 2 The police station has a confession from the suspect, admitting that he killed the victim Discredit 2 The confession from the suspect was made under extremely pressured circumstances at the police station Evidence 1 Laboratory tests revealed that blood found at the crime scene matched the blood type of the suspect.

Online judgments: Early vs. Late Early condition –more sensitivity to relatedness NB no diff between B1 & U1 in EARLY rules out asymmetric rebound effect Late condition – less sensitivity to relatedness (but note that over-discounting (E1 > U1) only sig for SAME not DIFF)

Problematic for both models BA cannot explain early condition because does not consider relations between evidence Story model needs to be applied/adapted to online processing, and somehow explain difference between early and late Any other models? Needed: online model that takes relations between items into account, but can also explain early/late difference

Speculations Even with online processing people construct network fragments As evidence is accumulated it is compactly stored /integrated Natural to integrate items according to valence (+ve or –ve wrt hypothesis) E.g., group +ve evidence together

Late condition Positive evidence A and B integrated GUILT A + B + A&B + C discredits both A and B (irrespective of relatedness) C

Early condition DIFF B unaffected by C’s discredit of A GUILT A + B + C A + C

Early condition SAME GUILT A + A* A* discredited by C too (because similar to A) C GUILT A + C

Ongoing research Look at both witness and alibi statements  E.g., How does discrediting of an alibi affect evaluation of a positive witness?  Are there asymmetries in dealing with positive vs. negative evidence? More generally, look at positive and negative evidence (including forensic tests)  Are there differential affects of discrediting?  Can evidence integration idea explain these? Manipulate deception vs error