1. Slide 1 UCL JDI Centre for the Forensic Sciences 21 March 2012 Norman Fenton Queen Mary University of London and Agena Ltd norman@agena.co.uk Bayes and the Law

2. Slide 2 R vs Levi Bellfield, Sept 07 – Feb 08

3. Slide 3 R v Gary Dobson 2011 Stephen Lawrence

4. Slide 4 Questions What is 723539016321014567 divided by 9084523963087620508237120424982? What is the area of a field whose length is approximately 100 metres and whose width is approximately 50 metres?

5. Slide 5 Court of Appeal Rulings “The task of the jury is to evaluate evidence and reach a conclusion not by means of a formula, mathematical or otherwise, but by the joint application of their individual common sense and knowledge of the world to the evidence before them” (R v Adams, 1995) “..no attempt can realistically be made in the generality of cases to use a formula to calculate the probabilities... it is quite clear that outside the field of DNA (and possibly other areas where there is a firm statistical base) this court has made it clear that Bayes theorem and likelihood ratios should not be used” (R v T, 2010)

6. Slide 6

7. Slide 7 Revising beliefs when you get forensic evidence Fred is one of a number of men who were at the scene of the crime. The (prior) probability he committed the crime is the same probability as the other men. We discover the criminal’s shoe size was 13 – a size found in only 1 in a 100 men. Fred is size 13. Clearly our belief in Fred’s innocence decreases. But what is the probability now?

8. Slide 8 Are these statements equivalent? the probability of this evidence (matching shoe size) given the defendant is innocent is 1 in 100 the probability the defendant is innocent given this evidence is 1 in 100 The prosecution fallacy is to treat the second statement as equivalent to the first The only rational way to revise the prior probability of a hypothesis when we observe evidence is to apply Bayes theorem.

9. Slide 9 Fred has size 13

10. Slide 10 Imagine 1,000 other people also at scene Fred has size 13

11. Slide 11

12. Slide 12 Fred is one of 11 with size 13 So there is a 10/11 chance that Fred is NOT guilty That’s very different from the prosecution claim of 1%

13. Slide 13 Ahh.. but DNA evidence is different? Very low random match probabilities … but same error Low template DNA ‘matches’ have high random match probabilities Principle applies to ALL types of forensic match evidence Prosecutor fallacy still widespread

14. Slide 14 Bayes Theorem E (evidence) We now get some evidence E. H (hypothesis) We have a prior P(H) We want to know the posterior P(H|E) P(H|E) = P(E|H)*P(H) P(E) P(E|H)*P(H) P(E|H)*P(H) + P(E|not H)*P(not H) = 1*1/1001 1*1/1001+ 1/100*1000/10001 P(H|E)  = = 0.000999 0.000999 + 0.00999 0.091

15. Slide 15 Determining the value of evidence –Prosecution likelihood (The probability of seeing the evidence if the prosecution hypothesis is true) (=1 in example) Posterior Odds = Likelihood ratio x Prior Odds Bayes Theorem (“Odds Form”) Likelihood ratio = Prosecutor likelihood Defence likelihood (=100 in example) –Defence likelihood (The probability of seeing the evidence if the defence hypothesis is true) (=1/100 in example)

16. Slide 16 Likelihood Ratio Examples Prosecutor 1 100 25 X = Defence 4 1 1 Prior oddsLikelihood ratioPosterior Odds Prosecutor 1 100 1 X = Defence 10001 10 LR > 1 supports prosecution; LR <1 supports defence LR = 1 means evidence has no probative value

17. Slide 17 Example: Barry George H: Hypothesis “Barry George did not fire gun” E: small gunpowder trace in coat pocket Defence likelihood P(E|H) = 1/100 … …But Prosecution likelihood P(E| not H) = 1/100 So LR = 1

18. Slide 18 R v T Good: all assumptions need to be revealed Bad: Implies LRs and Bayesian probability can only be used where ‘database is sufficiently scientific’ (e.g. DNA) Ugly: Lawyers are ‘erring on side of safety’ – avoiding LRs, probabilities altogether in favour of vacuous ‘verbal equivalents’

19. Slide 19 But things are not so simple…

20. Slide 20 But things are not so simple…

21. Slide 21 But things are not so simple…

22. Slide 22 The problems ‘Simple’ Bayes arguments do not scale up in these ‘Bayesian networks’ Calculations from first principles too complex …. But Bayesian network tools provide solution

23. Slide 23 Assumes perfect test accuracy (this is a 1/1000 random match probability)

24. Slide 24 Assumes false positive rate 0.1 false negative rate 0.01

25. Slide 25 Bayesian nets in action Separates out assumptions from calculations Can incorporate subjective, expert judgement Can address the standard resistance to using subjective probabilities by using ranges. Easily show results from different assumptions …but must be seen as the ‘calculator’

26. Slide 26 Bayesian nets: where we are Used to help lawyers in major trials Developing methods to make building legal BN arguments easier (with Neil and Lagnado): –set of ‘idioms’ for common argument fragments (accuracy of evidence, motive/opportunity, alibi evidence) –simplifying probability elicitation (mutual exclusivity problem etc) –tailored tool support Biggest challenge is to get consensus from other Bayesians

27. Slide 27 Conclusions and Challenges The only rational way to evaluate probabilistic evidence is being avoided because of basic misunderstandings But real Bayesian arguments cannot be presented from first principles. Use BNs and focus on the calculator analogy (argue about the prior assumptions NOT about the Bayesian calculations) How to consider all the evidence in a case and determine (e.g. as CPS must do) the probability of a conviction

28. Slide 28 A Call to Arms Bayes and the Law Network Transforming Legal Reasoning through Effective use of Probability and Bayes https://sites.google.com/site/bayeslegal/ Contact: norman@eecs.qmul.ac.uknorman@eecs.qmul.ac.uk Further reading: Fenton, N.E. and Neil, M., 'Avoiding Legal Fallacies in Practice Using Bayesian Networks', Australian Journal of Legal Philosophy 36, 114-151, 2011 www.eecs.qmul.ac.uk/~norman/papers/fenton_neil_prob_fallacies_June2011web.pdf Fenton, N.E. and Neil, M., 'On limiting the use of Bayes in presenting forensic evidence' www.eecs.qmul.ac.uk/~norman/papers/likelihood_ratio.pdf