Forensic inference – is the law a ass? BA Festival of Science 6 September 2005 Forensic inference – is the law a ass? Ian Evett Forensic Science Service UK  The Forensic Science Service 2004

“If the law supposes that,” said Mr. Bumble... “the law is a ass – a idiot.” Charles Dickens: Oliver Twist ch 51.

The offender left a small quantity of cellular material... A crime has been committed The offender left a small quantity of cellular material... ...which has been DNA profiled Crime sample The two profiles match ...and he has been DNA profiled also Suspect sample A suspect has been arrested...

There are two key questions Crime sample Match Suspect sample What does this mean? There are two key questions Did the suspect leave the crime sample?

If I take two samples from the same person – do I expect them Crime sample Suspect sample Match Key question 1. If I take two samples from the same person – do I expect them to give the same DNA profile? It is almost certain that two samples from the same person would give the same DNA profile

*Because we inherit our DNA from our parents, Crime sample Suspect sample Match Key question 2. If I take samples from two different persons – do I expect them to have the same DNA profile? It is very unlikely that two unrelated* people would have the same DNA profile *Because we inherit our DNA from our parents, close relatives are much more likely to share features in their profiles

Match What does this mean? Crime sample Suspect sample Match What does this mean? If the suspect had left the crime sample, it is almost certain we would see a match If some other unknown person had left the crime sample, there is a very small chance that we would see a match

Did the defendant leave the crime sample? Suspect sample Match Did the defendant leave the crime sample? The probability of a match if some other person had left the crime sample is very small The probability of a match if the defendant had left the crime sample is very large Close to one e.g About one in a million

the Prosecutor’s Fallacy Crime sample Suspect sample Match A common mistake: the Prosecutor’s Fallacy The probability of a match if the crime sample had come from someone else is 1 in a million I see. There is a 1 in a million probability that the crime sample had come from someone else... Hence it is almost certain that the crime sample came from the defendant

The probability of a match if the crime sample had come from someone else is 1 in a million I see. There is a 1 in a million probability that the crime sample did come from someone else... The judge has made a serious mistake. This is known as the Prosecutor’s fallacy

If an animal is an elephant then there is a very high probability that it has four legs. Shorthand: Pr(Four legs | Elephant) = 0.9999

four legs then there is a very high SO!! - if an animal has four legs then there is a very high probability that it is an elephant. Pr(Four legs | Elephant) = 0.9999 Pr(Elephant | Four legs) = 0.9999 ???

I am interested in the probability that Crime sample Suspect sample Match I am interested in the probability that the crime stain was left by the defendant That depends not only on the scientific evidence, but also on other non-scientific evidence ..such as where the defendant was when the crime occurred

Match The court will hear non-scientific evidence. Such as: time, place, opportunity what the defendant says what the witnesses say It is in the light of that evidence that they must consider the evidence of the matching DNA Crime sample Suspect sample Match

In many aspects of our lives we update our uncertainty about something in the light of new information

I think there is a very good chance that the market in pickled horse chestnuts will go through the roof I think there is a good chance that the market in pickled horse chestnuts will collapse New research has shown that pickled horse chestnuts are a guaranteed cure for baldness

Prior New information Posterior

Non scientific evidence: time, place, opportunity what the defendant says what the witnesses say Prior belief DNA evidence Crime sample Suspect sample Match Posterior belief

in the light of new information Updating uncertainty in the light of new information Bayes’ Theorem Likelihood ratio Posterior Odds Posterior belief New information Prior Odds Prior belief = X

Did the defendant leave the crime stain? Or did someone else leave it? Crime sample Suspect sample Match Did the defendant leave the crime stain? Or did someone else leave it? Prior odds If the defendant had left the crime stain the probability of a match is approximately one If someone else had left the crime stain, the probability of a match is 1 in a million The match is a million times more probable if the defendant left the crime stain than if someone else left it Posterior belief

DNA match between profile from vaginal swab and Mr Adams R v. Dennis John Adams Rape: Hemel Hempstead DNA match between profile from vaginal swab and Mr Adams Match probability 1 in 200 million Victim said that Mr Adams did not look like the rapist Mr Adams had an alibi

The central problem in this case was that of combining statistically R v. Dennis John Adams The central problem in this case was that of combining statistically based scientific evidence in support of the prosecution... ...with non scientific evidence in support of the defence

To cut a long story short... R v. Dennis John Adams To cut a long story short... ...defence argued that the problem should be approached by inviting the jury to quantify their assessment of the non- scientific evidence ...then combining all of the various elements by means of Bayes’ theorem There were two trials and two appeals.

... we have very grave doubts as to whether R v. Dennis John Adams ... we have very grave doubts as to whether the evidence relating to Bayes’ Theorem was properly admissible, Judgment: First appeal ...because it trespasses on an area peculiarly and exclusively within the province of the jury, namely the way in which they evaluate the relationship between one piece of evidence and another.

...the attempt to determine guilt or innocence R v. Dennis John Adams Judgment: First appeal ...the attempt to determine guilt or innocence on the basis of a mathematical formula, applied to each separate piece of evidence, is simply inappropriate to the jury’s task. Jurors evaluate evidence... by the joint application of their individual common sense and knowledge of the world...

...the jury ...would have to ask themselves R v. Dennis John Adams Judgment: Second appeal ...the jury ...would have to ask themselves whether they were satisfied that only X white European men in the UK would have a DNA profile matching that of the rapist who left the crime stain It would be a matter for the jury, having heard the evidence, to give a value of X.

The probability of finding that profile Dennis Adams judgment Crime sample Suspect sample Match The probability of finding that profile if the crime stain had come from someone else is 1 in 200 million How many adult white males in the UK would have that profile? Approximately one tenth of a man

straightforward, isn’t it? R v. Dennis John Adams It would be a matter for the jury, having heard the evidence, to give a value of X. Well, that’s pretty straightforward, isn’t it? X is approximately one tenth of a man They would then have to ask themselves whether they were satisfied that the defendant was one of those men.

...consideration of the case along the lines R v. Dennis John Adams ...consideration of the case along the lines indicated would, in our judgment, reflect a normal course for a properly instructed jury to adopt.... It is the sort of task which juries perform every day. This appears to be a deliberate instruction for juries to reason irrationally

Doheny and Adams judgment Crime sample Suspect sample Match When the scientist gives evidence it is important that he should not overstep the line which separates his province from that of the jury He will properly explain to the Jury the nature of the match... He will...give the Jury the random occurrence ratio – the frequency with which the matching DNA characteristics are likely to be found in the population at large

Doheny and Adams judgment Crime sample Suspect sample Match Doheny and Adams judgment The scientist should not be asked his opinion on the likelihood that it was the Defendant who left the crime stain, ...nor when giving evidence should he use terminology which may lead the jury to believe that he is expressing such an opinion

Either: the defendant left the crime stain Crime sample Suspect sample Match Either: the defendant left the crime stain Or: some unknown person left the crime stain The evidence is a million times more probable if the first proposition is true This is extremely strong support for the first proposition That statement is counter to the Doheny/Adams judgment.

Either: the defendant wrote the questioned handwriting Or: some unknown person wrote it. In my opinion, there is extremely strong support for the first proposition Thank you, Dr Bloggs, that is very helpful.

Thank you, Mr Holmes - that is In my opinion, the crime mark and control print were made by the same person. Thank you, Mr Holmes - that is amazingly helpful.

If an expert is unable to carry out a statistical The present position If an expert is unable to carry out a statistical assessment of the weight of the evidence* then courts allow him to express an opinion of identity of source If an expert is able to carry out a statistical assessment of the weight of the evidence** then courts will not allow him to express an opinion of identity of source - the statistic is put to the jury * e.g. fingerprints, handwriting, toolmarks, footwear ** DNA profiling

It is our hope, as forensic scientists, that expert opinions will increasingly be based on databases and sound statistical methods The DNA experience suggests that if the weight of evidence can be measured by a numerical statistic.... ...then it is the statistic that should be put to the jury, rather than an expert opinion of identity of source

...then it is the statistic that should be put to the jury, rather than an expert opinion of identity of source Most forensic scientists have no problem with this view... Nevertheless, it would seem desirable that we should explore with the judiciary and the legal profession ways of presenting statistical evidence... ...in a manner that is balanced, robust and logical

The most rational compromise would be to explain the meaning of the likelihood ratio And present illustrations based on a range of priors agreed by the court

The probability that the defendant Crime sample Suspect sample Match The evidence is a million times more probable if the defendant left the crime stain than if some unknown person left it. The probability that the defendant left the crime stain depends on this evidence and also all of the other evidence

Crime sample Suspect sample Match I can illustrate the weight of evidence associated with the DNA match by some illustrative examples

? Non-scientific evidence If a person thought, based on the non-scientific evidence that the probability that the defendant left the crime sample is very small....say a one in a thousand Crime sample Suspect sample Match A match is a million times more probable if he left it than if some unknown person left it ...the effect of the DNA evidence is to turn that into a 99.9% probability that the defendant left the crime sample

Several illustrations could be presented, according to the requirements of the court This would set the DNA evidence into the context of all the other evidence in a balanced, robust and logical manner.

So – is the law a ass?

Don’t ask me folks!