Teaching Probability and Statistics to Law Students Philip Dawid University College London TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A AA

Ll.M. Evidence and Proof Mixture of first-time and mature law students Optional module, students –largely given over to Wigmorean Analysis Since ~ 1984: 4 £ 2hr sessions on “Probability and Proof” Arose from public lecture –Twining

Outline 1.Pitfalls of Statistical Evidence 2.Conceptions of Probability 3.The Mathematics of Pascalian Probability 4.Probability and Statistics in Court

Main Point:

1. Pitfalls of Statistical Evidence

Example Data: A survey has shown that 80% of all alcoholics were bottle ‑ fed as babies. Inference: A bottle ‑ fed baby has a high chance of becoming alcoholic.

QUESTIONS –What additional information would be needed to provide a suitable comparison for the interpretation of the figures given? –Is the need for such additional information weakened by the fact that 80% is such a high proportion? –What can be said, on the basis of the data, about the chance that a bottle ‑ fed baby will become alcoholic? –What could be said if the additional relevant statistical information were also available? –Reconsider the above questions in the light of alternative data: “99% of all alcoholics were fed on milk as babies,” and the derived infer­ence “A milk ‑ fed baby has a very high chance of becoming alcoholic.” –Identify other real instances of this kind of fallacious reasoning.

Example Table

AIDS example The enzyme linked immunosorbent assay has a false positive rate of about 1%. In Britain in 1985 the proportion of blood donors with infective acquired immune deficiency syndrome (AIDS) was around 1 in 10,000. The odds of having true AIDS with a positive test are therefore… 100:1 against

Tabular Solution PositiveNegativeTotal AIDS Non-AIDS Total10000

Tabular Solution PositiveNegativeTotal AIDS1 Non-AIDS Total10000

Tabular Solution PositiveNegativeTotal AIDS1 Non-AIDS9999 Total10000

Tabular Solution PositiveNegativeTotal AIDS11 Non-AIDS9999 Total10000

Tabular Solution PositiveNegativeTotal AIDS101 Non-AIDS9999 Total10000

Tabular Solution PositiveNegativeTotal AIDS101 Non-AIDS Total10000

Tabular Solution PositiveNegativeTotal AIDS101 Non-AIDS Total10000

Tabular Solution PositiveNegativeTotal AIDS101 Non-AIDS Total

Tabular Solution PositiveNegativeTotal AIDS101 Non-AIDS Total

2. Conceptions of Probability

Statistical Probability One of the most straightforward conceptions of probability is as a simple proportion in a suitable population or subpopulation

Marginal Probabilities

Conditional Probabilities

Joint Probabilities

We easily verify the following properties:

These properties may easily be seen to hold no matter what values are used as data in the above table. For any attributes A and B that may be possessed by the individuals in a population, statistical probabilities will always obey the following rules: 1.0  P(A)  1. 2.If A is possessed by all members of the population, P(A) = 1. 3.If A and B cannot occur together, P(A or B) = P(A) + P(B). 4.P(A and B) = P(A  B)  P(B).

These rules form the basis of the formal mathematical theory of probability. They apply to probabilities based on other interpretations than the purely statistical. To manipulate probabilities, no matter what their origin or interpretation, we can pretend that they have been formed as proportions in a population and its subpopulations.

Other “Proportion-Based” Conceptions Classical Probability (cards, dice,…) –proportions of “equally possible” cases Empirical Probability –(limiting) proportions in a sequence of outcomes Metaphysical Probability –proportions in hypothetical parallel universes

Epistemological Conceptions Subjective Probability –Bayes, Laplace,… Logical Probability –Carnap, Keynes, Jeffreys Non-Pascalian Probability –Belief functions –Fuzzy logic –Baconian logic

3. The Mathematics of Pascalian Probability

Axioms A1: A2: A3: If then

Conditional Probability Bayes’s Theorem: Generalized Addition Law:

Example L = “accused left town after the murder” G = “accused is guilty of the murder.” Assess Want –also need –say

Tabular Method

Simplification Only need probabilities (under the various hypotheses considered) for the ACTUAL evidence presented –LIKELIHOOD

Bayes’s Theorem POSTERIOR ODDS = PRIOR ODDS  LIKELIHOOD RATIO

4. Probability and Statistics in Court

Identification Evidence DNA match M “Random match probability” P –e.g. 1 in 10 million Assume suspect would match if guilty What is the strength of the evidence?

Prosecutor’s Argument The probability of the observed match between the sample at the scene of the crime and that of the suspect having arisen by innocent means is 1 in 10 million With a probability overwhelmingly close to 1, the suspect is guilty.

Defence Argument There are ~ 30 million individuals who might possibly have committed this crime. One of these is the true culprit. Of the remaining 30 million innocent individuals, each has a probability of 1 in 10 million of providing a match to the crime trace. So we expect about 3 innocent individuals to match. In the whole population we expect 4 matching individuals: 1 guilty, and 3 innocent. We know that the suspect matches, but that only tells us is that he is one of these 4. So the probability that he is guilty is only 0.25.

Bayesian Argument Posterior odds = Prior odds £ 10 million Prior odds 1 ) Prosecution Prior odds 1 in 3 million ) Defence

Adams Case Rape M: DNA match B 1 : Victim did not recognise suspect B 2 : Suspect had alibi ~ 200,000 potential perpetrators

Illustrative Analysis

Legal Process Convicted Appeal: –The task of the jury is to “evaluate evidence and reach a conclusion not by means of a formula, mathematical or otherwise, but by the joint application of their individual common sense and knowledge of the world to the evidence before them” Appeal granted on the basis that the trial judge had not adequately dealt with the question of what the jury should do if they did not want to use Bayes’s theorem.

Retrial –“Bearing in mind that Mr Adams differed from Ms Marley’s description of her assailant and the fact that she failed to pick him out at the identity parade, what is the probability that she would say at the magistrates’ hearing that he did not look like her assailant, if he had not been her assailant?” Reconvicted Second Appeal – It might be appropriate for the jury “to ask themselves whether they were satisfied that only X white European men in the UK would have a DNA profile matching that of the rapist who left the crime stain… It would be a matter for the jury, having heard the evidence, to give a value of X.” Appeal dismissed Legal Process

What is X? Expected number of innocent individuals matching the crime DNA is 1/10. Likelihood ratio from the other evidence is 1/18. So count Adams as only 1/18 of an individual. Probability Adams, rather than anyone else, is the guilty party is OK for jury ???

Current (UK) Position Bayesian arguments, while not formally banned, need to be presented with a good deal of circumspection. Reexamination of role and use of expert evidence.