1. Scientific Validation of Mixture Interpretation Methods 17th International Symposium on Human Identification Sponsored by the Promega Corporation October, 2006 Mark W Perlin, PhD, MD, PhD Pittsburgh, PA USA perlin@cybgen.com Cybergenetics © 2003-2006

2. Admissible Evidence FRE 702: reliable DataMethodApplication Frye (1923) General acceptance Daubert (1993) General acceptance Testable Error rate Peer review Standards

3. DNA Mixture Interpretation different labs, different methods same data, different profiles concordance study not possible – no direct profile comparison need general validation approach – for Frye & Daubert requirements

4. Inadmissible Mixture Evidence FRE 702: unreliable √ Data X Method X Application Frye & Daubert X General acceptance X Testable X Error rate X Peer review X Standards

5. Thompson

6. Forensic Comparison Phases 1.Infer DNA profile 2.Match with suspect 3.Relative frequency 4.Draw conclusions

8. 11 1213 D5S818 locus

9. Government Reviewer A Reviewer B Private Lab Reviewer A Reviewer B Computer TrueAllele Review Compare 5 Mixture Reviews Same data, different interpretations

10. Conservative: Government Review Avoid overcalling the results Report 0, 1 or 2 alleles uncertain data a b Allele 1: b Allele 2: anything

11. Aggressive: Private Lab Review Try ruling out unlikely combinations Report list of allele calls uncertain data a b Allele calls: 1. a b 2. b b

12. Objective: TrueAllele Computer Review Preserve match information Genotypes with probability abab cc acacbcbc? 1/2 genetic data a b abab bb 1/2

13. Government Review A 910111213 9 10 11 12 13 ** InferredKnown1212 13 12 11 10 9 131211109

14. Government Review A Known 910111213 9 10 11 12 13 **1212 Match Information (MI) = -log(1.00) = 0 Match Inferred 100% Relative frequency of inferred profile

15. Match Information Logarithm measures information match information (MI) = -log(LR) Match LR random match probability Prob(random match) Prob(specific match) = ~ relative frequency ~ Compare against the true profile

16. Government Review B 910111213 9 10 11 12 13 12*1212 Known Match Inferred Match Information (MI) = -log(0.50) = 0.3 50% Relative frequency of inferred profile

17. Private Lab Review A 910111213 9 10 11 12 13 12*1212 Known Match Inferred Match Information (MI) = -log(0.50) = 0.3 50% Relative frequency of inferred profile

18. 910111213 9 10 11 12 13 12121212 Known Match Private Lab Review B 1213 Inferred Match Information (MI) = -log(0.20) = 0.7 20% Relative frequency of inferred profile

19. 910111213 9 10 11 12 13 12121212 Known Match Inferred TrueAllele ® Computer Review Match Information (MI) = -log(0.10) = 1 10% Relative frequency of inferred profile

20. Profile Match Information Add up the match information (MI) from every locus in the profile

22. Scientific Validation of Mixture Interpretation One mixture, four different answers Accuracy Precision Reproducibility

23. Accuracy Digging in the right place? Is it the true profile?

24. Accuracy Digging in the right place? Is it the true profile? average

25. Precision How deep can we go? How close to true profile?

26. Precision How deep can we go? How close to true profile? average difference

27. Reproducibility Do it again the same way? How repeatable is our answer?

28. Reproducibility Do it again the same way? How repeatable is our answer? standard deviation

29. Validation Results Government11.91-5.410.140 AccuracyPrecisionReproducibility

30. Private Lab Data

31. Validation Results Government11.91-5.410.140 Private Lab15.93-1.390.833 AccuracyPrecisionReproducibility Tradeoff?

32. TrueAllele Data

33. Validation Results Government11.91-5.410.140 Private Lab15.93-1.390.833 TrueAllele17.24-0.090.095 AccuracyPrecisionReproducibility

34. Scientific Validation of Mixture Interpretation Methods law requires admissible evidence mixture interpretation not validated match information (MI) comparison simple spreadsheet calculation – accuracy, precision, reproducibility