1. F O C U S

2. Autonomy and the Practice of Forensic Science R. E. Gaensslen Director of Graduate Studies Forensic Science Group University of Illinois at Chicago ASLME DNA and Civil Liberties Workshop 2, Sept 2004, Boston

3. Laboratory Autonomy Operational Forensic Science Laboratories Does the administrative housing of the laboratory contribute to creation, or prevention, of intrinsic bias or error in testing and / or interpretation ?? Law Enforcement Department of Health Medical Examiner Independent Commercial

4. ERROR / BIAS Systemic: > intrinsic to the lab / organization > independent of individual examiner Individual: > intrinsic to the person > independent of the lab / organization (?) Intentional / Unintentional

5. The Los Angeles Times Sunday, August 21, 1994 Scores of Convictions Reviewed as Chemist Faces Perjury Accusations Forensics: Fred Zain's expert testimony and lab tests helped put scores of rapists and murderers behind bars. But college transcript shows he flunked some chemistry classes and barely passed others. He is also accused of evidence-tampering. by SAU CHAN, ASSOCIATED PRESS Crime lab chemist goes on trial for alleged fraud By The Associated Press CHARLESTON, W.Va. At one time, Fred Zain was a prosecutor's dream: a respected crime lab chemist with a compelling courtroom demeanor whose testimony helped convict hundreds of people. The trouble, authorities now say, is that much of what Zain had to say was questionable at best, or outright lies at worst. Allegations of individual bias - apparently intentional

6. Allegations of individual error – motivation less evident

7. … questions about her work serve as a reminder of the grave harm that a single person in the criminal justice system apparently can cause – either through malice or incompetence – if the rest of the system offers little more than malign neglect.

8. USDOJ/OIG Special Report The FBI Laboratory: An Investigation into Laboratory Practices and Alleged Misconduct in Explosives-Related and Other Cases (April 1997) Allegations of organizational bias, or at least neglectful leadership The Chicago Police Dept Laboratory displayed serious deviations from established crime laboratory standards for care of crime laboratories in 1986.

9. A fraction of – but not all – these cases were handled by the Chicago Police Department, and the laboratory work was done in the CPD Laboratory The day before leaving office, Illinois Gov. George Ryan commuted the sentences of everyone on death row.

10. Tests, procedures, protocols themselves Results Sources of Error / Bias Recording, transposing data Specimen mixups Interpretation

11. RESULTS INTERPRETATION

12. RESULTS 1. Raw, unfiltered data 2. “Interpreted” raw data

13. INTERPRETATION 1 Of the test results in terms of what is known about the science 2 Of the first level interpretation in terms of the case or situation at hand

14. RESULTS 1 Raw, unfiltered data 2 “Interpreted” raw data INTERPRETATION 1 Of the test results in terms of what is known about the science 2 Of the first level interpretation in terms of the case or situation at hand

15. Subjective Objective

16. LEVELS OF CERTAINTY in Match / Inclusion Cases Is Could Be Estimate of Probability / Likelihood Could Be Frequency Estimate Could Be / Consistent With

17. Sins of Commission / Sins of Omission All the professional organizations have codes of ethics, but there is not always widespread agreement in the profession about exactly what constitutes an ethical violation Debate often centers on exactly how much responsibility rests on the scientist / expert vs. how much lies elsewhere, e.g. with the prosecutor or defense attorney, with the court, with the jury, etc. It is probably harder to get consensus about “sins of omission” being unethical than it is about less equivocal “sins of commission”

18. Strategies for Avoiding / Detecting / Correcting Error or Bias QA - QC Lab Audits – internal / external Proficiency testing – open or blind Testimony audits Accreditation guidelines and goals include many of these strategies Criminalist / analyst certification Reanalysis by opposing side experts

19. Should analysts be blinded? To perform their functions properly, criminalists have to know what the questions are Avoiding / Detecting / Correcting Error or Bias Choices / judgments are part of the examination

20. Effect of the laboratory administrative setting Real / Perceived Explicit / Subtle (the “culture”) Avoiding unintended consequences of change Teasing out the laboratory’s agenda