1. Complex legal requirements in electronic discovery search and retrieval: A NARA case study NSF Collaborative Expedition Workshop # 45 Advancing Information Sharing, Access, Discovery and Assimilation of Diverse Digital Collections Governed by Heterogeneous Sensitivities November 8, 2005 Jason R. Baron Director of Litigation Office of General Counsel National Archives and Records Administration

2. U.S. v. Philip Morris et al. Civil lawsuit brought by Clinton Administration against tobacco companies in 1999 RICO case – racketeering allegation that companies have conspired since 1953 to defraud the American public as to the true health effects of smoking Trial concluded in June 2005 – parties are awaiting decision from Judge Gladys Kessler

3. U.S. v. Philip Morris – E-discovery 1,726 Requests to Produce propounded by tobacco companies on U.S. (30 federal agencies, including NARA) for tobacco related records Along with paper records, email records were made subject to discovery 32 million Clinton era email records held by NARA

4. Tobacco Email Winnowing Process 20 million  200,000  100,000  80,000  20,000 email hits based relevant produced placed on records on keyword emails to opposing privilege terms used party logs  only a handful entered as exhibits at trial

5. Tobacco Case – NARA Search Terms Round One  tobacco  cigarette  smoking   nicotine  smokeless  Synar Amendment  Philip Morris  R.J. Reynolds  Brown and Williamson  BAT Industries  Liggett group

6. Tobacco NARA Search Terms (Con’t) Round Two (selected examples)  Liggett  PMI (Philip Morris Institute)  MSA (Master Settlement Agreement)  ETS (Environmental Tobacco Smoke)  B&W (Brown & Williamson)  TI (Tobacco Institute)

7. Tobacco Search Terms: False Positives Upper Marlboro, Maryland Presidential Management Intern (PMI) program Medical Savings Accounts or Metropolitan Standard Area (MSA) Educational Testing Service (ETS) Black & White photos (B&W) TI...

8. Tobacco Search String NARA Used For Round Two (((master settlement agreement OR msa) AND NOT (medical savings account OR metropolitan standard area)) OR s. 1415 OR (ets AND NOT educational testing service) OR (liggett AND NOT sharon a. liggett) OR atco OR lorillard OR (pmi AND NOT presidential management intern) OR pm usa OR rjr OR (b&w AND NOT photo*) OR phillip morris OR batco OR ftc test method OR star scientific OR vector group OR joe camel OR (marlboro AND NOT upper marlboro)) AND NOT (tobacco* OR cigarette* OR smoking OR tar OR nicotine OR smokeless OR synar amendment OR philip morris OR r.j. reynolds OR ("brown and williamson") OR ("brown & williamson") OR bat industries OR liggett group)

9. False Positives Relevant Smoking Policy Emails OMB VP Chief of Staff Ron Klain Office of the U.S. Trade Rep. White House Counsel

10. Real User Needs at NARA Finding Responsive Needles in Haystacks of Data Maximizing Recall of Responsive Docs Weeding out False Positives Evaluating competing search products in the marketplace against some objective standard lawyers will embrace

11. Opportunities & Avenues to Explore Future Research Exploring collaborative strategies amongst lawyers for the purpose of ensuring richer data sets responsive to queries Sponsoring challenge to private sector to have automated search methodologies compete based on objective criteria TREC 2006 Legal Track Future partnership opportunities with Sedona Conference legal community

12. Jason R. Baron Director of Litigation Office of General Counsel National Archives and Records Administration 8601 Adelphi Road Suite 3110 College Park, MD 20740 (301) 837-1499 Email: jason.baron@nara.gov