The Team Players and Play in a Complex Document Review Project: Past, Present and Future Ralph C. Losey, Jackson Lewis Principal and National e-Discovery Counsel. Jackson Lewis, P.C., Labor and Employment firm with over 800 attorneys located in 56 offices around the country. Supervises and develops standardized best practices. Sub-Specialty is the use of “Active Machine Learning” to assist in search for relevant evidence. Ralph C. Losey Jackson Lewis Phone: (407) 496-2108 (cell) Email: ralph.losey@jacksonlewis.com Author, Six Books on e-Discovery by ABA and West-Thompson. Creator of Open-Sourced Methodology for AI-Enhanced Document Review - TARcourse.com & general e-discovery practice at EDBP.com. Author/Publisher of e-DiscoveryTeam.com and AI-Ethics.com.

Doc Review in the Dark Ages: 1980-2000 Bloody Paper Nightmare Paper cuts, staples, burns, holes, clips, yellow stickies All Docs Were Filed by Expert Custodians High-Tech = Copiers, Fax Key Players and Roles Most lawyers were Men with the “Boys” in charge of Doc Review Women still did most of the work, often the real SMEs. Secretaries (remember them?) Paralegals Acted as Project Managers

Doc Review in the Dark Ages: 1980-2000 ALL PAPER in almost all projects until late 90s. Keyword Search by late 80s. 1986: Blair & Maron – 75% thought found; actually 20% Big Project was 100,000 docs, 1,000,000 pages. All Linear Review 2,000 Relevant Docs Found – 2% Estimated Recall = 25%-50% Privilege Log – 50 docs 10 Highly Relevant Docs – 0.001% Estimated Recall = 80% No Preservation Holds No Spoliation Cases Were parties & lawyers more honest?

Doc Review in the Dark Ages: 1980-2000 Players & Play Opposing Counsel & Rules Start the Process Client Owns & Knows the Data Senior Associate or Jr. Partner (often did collection) In charge of doc review. Studies the Request of Opposing Counsel Makes Objections & Motions May never even talk to Opposing Counsel until Hearing on Motions Judge Rules on Motions Review Staff who Skim the documents New Associates Paralegals Litigation Secretaries Bate Stampers! Paper Documents Usually in labeled Folders and Boxes Reviewed and stacked them them into Three Piles: Irrelevant, Relevant, Hot

Doc Review Today: 2018 All Ephemeral: 0s & 1s Everyone is a Custodian Paper Cut Nightmares replaced by the Missed Privileged and Spoliation Fears Email Families with Orphans & Irrelevant Children Everyone is a Custodian High Tech = Online Reviews and AI Machine Learning ESI Review has become a Legal/Technical Specialty Statistics & Information Science Prevalence, Recall, Precision, F1 Project Management, QC & QA Key Players and Roles New Importance of SMEs and Technology Are Contract Review Lawyers the New Secretaries?

Doc Review Today: 2018 Big Project is 100 Million ESI files Nobody Counts Pages Linear Review is Almost Dead Hybrid Multimodal IST Predictive Coding 4.0 (TARcourse.com) 20,000 Relevant ESI Found – .02% (was 2,000 docs & 2%) Estimated Recall = 80% (was 50%) Privilege Log – 500 files (was 50) 12 Highly Relevant Docs – 0.000012% (Was 10 and 0.001% Estimated Recall = 100% (was 80%)

Doc Review Today: 2018

Doc Review in Future and Far Future – 2028, 2038

Prepare Now for Tomorrow “Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control.”

AI-Ethics.com

One Last Book Recommendation “AI will take over more and more of the decision-making territory once occupied by humans.” “Machines are becoming so capable that, today, it is hard to see the higher cognitive ground that many people could move to.” “Work in the age of smart machines means constant change and adaptation.” “The key isn’t competition but instead “augmentation,” in which “humans and computers combine their strengths to achieve more favorable outcomes than either could alone.”

e- over Sixty 800+ ralph.losey@jacksonlewis.com