TAF Education Fund Annual Conference & Awards 2018 Predictive Coding –A Love Story October 4, 2018

I. Hypothetical A. Case Background: Relator, a former employee of Defendant, brought a case in federal court against a large defense contractor (Relator’s former employer) alleging violations of the False Claims Act. The government declined to intervene and the Relator is vigorously litigating her claims. The district court denied Defendant’s motion to dismiss and the case is in the midst of document discovery.

I. Hypothetical Relator’s Discovery Requests: Relator served Initial Requests for Production, seeking custodial and non-custodial documents from Defendant. In particular, Relator seeks: Defendant’s communications with the government: Invoices Receipts Custodial documents from Defendant’s current and former employees who created, sent or received documents (e.g., emails) concerning the allegations in the Complaint.

I. Hypothetical Defendant's Response: The document and data collection from just 20 initial custodians amounts to approximately 3 million documents (e.g., emails, loose electronic documents). A linear, manual review of 3 million documents is unduly burdensome and disproportionate to the needs of the case. In order to reduce the burden associated with such a review, Defendant proposes to use keyword search terms to cull the universe of collected documents and data.

I. Hypothetical Defendant’s TAR Process. After several months of applying keyword search term iterations Defendant argues to Relator that the keyword search terms are not culling the collected documents and data sufficiently and that reviewing the custodial emails culled-in by the keyword search terms would still be too burdensome.

I. Hypothetical Defendant’s TAR Process Specifically, after applying the latest iteration of proposed keywords, the 20 custodians’ 3 million documents are reduced to 600,000 documents. As a result, Defendant is now going to abandon a linear manual review process and instead apply a Technology-Assisted Review (“TAR”)(a/k/a "Predictive Coding") process to those 600,000 document instead.

II. Panel Discussion

III. Takeaways A. From the outset, negotiate with opposing counsel on an ESI Stipulation that addresses keyword searching and TAR (even if the defendant is unclear which it will use.)

III. Takeaways If keyword search terms, negotiate provisions … that ensure transparency in process of developing keyword search terms; that require “hit rate reports”; that require validation and testing of documents culled-out by keyword search terms; and for judicial intervention.

III. Takeaways If TAR, negotiate provisions… that ensure transparency in the overall TAR process and workflow; that allow for Relator/Plaintiff to provide sample documents for the seed set; that require validation metrics (e.g., recall and precision); that ensure transparency for how documents that fall outside of the TAR process and workflow will be handled; for judicial intervention. See Order Regarding Search Methodology for ESI, In Re Broiler Chicken Antitrust Litigation, 1/3/18. NDIL. Maura Grossman

III. Takeaways B. Hire an expert consultant to assist with analysis and negotiation of keyword search process or TAR process. C. Consider applying your own TAR process when you receive the documents produced, particularly if you suspect low precision. D. Consider alternative workflows to identify particular documents and data (e.g., non-custodial documents such as contracts, invoices, etc.)