TAR, CAR, Predictive Coding, Integrated Analytics What Does It All Mean? Huron Legal provides advisory and business services to assist law departments and law firms enhance organizational effectiveness and reduce legal spend. Huron Legal advises on and implements strategy, organizational design and development, outside counsel management, operational efficiency, discovery solutions, and provides services relating to the management of matters, contracts, documents, records, digital evidence and e-discovery.

What is Predictive Coding? 2 “The use of software to automate the document review process in ediscovery based on artificial intelligence techniques.” Evan Koblentz, Law Technology News, March 2012

Better Name: Technology Assisted Review © 2011 Huron Consulting Group. All rights reserved. Proprietary & Confidential. 3 Using technology to assist with locating information responsive to discovery requests Requires a defined process and workflow that is documented and defensible “Seed sets,” data analytics, statistical sampling Lawyers will be expected to understand, articulate, implement and defend technology to identify relevant data

Brand New Technology? Not Exactly 4 What about Keyword search Clustering Near duplicate identification Concept search Other industries

Target’s Pregnancy Predictor--Real World Data Analytics © 2013 Huron Consulting Group. All rights reserved. Proprietary & Confidential. 5

What Do I Need to Know About Technology Assisted Review? 6 “Everyone” in the ediscovery world is talking about it How it works Cost implications Not all methods are created equal Developing case law

What Is Analytics? © 2013 Huron Consulting Group. All rights reserved. Proprietary & Confidential. 7

Cases 8 DaSilva Moore v. Publicis Groupe SA, 287 F.R.D. 182 (S.D.N.Y. 2012), adopted sub nom. Moore v. Publicis Groupe SA, 11 Civ ALCAJP, 2012 WL (S.D.N.Y. April 26, 2012) Global Aerospace, Inc. et al. v. Landow Aviation, L.P., Consol. Case No ( Va. Cir. Ct. Apr. 23, 2012) In re Actos (Pioglitazone) Products Liability Litigation, MDL No. 6:11- MD-2299 (W.D. La. July 27, 2012) Kleen Products, LLC, et al v. Packaging Corp. of America et al., Case No. 1:10-CV (N.D. Ill., Sept. 28, 2012)

Da Silva Moore 9 Magistrate Judge Andrew Peck ordered a protocol for predictive coding based agreement between the parties to use technology assisted review Protocol builds in standards for reliability and plaintiff participation Q/A of search methods, opportunity to contest poor results Plaintiffs objected to the number of documents for the seed set, as well as protocols to be used and appealed the ruling Also moved to recuse Peck based on his participation at Ediscovery conferences and writing on the subject of predictive coding District Judge Carter affirmed Peck’s ruling “There is simply no review tool that guarantees perfection.”

Global Aerospace © 2013 Huron Consulting Group. All rights reserved. Proprietary & Confidential. 10  “Defendants shall be allowed to proceed with the use of predictive coding for the purposes of processing and production of electronically stored information.”  “This is without prejudice to the receiving party raising with the court an issue as to completeness or the contents of the production or the ongoing use of predictive coding.”

Global Aerospace cont’d  5000 doc. sample of 1.3 million doc’s coded as responsive or nonresponsive  Results returned approx. 173,000 doc’s as likely responsive  400 doc’s deemed relevant by computer were reviewed; approx. 80% determined to be relevant  Sample of documents deemed irrelevant were reviewed; 2.9% of the sample were deemed relevant  Recall rate of 81%  Approximately $200,000 cost © 2013 Huron Consulting Group. All rights reserved. Proprietary & Confidential. 11

In Re Actos © 2013 Huron Consulting Group. All rights reserved. Proprietary & Confidential. 12 Experts from both sides work together to train the system One relevance call per document Initial review for privilege Relevancy score cutoff to be agreed to by the parties QC review for random set of documents below relevancy score cutoff Review of random set of documents above the agreed upon relevance score but were identified by human review as nonresponsive

Kleen Products 13 Plaintiffs in antitrust action sought an order from Magistrate Judge Nan Nolan to require defendants to use content-based advanced analytics (“CBAA”) to identify relevant documents for production Defendants had completed the majority of their production before the motion was filed Two days of hearings, expert testimony, lots of briefing

Kleen Products © 2013 Huron Consulting Group. All rights reserved. Proprietary & Confidential. 14 Sedona Principle 6: “[U]nder Sedona Principle 6, “[r]esponding parties are best situated to evaluate the procedures, methodologies, and techniques appropriate for preserving and producing their own electronically stored information.” “The parties had a fundamental dispute over what search methodology Defendants should utilize to identify documents responsive to Plaintiffs’ RPD. Defendants argued that in order to best identify potentially responsive ESI, they engaged leading consulting companies to develop Boolean search terms. During an iterative process, Defendants and their consultants revised and refined the search terms over the course of several months.

Kleen Products Continued Sampling procedures were used throughout the process to evaluate the effectiveness and reliability of the search terms.” “As to any documents or ESI beyond the First Request Corpus, Plaintiffs agreed not to argue or contend that Defendants should be required to apply CBAA or “predictive coding” methodology with respect to any requests for productions served on any Defendant prior to October 1, With respect to any requests for production served on any Defendant on or after October 1, 2013 that requires the collection of documents beyond the First Request Corpus, the parties agreed to meet and confer regarding the appropriate search methodology to be used for such newly collected documents.” © 2013 Huron Consulting Group. All rights reserved. Proprietary & Confidential. 15

The Current Approach to Document Analysis Is Not Sustainable – Data volumes continue to rise rapidly – Review workflows increasingly are not scalable to meet demand – Even with lower pricing, costs are not proportional to litigation risks – Most “predictive coding” solutions are fragmented, requiring merits counsel to fill in the gaps Why TAR? 16 © 2012 Huron Consulting Group. All rights reserved. Proprietary & Confidential.

Document Review Data Analytics Production HostingProcessingCollectionPreservation Domestic and Offshore Review Centers: seat capacity in US, Europe and India Domestic and Offshore Review Centers: seat capacity in US, Europe and India Hard Copy & ESI TIFF responsive Docs Load file creation Hard Copy & ESI TIFF responsive Docs Load file creation Data Analytics Dedicated Analytics Center Multi- disciplinary team Search term analysis Defensible culling Data Analytics Dedicated Analytics Center Multi- disciplinary team Search term analysis Defensible culling 17 70% of all eDiscovery Costs Document Review Volumes doubling every months < 20% responsive rates, even when using search terms REVIEW IS LESS ABOUT HIGH-LEVEL LEGAL ANALYSIS AND MORE ABOUT DISCARDING IRRELEVANT DOCUMENTS Ediscovery Process PROJECT MANAGEMENT © 2012 Huron Consulting Group. All rights reserved. Proprietary & Confidential. eDISCOVERY CONSULTING

© 2011 Huron Consulting Group. All rights reserved. Proprietary & Confidential. 18 The sampling population contains only one copy of each document and does not include documents without concepts (e.g., pictures, Excel files, etc.) Each sample document selected is called an anchor Each document in the population gets aligned with an anchor document based on conceptual similarity Integrated Analytics: “Anchor” Concept

© 2011 Huron Consulting Group. All rights reserved. Proprietary & Confidential. 19 Nearest Neighbor 1:.75 SIM score Nearest Neighbor 2:.35 SIM score IA : Similarity Scores

© 2011 Huron Consulting Group. All rights reserved. Proprietary & Confidential. 20 High SIM score IA: System Learning Low SIM score Goals: 1)Have little to no document “movement” when new anchors introduced 2)Have relatively high SIM scores throughout population

Integrated Analytics Workflow © 2012 Huron Consulting Group. All rights reserved. Proprietary & Confidential. 21

Integrated Analytics  A Comprehensive, Integrated Process Multi-Disciplinary Expert Team: Data Miners, Statisticians, Lawyers Multi-tool approach (latent semantic indexing and machine learning) Integrated process in a dedicated facility  Independent Statistical Verification of Process Developed in collaboration with industry and academic experts 99% confidence level (+/- 2.5% confidence interval) “Accept on Zero” statistical validation  Expert/Defensibility Resources Independent, dedicated resources (legal + academic experts)  Pricing Pricing based on relevance percentage (< relevant = < price) Capped Pricing: Places financial risk on Huron to properly cull out documents Includes defensibility resources as part of pricing Benchmarked cost savings of 30% 22