MIAMI USER GROUP Selling Relativity Analytics to Internal Stakeholders
© 2015 kCura. All rights reserved.
The Miami Steering Committee • Christy Valdes, Hogan Lovells
• Jason Temple, Greenberg Traurig • Jose Gonzalez, Hogan Lovells
© 2015 kCura. All rights reserved.
Who are you?
© kCura LLC. All rights reserved.
Agenda •
Relativity Analytics Adoption
•
Overview of Relativity Analytics
•
Articulating the Value of Relativity Analytics
•
Common Objections
•
Takeaways
•
Selling Analytics: Scenarios
© kCura LLC. All rights reserved.
Relativity Analytics Adoption
© kCura LLC. All rights reserved.
246 Customers with Analytics Subscriptions © kCura LLC. All rights reserved.
Current RA Subscription Mix by Client Type 160
140
30 55
120
100
80
60
116
40
7
74
4 20
35
25
0
Corporation
Government Subscription
© kCura LLC. All rights reserved.
Law Firm No Subscription
Litigation Service Provider
over
1.6 Billion documents analyzed
© kCura LLC. All rights reserved.
Mar-09
© kCura LLC. All rights reserved.
Mar-16
Jan-16
Nov-15
1,100,000
Sep-15
Jul-15
May-15
Mar-15
Jan-15
Nov-14
Sep-14
Jul-14
May-14
Mar-14
Jan-14
Nov-13
Sep-13
Jul-13
May-13
Mar-13
Jan-13
Nov-12
Sep-12
Jul-12
May-12
Mar-12
Jan-12
Nov-11
Sep-11
Jul-11
May-11
Mar-11
Jan-11
Nov-10
Sep-10
Jul-10
May-10
Mar-10
Jan-10
Nov-09
Sep-09
Jul-09
May-09
Cumulative GBs Indexed through Analytics 1,200,000
1,125,561
1,000,000
900,000
800,000
700,000
600,000
500,000
400,000
300,000
200,000
100,000
0
Analytics – Average Monthly GB’s Indexed By Year 60,000
56,609
50,000
40,000
35,924
30,000
18,532
20,000
12,342
7,393
10,000
3,664 1,033 0 2010
2011
2012 2010
© kCura LLC. All rights reserved.
2013 2011
2012
2013
2014 2014
2015
2016
2015
2016
Trend: Using Technology to the Fullest •
Sedona Principles: – Number 11 – use of “electronic tools and processes, such as data sampling, searching, or the use of selection criteria, to identify data reasonably likely to contain relevant information.” – Commentary to Principle 11 - possible to use technology to search for ‘concepts,’ which can be based on ontologies, taxonomies, or data clustering approaches, for example.”
© 2015 kCura. All rights reserved.
Trend: Openness and Transparency •
Sedona Conference Cooperation Proclamation: – Leading jurists, trial attorneys, corporate counsel, government lawyers, and others are signing onto “The Cooperation Proclamation”. By doing so, they are pledging to… • Reverse the legal culture of adversarial discovery that is driving up costs and delaying justice • Help create “toolkits” of model case management techniques and resources for the Bench, inside counsel, and outside counsel to facilitate proportionality and cooperation in discovery • Help create a network of trained electronic discovery mediators available to parties in state and federal courts nationwide, regardless of technical sophistication, financial resources, or the size of the matter.
© 2015 kCura. All rights reserved.
Overview of Relativity Analytics
© kCura LLC. All rights reserved.
Relativity Analytics Features Structured
Conceptual
• Email threading
• Concept searching
• Near duplication
• Categorization
• Language identification
• Clustering
• Keyword expansion
© kCura LLC. All rights reserved.
Email Threading What is it? • Identifies and arranges emails that were part of a single thread or conversation. What is it used for? • Allows you to: – Easily see the order of each email in a thread. – See which emails are inclusive (i.e. have unique content). – Identify email duplicate spares (i.e. emails with the same content). How will it help me? • Sort and organize emails by thread for more intuitive review. • Saves time if only reviewing the non-duplicative inclusive emails. ◊
© kCura LLC. All rights reserved.
2/24/99 11:25 a.m.
4/29/99 6:45 p.m.
4/30/99 9:03 p.m.
?
Barry Pearce
Bob Crane & Jeff Harbert
Maria Nartey
Richard Sage & Mark Elliott
© kCura LLC. All rights reserved.
4/30/99 7:00 p.m.
4/30/99 7:22 p.m.
4/30/99 10:24 p.m.
5/1/99 12:57 a.m.
Textual Near Duplicate Identification What is it? • Identifies documents with highly similar text and places them into relational groups. What is it used for? • Allows you to: – Use near dupe groups in searching or filtering. – Conflict check coding decisions amongst near dupes prior to production.
How will it help me? • Saves time by identifying very similar documents prior to the start of review. You can also use the near dupe groups for review and QC. ◊
© kCura LLC. All rights reserved.
Can you spot the difference? Version A
© kCura LLC. All rights reserved.
Version B
Language Identification What is it? • Determines a document’s primary language and up to 2 secondary languages. What is it used for? • Allows you to see how many languages are present in your collection, and the percentages of each language by document. How will it help me? • Easily filters documents by language and batch out files to native speakers for review. • Determines if translation is needed. ◊
© kCura LLC. All rights reserved.
Concept Searching What is it? • Searches using a sentence, paragraph, or entire document • Returns documents related to the concept of the query What is it used for? • Searching for documents based on ideas instead of absolutes • More natural querying
How will it help me? • Find documents even if terms differ. ◊
© kCura LLC. All rights reserved.
?
?
© kCura LLC. All rights reserved.
Categorization What is it? • A classification method that uses example documents which are defined by a reviewer What is it used for? • Prioritizing review by using previously coded data as examples • Identifying documents which are related to the hot documents identified by expert reviewers How will it help me? • Prioritize review by a time or reviewer standpoint • Sort through a large volume of data quickly • Run a quick production after validating results ◊
© kCura LLC. All rights reserved.
Price Fixing: 8 documents ◊
© kCura LLC. All rights reserved.
Clustering What is it? • Use the power of the conceptual index to identify groups of conceptually related documents. What is it used for? • This can be used as a tool for investigation, analysis, review, or QC. How will it help me? • Investigate a large unknown dataset • Cull out non-relevant documents quickly • Speed up a linear review by batching conceptually related documents together ◊
© kCura LLC. All rights reserved.
Custodian Name
Custodian Name
Custodian Name
Custodian Name © kCura LLC. All rights reserved.
Keyword Expansion What is it? • Uses the concept space to allow users to submit terms and returns conceptually related words What is it used for? • Investigating the language of the workspace using known keywords How will it help me? • Allows you to find code words • Assists in expanding the keyword list • Familiarize yourself with the language of the case. ◊
© kCura LLC. All rights reserved.
© kCura LLC. All rights reserved.
Articulating the Value of Relativity Analytics
© kCura LLC. All rights reserved.
Articulating Value – Use Cases • Audiences identify with use cases over features • More memorable for your internal client • More natural method of “selling”
© kCura LLC. All rights reserved.
Use Case Features Use Case
Feature
•
• •
Narrowing the review set
© kCura LLC. All rights reserved.
Email threading Foreign language identification
Use Case Features Use Case
Feature
• •
• •
Narrowing the review set Quality control
© kCura LLC. All rights reserved.
Near duplicate identification Cluster visualization
Use Case Features Use Case
Feature
• • •
•
Narrowing the review set Quality Control Investigation
© kCura LLC. All rights reserved.
Keyword expansion
Use Case Features Use Case
Feature
• • • •
• •
Narrowing the review set Investigation Quality control Organizing large sets of data
© kCura LLC. All rights reserved.
Clustering Categorization
Common Objections
© kCura LLC. All rights reserved.
“Analytics is best only for the largest cases.” • Messaging with analytics in e-discovery has focused on large case wins. • There are a number of uses for a majority of cases. For example: – Batching - Reviewing conceptually related documents increases review speed – Production prep – Analytics can help to avoid mistakenly producing privileged docs – Keyword sampling – Address keyword issues to help identify other potentially relevant documents – Threading – Only review inclusives and reduce the volume of email
© kCura LLC. All rights reserved.
“Analytics is too expensive” • Email threading will save clients money on almost every case • Other workflows that can help save your client money: – Batching by Clusters – Categorization – Relativity Assisted Review
© kCura LLC. All rights reserved.
Confidential - kCura 2015
“Analytics is not defensible” •
Da Silva Moore, et al. v. Publicis Groupe, No. 11 Civ. 1279 (ALC)(AJP), 2012 WL 607412 (S.D.N.Y. Feb. 24, 2012). – First court to explicitly approve the use of TAR in e-discovery – “What the Bar should take away from this Opinion is that computer-assisted review is an available tool and should be seriously considered for use in large-data-volume cases where it may save the producing party (or both parties) significant amounts of legal fees in document review. Counsel no longer have to worry about being the ‘first’ or ‘guinea pig’ for judicial acceptance of computer-assisted review.”
© kCura LLC. All rights reserved.
Confidential - kCura 2015
“Analytics is not defensible” •
Dynamo Holdings Ltd. P’ship v. Comm’r of Internal Revenue, Nos. 2685-11, 8393-12 (T.C. Sept. 17, 2014). – Tax court petitioner sought permission to use predictive coding – First time the tax court sanctioned the use of TAR – Court recognized that it is widely accepted and does not cause an undue burden
© kCura LLC. All rights reserved.
Confidential - kCura 2015
“Analytics is not defensible” •
Bridgestone Americas, Inc. v. Int. Bus. Machs. Corp., No. 3:13-1196 (M.D. Tenn. July 22, 2014). – Defendant objected to use of predictive coding because it would change the original case management order – Screening of search terms already completed – Rule 26 requires discovery be tailored “by the court to be as efficient and cost effective as possible.” – Magistrate allowed plaintiff to switch horses in midstream – Openness and transparency are critical and expected
© kCura LLC. All rights reserved.
Confidential - kCura 2015
“Analytics is not defensible” •
Progressive Cas. Ins. Co. v. Delaney, No. 11–CV–00678, 2014 WL 3563467 (D. Nev. July 18, 2014). – Production to the FDIC by Progressive. Keyword culling was done to narrow population from 2 million to 565,000 documents that hit on key words. – Progressive used TAR without defendant’s agreement and without leave of court. Court rejected the use of TAR because Progressive failed to inform. – No discovery about discovery – exceeds the scope of Rule 26 – a reasonable search is required – Progressive ordered to produce all 565,000, but permitted to apply a privilege filter
© kCura LLC. All rights reserved.
Confidential - kCura 2015
Takeaways
© kCura LLC. All rights reserved.
Takeaways •
Document sets continue to grow along with the pressure to drive down cost.
•
Relativity Analytics can save your case team money.
•
Analytics should be used on every case (QC, Review speed, organizing large sets of documents, narrowing down the set).
•
Be an advisor. Be prepared to explain your workflow and what happens next.
•
Know the use cases and your own success stories. *
© kCura LLC. All rights reserved.
Selling Analytics: Scenarios
© kCura LLC. All rights reserved.
Scenario #1 You have a discovery deadline quickly approaching. You were on#1 target for your Challenge deadline until you were just dropped with 100 GB of data to review. How will you get through this data in time for your deadline?
© kCura LLC. All rights reserved.
Scenario #1 – Solution You have a discovery deadline quickly approaching. You were on target for your deadline until you were just dropped with 100 GB of data to review. How will you get through this data in time for your deadline? 1. 2. 3. 4.
Use your existing coded documents as examples in a categorization set. Create a search to find documents categorized as Responsive with a rank higher than 80. Batch these out to second level review. Create a search to find documents categorized as Not Responsive and rank greater than 80, or where they are not categorized. 5. Batch these out to first level review.
The key is prioritization. © kCura LLC. All rights reserved.
Scenario #2 Your attorney received 5 paragraphs from a subject matter expert depicting potential Challenge #1people that conversations among three corporate counsel believes to be important. How will you find these types of conversations between these three custodians? © kCura LLC. All rights reserved.
Scenario #2 – Solution Your attorney received 5 paragraphs from a subject matter expert depicting potential conversations among three people that corporate counsel believes to be important. How will you find these types of conversations between these three custodians? 1. 2. 3. 4.
Create a search to find these three custodians’ documents. Create a categorization set. Add each paragraph as an example, using Text Excerpts. Run categorization against the documents in Search #1.
© kCura LLC. All rights reserved.
Scenario #3 You need to QC your production to make sure Challenge #1 no privileged documents go out the door. How will you speed up this process to be as efficient as possible?
© kCura LLC. All rights reserved.
Scenario #3 – Solution You need to QC your production to make sure no privileged documents go out the door. How will you speed up this process to be as efficient as possible? 1. Run Textual Near Duplicate Identification against all documents. 2. Run a search using metadata fields to find privileged documents. 3. Include Textual near duplicates on the search and filter down to find inconsistencies.
© kCura LLC. All rights reserved.
Scenario #4 You’ve already coded your own documents, and you just received a production from Challenge #1 the opposing counsel. You’ve been data dumped! How will you find the relevant documents that you need? © kCura LLC. All rights reserved.
Scenario #4 – Solution (Option A) You’ve already coded your own documents, and you just received a production from the opposing counsel. You’ve been data dumped! How will you find the relevant documents that you need? Option A: 1. Cluster the received production data. 2. Evaluate the clusters to see if there is any junk that can be quickly eliminated from review.
© kCura LLC. All rights reserved.
Scenario #4 – Solution (Option B) You’ve already coded your own documents, and you just received a production from the opposing counsel. You’ve been data dumped! How will you find the relevant documents that you need? Option B: 1. Use your existing documents as examples in a new categorization set. a. Designation, Issues, Privilege, Hot Docs, etc. b. Use the production as the “Documents to be Categorized” search 2. Cluster the received production documents. 3. Use Pivot and compare each cluster with the categorized issues. © kCura LLC. All rights reserved.
One More Thing…
© kCura LLC. All rights reserved.
Mobile Beta Program • New iPad app separate from Relativity Binders • Mobile-friendly design for saved searches, views, document lists, and coding layouts as they exist in Relativity • All coding on mobile is synchronized to and viewable in Relativity
© kCura LLC. All rights reserved.
Join the Beta!
© kCura LLC. All rights reserved.
•
Experience the new app before it becomes publicly available.
•
Explore new features added every month.
•
Guide the development of the product – a direct-line to Product Management.
•
Email
[email protected] to participate.
In the Future • Administer Relativity from your smartphone • Case strategy and construction • Reporting dashboards • Support for additional devices and operating systems
© kCura LLC. All rights reserved.
One More “One More Thing”…
© kCura LLC. All rights reserved.
© kCura LLC. All rights reserved.