Relativity Bag of Tricks
St. Louis User Group Relativity Bag of Tricks: Solving Common Problems in Unique Ways
© kCura LLC. All rights reserved.
Agenda • Introductions • Tricks from the Relativity bag – User experiences and tips – Let’s have a conversation
– Ask questions – Contribute your stories
© kCura LLC. All rights reserved.
Bag of Tricks • QC Protocols for contract reviews (and in house too)
• Effective communication between the case team and the review team • Hints to manage your project in Relativity • Best practices to keep your stakeholders informed with reporting tools • Quick wins when using Relativity Assisted Review © kCura LLC. All rights reserved.
QC workflows •
Create a view for Privilege QC
•
Encourage reviewers to do their own QC
•
Use a second level review layout
•
Take advantage of built in Sampling
© kCura LLC. All rights reserved.
View: Privilege QC
© kCura LLC. All rights reserved.
View: Self QC
© kCura LLC. All rights reserved.
QC Layout
© kCura LLC. All rights reserved.
QC: Create a Good Layout
© kCura LLC. All rights reserved.
QC: Create a Good Report to Track Results
© kCura LLC. All rights reserved.
Sampling for Success
© kCura LLC. All rights reserved.
© kCura LLC. All rights reserved.
QC: Use Sampling to Your Benefit 1
© kCura LLC. All rights reserved.
2
QC: Save the Results
© kCura LLC. All rights reserved.
Effective communication between the case team and the review team •
Processing detail (summary)– De-duplication
– Exceptions – Other issues- Missing files, OCR failures •
QC searches– Responsive Set – Family Conflicts
– Needs Redaction – Tech Assist © kCura LLC. All rights reserved.
LayoutsDocument Metadata Has hidden data and exceptions ReportsBatch progress- update RAR-update STRs (key terms and privilege)- iterative Security viewsLayout mockup Preview for case team
Hints to manage your Relativity project •
Establish the big picture
•
Prepare for new workspace and data
•
How do you handle new data
•
Tips to conduct the review
•
Readying a solid production
© kCura LLC. All rights reserved.
Keeping Stakeholders Informed •
Where in the process are you?
•
Who are the stakeholders?
•
What do they want to know?
© kCura LLC. All rights reserved.
Processing •
Client: wants to know what they gave you and how much it is going to cost – report showing data by custodian
– total size/cost •
Legal Team: needs data to support their discovery efforts – Volume at start
– Volume deduped – Volume left out from culling parameters
© kCura LLC. All rights reserved.
Pre-Review •
Partner: wants to know how much there is to review and how long it will take. – Report on average review times for that much data
– Report data to show whether, in this particular case, analytics will help save review time/money •
Associate: wants to know best place to start review – Reports on search terms, concepts, domains
© kCura LLC. All rights reserved.
Review •
Dashboards to show progress of documents set to review will keep everyone up to date to the minute. Can help forecast how much longer it will take.
•
BUT…make sure to have a dashboard that shows status of QC as well, so there are no surprises.
© kCura LLC. All rights reserved.
Production •
QC dashboards from review will help alert Lit Support that a production is imminent, even if the legal team does not communicate well!
© kCura LLC. All rights reserved.
Post-Production •
Fact Manager can help the team keep up to date on what useful (or not useful) information has been located in the documents.
© kCura LLC. All rights reserved.
Quick wins using Relativity Assisted Review •
Time to PLAN
•
Train the Reviewers
•
Rounds of judgmental sampling
•
End of the Game – dealing with the NRs
© kCura LLC. All rights reserved.
PLANNING - Assisted Review Checklist* - Does this dataset meet the criteria? RAR Case Checklist
Meets Criteria
Minimum of 50,000 records with text
Concept-rich files, e.g. emails, word documents, limited excel files, or graphic images
Issue or privilege coding in separate field not part of Assisted Review workflow Is this previously reviewed data? If so, how were families reviewed? Expected timelines given Expected process given Number of reviewers Level of reviewers Level of accuracy
See https://help.kcura.com/9.4/Content/Recipes/Checklists/RAR_checklist.htm © kCura LLC. All rights reserved.
Train the Reviewers •
Selecting good example documents – Quality of the example is independent of responsiveness – it’s a second dimension
– Some documents may be highly responsive, yet undesirable as example documents for a computer-assisted review project – Four Corners Doctrine
•
Applying the Use as Example field
•
Using the Excerpt Text feature
© kCura LLC. All rights reserved.
PROJECT – Judgmental Sampling Options •
Hot Documents
• Important Custodian •
Keyword Search Hits
•
Previously Coded Documents
•
Clustering
© kCura LLC. All rights reserved.
END GAME: Uncategorized/Non Responsive Documents Options •
Save until last (still reviewing all)
•
One last sample
•
Keyword searches for hot documents
© kCura LLC. All rights reserved.
Thanks for attending!
© kCura LLC. All rights reserved.
© kCura LLC. All rights reserved.
Agenda • Introductions • Tricks from the Relativity bag – User experiences and tips – Let’s have a conversation
– Ask questions – Contribute your stories
© kCura LLC. All rights reserved.
Bag of Tricks • QC Protocols for contract reviews (and in house too)
• Effective communication between the case team and the review team • Hints to manage your project in Relativity • Best practices to keep your stakeholders informed with reporting tools • Quick wins when using Relativity Assisted Review © kCura LLC. All rights reserved.
QC workflows •
Create a view for Privilege QC
•
Encourage reviewers to do their own QC
•
Use a second level review layout
•
Take advantage of built in Sampling
© kCura LLC. All rights reserved.
View: Privilege QC
© kCura LLC. All rights reserved.
View: Self QC
© kCura LLC. All rights reserved.
QC Layout
© kCura LLC. All rights reserved.
QC: Create a Good Layout
© kCura LLC. All rights reserved.
QC: Create a Good Report to Track Results
© kCura LLC. All rights reserved.
Sampling for Success
© kCura LLC. All rights reserved.
© kCura LLC. All rights reserved.
QC: Use Sampling to Your Benefit 1
© kCura LLC. All rights reserved.
2
QC: Save the Results
© kCura LLC. All rights reserved.
Effective communication between the case team and the review team •
Processing detail (summary)– De-duplication
– Exceptions – Other issues- Missing files, OCR failures •
QC searches– Responsive Set – Family Conflicts
– Needs Redaction – Tech Assist © kCura LLC. All rights reserved.
LayoutsDocument Metadata Has hidden data and exceptions ReportsBatch progress- update RAR-update STRs (key terms and privilege)- iterative Security viewsLayout mockup Preview for case team
Hints to manage your Relativity project •
Establish the big picture
•
Prepare for new workspace and data
•
How do you handle new data
•
Tips to conduct the review
•
Readying a solid production
© kCura LLC. All rights reserved.
Keeping Stakeholders Informed •
Where in the process are you?
•
Who are the stakeholders?
•
What do they want to know?
© kCura LLC. All rights reserved.
Processing •
Client: wants to know what they gave you and how much it is going to cost – report showing data by custodian
– total size/cost •
Legal Team: needs data to support their discovery efforts – Volume at start
– Volume deduped – Volume left out from culling parameters
© kCura LLC. All rights reserved.
Pre-Review •
Partner: wants to know how much there is to review and how long it will take. – Report on average review times for that much data
– Report data to show whether, in this particular case, analytics will help save review time/money •
Associate: wants to know best place to start review – Reports on search terms, concepts, domains
© kCura LLC. All rights reserved.
Review •
Dashboards to show progress of documents set to review will keep everyone up to date to the minute. Can help forecast how much longer it will take.
•
BUT…make sure to have a dashboard that shows status of QC as well, so there are no surprises.
© kCura LLC. All rights reserved.
Production •
QC dashboards from review will help alert Lit Support that a production is imminent, even if the legal team does not communicate well!
© kCura LLC. All rights reserved.
Post-Production •
Fact Manager can help the team keep up to date on what useful (or not useful) information has been located in the documents.
© kCura LLC. All rights reserved.
Quick wins using Relativity Assisted Review •
Time to PLAN
•
Train the Reviewers
•
Rounds of judgmental sampling
•
End of the Game – dealing with the NRs
© kCura LLC. All rights reserved.
PLANNING - Assisted Review Checklist* - Does this dataset meet the criteria? RAR Case Checklist
Meets Criteria
Minimum of 50,000 records with text
Concept-rich files, e.g. emails, word documents, limited excel files, or graphic images
Issue or privilege coding in separate field not part of Assisted Review workflow Is this previously reviewed data? If so, how were families reviewed? Expected timelines given Expected process given Number of reviewers Level of reviewers Level of accuracy
See https://help.kcura.com/9.4/Content/Recipes/Checklists/RAR_checklist.htm © kCura LLC. All rights reserved.
Train the Reviewers •
Selecting good example documents – Quality of the example is independent of responsiveness – it’s a second dimension
– Some documents may be highly responsive, yet undesirable as example documents for a computer-assisted review project – Four Corners Doctrine
•
Applying the Use as Example field
•
Using the Excerpt Text feature
© kCura LLC. All rights reserved.
PROJECT – Judgmental Sampling Options •
Hot Documents
• Important Custodian •
Keyword Search Hits
•
Previously Coded Documents
•
Clustering
© kCura LLC. All rights reserved.
END GAME: Uncategorized/Non Responsive Documents Options •
Save until last (still reviewing all)
•
One last sample
•
Keyword searches for hot documents
© kCura LLC. All rights reserved.
Thanks for attending!
© kCura LLC. All rights reserved.
