eDiscovery Tiramisu, a delicious metaphor describing the layers of search applied throughout the eDiscovery process. If you’ve been involved in discovery in matters with electronic documents, you’ll have tasted the usual flavours: requesting documents directly from key people, running Boolean searches and undertaking subjective review to separate the yolk from the white. My recipe adds an additional layer (perhaps the tastiest) to the eDiscovery treat, Technology Assisted Review (TAR) and its evolution to Continuous Active Learning (CAL). A mouth-watering moreish mouthful!

What is TAR and why is it in my dessert? TAR is the process of training a computer algorithm to classify documents by learning from samples of documents reviewed by an expert. The premise being relevant documents have similar characteristics and irrelevant documents have different characteristics. Once the characteristics are understood the algorithm can assess other documents and can measure how close (or far away) they are from documents in the sample.

The premise being relevant documents have similar characteristics and irrelevant documents have different characteristics

I’ve been involved in a number of matters where we’ve added TAR to the eDiscovery mixing bowl in two ways. The first and more readily accepted approach is to prioritise the documents from those most likely to be relevant to those least likely to be relevant. It’s straightforward enough; the algorithm pushes documents with relevant characteristics to the front of the queue and those without those characteristics to the back of the queue. The review team gets the relevant documents first and then works their way through the ranked list.

The second method of using TAR is really an extension of the first, to leverage TAR to not review documents. As an example, on one project with over four million documents (responsive to keywords agreed with the other side…. That is a post for another day), TAR empowered our client to make an informed decision on when document review should cease. As review descended down the TAR rankings, less relevant documents were found each day with a corresponding increase in the number of irrelevant documents found.

When we confidently measured the recall of relevant documents against review effort, we were able to forecast the resources required to uncover further relevant documents. Armed with accurate forecasts of the value of continued review, the legal team considered they had enough documents about the facts in issue and couldn’t justify the additional expense of continuing review of nearly four million documents.

So how is the use of TAR defensible to a party that must discharge their obligations to undertake a reasonable search for documents? A good tiramisu has many layers that can be tasted in each spoonful, like a good search strategy.

The first layer may ask custodians to reveal relevant documents, interviews with the IT department help locate where relevant data is stored, searches are undertaken on the data for keywords, people, and dates of authorship, analytics help to visualise, group, and de-duplicate documents and its all topped off with a creamy dollop of TAR consumed during the review process to help get the relevant material sooner. Each layer in the process has an impact on the next and no layer acts in isolation.

The defensibility of the search relies on the multiple enquiries made to locate documents and once we’re reviewing the results, assessing the value of continued review.

In terms of black letter acceptance of TAR, you’ll find American and Irish case law endorsing its use in matters with voluminous documents. In Australia you will find expressions of willingness to embrace “predictive coding” by the Australian Law Reform Commission and more generally for the parties and courts to consider technology that facilitates the efficient resolution of issues in dispute.

In practical terms the algorithm learns from relevant documents you’ve been made aware of or become aware of through a natural train of enquiry

Notwithstanding I’m promoting the value of TAR, if you haven’t tasted it on a case yet, you are in for something even more deliciously sweet from the next generation of document review platforms. CAL has been referred to by some as TAR 2.0 and solves some of the training transparency gaps in the TAR training process. A recent study by Cormack & Grossman measures the benefit of CAL over TAR and promotes the use of starting with the documents you know are relevant to train the algorithm – in practical terms the algorithm learns from relevant documents you’ve been made aware of or become aware of through a natural train of enquiry.

CAL isn’t a single process; it is ongoing learning from additional relevant and irrelevant documents as review progresses. It is an element in a search strategy that combines investigative searching with document review to bubble up other documents of likely interest. It’s the layer in the tiramisu that seeps through other search and review layers; luckily it has a sweet balancing flavour!

For me, CAL is significantly different to TAR due to how subtly software vendors are ingraining CAL into the review workflow and user experience. You can’t help but train the system and get the feedback – whether or not you use it. However, you tell me how many legal teams aren’t going to be enticed to view documents with a high score that haven’t been reviewed yet over those with low scores.

It is an exciting time to be an eDiscovery chef. The flavours are familiar with a new twist. The legal service’s consumer is maturing generally and seeking out better ways of tackling common and resource intensive problems. Large document review is no exception. With complex technology like CAL being delivered in a simple and integrated way, the consumer’s expectations for greater efficiency and pragmatism when approaching document review are elevating. I anticipate CAL and other similar technologies will become commonplace in eDiscovery, if not demanded by clients and firms to get the most out of their legal teams, in the not too distant future.

Now if you’ll excuse me, I’m off to whip up another delicious dessert!

Benjamin is Manager, eDiscovery & Forensics at NuLegal, Experts in eDiscovery and eTrial solutions. https://www.nulegal.com.au/