Generative artificial intelligence (GenAI) is everywhere, and so are widespread, cross-industry discussions about how humans fit into the equation. In the world of intelligent document processing (IDP), when considering the promise of faster, scalable, more accurate processing, it’s no surprise that GenAI technology is on the forefront of everyone’s minds.

ChatGPT is, incredibly, only 2 years old, yet its immediate effect has been seismic. In just five days after its initial launch, the wildly popular chatbot had over one million users; today it has over 200 million weekly users. AI and deep learning (DL) systems, once reserved only for data scientists to understand, are now accessible to developers and other end users. And because of that accessibility, we’re seeing the growing appetite for its potential, especially in document processing.

As the interest in GenAI-powered intelligent document processing grows, so is the volume of voices asking big questions: How can we balance the efficiency of GenAI-powered IDP with maintaining high levels of accuracy? Are these processing systems secure? How can human oversight mitigate some of the risks?

Those concerns are real. Thankfully, a human-in-the loop (HITL) element can alleviate some fears and, if incorporated in a scalable way, can even make an AI solution faster and more accurate.

GenAI risks in IDP

When it comes to the use of large language model (LLM) technology, there is a choice between free, open-source approaches like ChatGPT and LLaMA, and private, proprietary models like GPT-4 and PaLM. Many assume that an open-source model lacks necessary safety measures, while a commercial approach can boast iron-clad security. In reality, it’s not that simple.

Consider an IDP solution that incorporates a third-party, private LLM, the data usage and retention terms of which don’t align with your specific compliance regulations. A chain is only as strong as its weakest link, and a non-compliant “link” is non-negotiable. We’ve seen examples of the outcomes of these weak links in the news, from a proposed class-action lawsuit against Google to an accidental leak of trade secrets by Samsung employees.

Businesses are rightfully concerned about disclosing sensitive data to LLMs. Countries around the world are implementing legislation relating to GenAI, most including privacy provisions. There are questions about the retention and redaction of sensitive information. How long should confidential data be retained by a third party? What kind of PII should be redacted to protect privacy?

Redact too much and you don’t see the advantages of automation; redact too little and you risk exposure. These instances further shine a spotlight on the need for human oversight in order to properly (and safely) leverage the technology. 

GenAI-powered Intelligent Document Processing rewards: who benefits?

With these potential pitfalls in mind, why assume that risk? Not surprisingly, the answer lies in the unique power of GenAI in IDP and the prospect of truly enhanced document analysis. Through its intricate neural networks, there are major improvements in noise reduction and image resolution. And because of its ability to understand context, it can visualize insights to extract relevant information and auto-fill missing data. The potential of GenAI in document processing is undeniable.

So, who does this affect? Clearly, the industries with the most documents. And which industries have the most documents? The same ones with the highest number of GenAI use cases: healthcare, insurance, and finance. Let’s take this out of the hypothetical and consider the capabilities in a real-life case.

How HITL approaches fill automation gaps in IDP

One ScaleHub client is a BPO that handles about 4.2 billion human resource documents per year. With candidate selection and employee onboarding processes come a tremendous volume of data – pre-screen info, background checks, health and insurance records – all of which contain personal identifiable information (PII). The client was looking for improved document processing speed, but never at the expense of data security. ScaleHub utilizes a HITL approach to meet both of these demands.

Enlisting ScaleHub’s GenAI-powered IDP service, the BPO was able to process 80K-100K documents with 13 different form types and 5 variations. Unstructured and varied documents are notoriously difficult to process, yet even at this volume, the total processing time needed was cut from five months to three weeks. Human work incorporated in a scalable way, when combined with all that AI can offer, isn’t just a safety net, it’s a benefit to an IDP solution’s accuracy and safety.

First, the BPO uploads the documents to ScaleHub’s ​​secure portal. Then, using a mix of AI and OCR, the documents are classified and the pertinent data is extracted. That data is then anonymized, snippeted, and sent to crowd contributors for validation.

This 2.3 million person pool of contributors makes the processing of any volume of documents possible; the humans are quite literally putting the “scale” in ScaleHub. Because the snippets are without context, separated from the rest of the document, total security remains fully intact.

After the data is verified by the crowd, the documents return to the client in whichever file type is needed: JSON, pdf, etc. Finally, when processing is complete, all files are permanently deleted within one to two weeks, eliminating the risk of potential exposure. 

The power couple: AI and human expertise

Like any good partnership, the whole is greater than the sum of its parts. One party picks up where another party leaves off, ideally with complementary strengths. Yes, it’s true: GenAI-powered Intelligent Document Processing technology is disrupting the IDP landscape. But we humans are pretty great, too. Our ability to adapt, think critically and creatively, and fully understand complex industry-specific concerns are irreplaceable. The key is combining the two in a scalable way.

Torsten Malchow is Chief Revenue Officer at ScaleHub.