Worry Me, Natural Language Processing Doesn’t

By Adrian Lee, Gartner

The Newsweek headline screamed: “Robots Can Now Read Better Than Humans, Putting Millions of Jobs At Risk” . I nearly drowned laughing into my morning coffee. It was a slow news day?

The story described a reading accuracy test that pitched Alibaba’s natural language processing (NLP) AI versus a ‘rival human’.  Yes, the robot outdid the human by a score of 82.44 versus 82.305. Yes, that’s right, a difference of 0.135. 

The article did not continue on to reveal which were the millions of jobs at risk. So, the alarmist stance that massive job cuts were on the horizon didn’t worry me. Yet. To put things into perspective, Gartner currently has over 900 research articles on the topic of conversational platforms, with more than 46 different analysts investigating the area of conversational AI.

Oh, Natural Language Processing (NLP) is the comprehension by computers of the structure and meaning of human languages, allowing users to interact with the computer using natural sentences.

NLP features are dynamic and are evolving within conversational platforms. The one refrain I have for our clients when I advise on conversational platforms is this: we have so much more to learn.

The ‘processing’ of structured language has increased dramatically, especially in the advancement of entity recognition, machine translation, and text categorisation driven by ever faster and low latency machine learning instances in the cloud.  But the ‘understanding’ of meaning – inference, sentiment, relations, and variations, are still work in progress.

Seek not to process, but to understand. By which, I refer to Natural Language Understanding (NLU), and that’s important. for you to see the big picture. Below. I meant it, literally.

We still need to make sure the semantic aspects of the technology work. This is what drives the ‘understanding’ of human by machine.  As chatbots proliferate across customer service, financial services, retail, government and even healthcare; the ability of technology vendors to provide contextually aware, multiturn-based natural-language conversations presents an ongoing challenge to natural language understanding (NLU) handling. This is especially the case in the areas of semantic parsing and natural-language inferences.

Ambiguities in semantics arise when there are multiple grammatical interpretations possible. “Giant road bike” and “Giant road bike” on a digital commerce site can mean very different outcomes if the semantics engine cannot infer that “Giant” refers to a brand of bicycle and not the size of the bike in question. But you already knew that.

I shall stop here. If you want to read more about NLP or NLU and how it impacts digital commerce, please click here . Or we could get that same bot to do it.

Either way, you shouldn’t let it worry you.