How do we analyze user input?

Natural Language Understanding

Natural language understanding (NLU) is a subtopic of natural language processing in artificial intelligence that deals with machine reading comprehension.

Language understanding is a fairly complicated task as it involves understanding the meaning of the sentence, extracting the entities, and making a decision regarding the action to be taken.

The NLU's ability to understand the user relies on training of knowledge domains. This means that we can expand NLU capability to understand new domains by adding a new knowledge domain for it to train on.

The main goal is to understand user textual input and convert it into structured data that holds, among other things, the extracted entities, an action to be taken, and a textual response for the user.

How do we classify?

The main tasks we perform in the classification node are Entity extraction and Intent classification:

1. Entity extraction - extract interesting parts of the text like names and locations.

2. Intent classification - understand the intention of the text and classify it into predefined classes (each intent represents a class).

Our classification pipeline consists of a few steps:

Text Preprocessing

Use advanced techniques to normalize the text: perform text lemmatization - tag the Part of Speech (noun, verb, adjective, etc.) to find the important parts of the text, remove stop-words (words such “a”, “an” “the”).

Vectorization

Represent the text by numbers. We use a semantic representation, meaning word synonyms have an identical representation.

Classification

Use a machine learning fine-tuned classification algorithm to get the final classification label.