What is natural language processing (NLP)?
NLP, short for “natural language processing,” stands for any type of automated processing of speech or text. As such, the discipline is already quite old: In theory, it has been around since the 1950s. In 1954, for example, a team of researchers promised that automated translations would be a problem solved in 5 years.
But here you also have to make it very clear that there is still a huge difference between “processing” and “understanding.” (By the way, Alexa, Siri and Co are also quite a bit away from the latter). However, this is often not necessary at all to complete initial and “simple” tasks. In many languages, the structure of sentences is already quite different — who doesn't remember the first English grammar lessons “Subject -> Predicate -> Object”?
In recent years, NLP has increasingly crossed over with the topic of “artificial intelligence” and “machine learning.” Progress there in particular has developed into incredible, until recently unimaginable, opportunities. More about that later.
What is NLP used for and what makes it so interesting right now?
Covering the full spectrum in this article is of course not possible, but a few examples should make it clear how versatile language is automatically processed in our everyday lives. Because that too has changed in the last 3-4 years: Using NLP has become much easier for coders and for end users!
Before we get closer to the examples, a short trip to shed light on why we may be in the golden age of NLP right now.
The fact that language and communication have special significance for us humans is nothing new. What has changed, however, is that language and texts are freely available on the Internet in all possible forms in incomprehensible quantities and to a large extent. (Unfortunately not in all parts of the world and certainly not for everyone. And I don't mean dead spots “in the countryside.”)
As mentioned above, NLP is now very closely linked to machine learning or AI. And AI simply (almost) always works much better when it receives more input to learn. Since many clever people have been working on this recently, new ideas and lots of computing power have made this available wealth of learning more and more accessible. And luckily for us, these people make the learning results available to the Internet community, the professional likes to talk about “trained models.” What they did exactly is a story for another day.
Even mainstream news reported on the GPT-3 OpenAI language model in 2022 and then on the newer model GPT-4. And it is thanks to the hard work of the open source community that these models can then also be more easily integrated into “normal programs.” In this way, we can let our creativity run wild when using it without first taking courses on AI at an “Ivy League” university.
Latest NLP examples
Here are a few exciting examples of what NLP can already do today.
Even though I smiled a bit in the first section about how long good automated translations should have existed, it has to be said that translations that you can now get from industry giants such as deepl.com for full sentences or sections between various languages are already of a fairly high level. And there are initial approaches (this is the connection with many other machine learning techniques, even beyond NLP) that the computer plays the simultaneous translator in video chats. That is far from being as good as a Babel fish (if you don't know him — read Douglas Adams' “Hitchhiker's Guide to the Galaxy”!) , but fascinating enough as it is.
Before we get to what moinAI NLP can do, here are two of my favorite applications, because they show that we can use the language models But there has already been a long way towards “understanding language”: With the help of such models, an NLP algorithm can answer questions about a (longer) text. What is already working quite acceptably is hidden behind the technical term “extractive question answering.” An answer isn't really formulated here yet, but if you have a question about a long Wikipedia article, for example, then the algorithm says in which section or sentence of the article I'm likely to find an answer to my question.
This NLP algorithm, so to speak, takes readers by the hand and guides them to the passage of text they are looking for. A very similar metaphor to the one we often use for using moinAI on websites. Finally, he takes AI chatbot from moinAI also provides website visitors with assistance and guides them to their desired destination, with the chatbot either answering the question directly or providing assistance and appropriate links so that users can navigate better.
But NLP can also “generate language” itself. In this way, models such as GPT-3 (the freely available predecessor GPT-2 was also able to do this quite well) can continue texts. Think of it as if you're telling the beginning of a story, and then the computer continues that story. This is still more in the “toy phase” at the moment, but the results are already impressive.
NLP chatbot — what makes moinAI's NLP stand out?
And what does all this have to do with moinAI? And even more importantly, what do we do differently or even better than others?
Among other things, we use one of the more sophisticated disciplines of NLP, which, with the newly available models and methods, reaches a completely different level than just a few years ago. The technical term is”Intent Detection“.
What is hidden behind this is the following: Imagine that you receive a lot of inquiries from customers or employees in your company. From your practical experience, you know pretty much exactly which topics these inquiries need to be addressed to. It often takes a specialist like you to identify which topic is meant by a specific question. The beauty is — the moinAI chatbot It's very easy to prepare to be that specialist! moinAI recognizes which topics are meant by a question. Once the topic is identified, a suitable pre-formulated answer can of course be played back.
To make this work, we also use language models specially selected by us. With these, moinAI is particularly easy to recognize when certain sentences are similar in their content. That's why just a few examples are enough for MoinAI AI to identify a topic.
Gain customer insights with AI & NLP
But moinAI is not over at the point mentioned above: The knowledge about the similarity of sentences is used as part of another feature. moinAI makes its own AI suggestions when the AI registers that there are many inquiries on topics for which there are no pre-formulated answers at all. In this way, MoiniAI helps companies to respond more and better to the needs of their customers. By the way, this AI feature is called Dreaming. Dreaming makes it possible that the KI develops independently and at the same time gives companies an insight into the wishes and questions of users.
A detailed explanation of this AI feature, which can be used to generate new suggestions for topics independently, is also available in this video:
What is the quality of NLP?
Our sophisticated algorithms are certainly a big help here. But what sets us apart is the combination of human and artificial intelligence. Similar to humans, the learning outcome of NLP and machine learning models depends very much on the teacher and the learning content.
That's why our customer success managers ensure right from the start that only good and meaningful example sentences and questions are included in the AI learning process. They design the scope and type of topics in such a way as to create the best possible experience for the end customer.
The many years of experience of Customer success teams and the AI teams Overall, from moinAI ensure that the sometimes very fine and difficult techniques that work in the background do not pose any hurdles for moinAI customers — and that the focus can therefore be directed accordingly on the content that companies want to communicate using a chatbot.
The NLP should be used as simply and easily as possible by moinAI customers. After all, you use your own smartphone even without understanding what exactly happens behind the display with the ever smaller and ever faster chips.
Anyone who has become even more interested in the sophisticated techniques is also welcome to talk to us directly — we will be happy to answer any questions you may have about moinAI and NLP personally. Click here for the contact option.
