Comment Wikit améliore la fiabilité de ses chatbots ?

Qu’est-ce que les hallucinations d’IA générative pour un chatbot ?

Chatbots based on generative Artificial Intelligence are marvels of technology, capable of autonomously producing text by imitating human language. However, despite their impressive capabilities, these systems are not immune to what are commonly known as hallucinations or generation errors. Hallucinations in the context of generative AI refer to outputs or productions generated by an artificial intelligence model, which may be strange, unexpected or completely disconnected from reality, thus calling into question the veracity of the results obtained, often due to the limitations of its training or training data. The chatbot may, for example, invent information on a subject it doesn't really know, or answer a question incorrectly due to a misinterpretation of the context.

The origins of hallucinations

Les « hallucinations » dans les chatbots basés sur l’IA générative trouvent leurs origines dans plusieurs facteurs. Tout d’abord, ces systèmes dépendent fortement des données d’entraînement sur lesquelles ils sont basés. Si ces données sont incomplètes, biaisées ou peu représentatives, le chatbot peut avoir du mal à produire des réponses précises et pertinentes. En effet, les chatbot basés sur IA générative ne sont pas magiques ! Pour fonctionner et générer des réponses ils doivent être alimentés par des sources documentaires en quantité suffisante et de grande qualité pour fournir une base solide afin de former le chatbot, le tester puis identifier les ajustements nécessaires.

In addition, the large language models (LLMs) used by chatbots may have gaps in their understanding of natural language, making them likely to produce erroneous or inappropriate responses. Finally, chatbots may be confronted with ambiguous or unfamiliar situations, which may result in unpredictable or confusing responses for the user.

Comment Wikit limite les hallucinations et améliore la fiabilité de ses chatbots ?

At Wikit, managing hallucinations is a major technical challenge, and we're doing everything we can to improve their detection and limit them. Our approach is based on a number of essential steps to ensure that our AI models function optimally and that our chatbots are reliable.

1. Careful data preparation

La base de connaissances est le socle sur lequel le chatbot s’appuie pour répondre aux requêtes des utilisateurs. Plus l’agent conversationnel disposera de données pertinentes en abondance, mieux il comprendra les requêtes des utilisateurs. En effet, la fourniture de données structurées de haute qualité au modèle de langage (LLM) est l’une des principales conditions préalables à la mise en œuvre de cas d’utilisation d’IA générative. Des données structurées permettent au modèle de comprendre les relations entre les différentes entités, contextes et significations. Cela améliore significativement la pertinence des réponses générées par le LLM, favorisant ainsi une meilleure compréhension des requêtes posées. Ainsi, pour créer un chatbot qui repose sur une IA, il est essentiel de constituer une base de connaissances basées sur un ensemble de données de haute qualité, représentatives et actualisée, afin de fournir au chatbot une base solide sur laquelle s’appuyer pour générer des réponses précises. Ainsi, si les informations données au chatbot ne sont plus d’actualités, les réponses seront forcément erronées.

At Wikit, our teams guide customers in the creation of their knowledge base to ensure that it contains all the information that covers the chatbot's scope, no more, no less. In particular, they help themselect useful documents, pass on best practices for formatting these documents, and help limit and reformat elements that are more difficult to interpret, such as images.

In addition, our teams are actively working on data formatting, to make it as easy as possible for AI to handle. Indeed, data sources are many and varied in nature (PDF, Word, Markdown documents, Powerpoint presentations, websites, etc.), requiring the development of specific formatting methods for each one. For example, in order to limit the amount of information sent to the chatbot at any one time, documents need to be broken down into fragments. Cutting up these fragments is a crucial step, as it must enable the creation of relevant fragments, which contain a group of standardized, coherent, condensed information, accompanied by the context required to understand them. The rest of the chain depends on the quality of this breakdown.

2. AI models coupled with proprietary RAG technology

Chez Wikit, nous couplons notre IA à un technologie RAG propriétaire (Retrieval Augmented Generation, ou Génération Augmentée de Récupération). Cette approche combine deux étapes : une étape de récupération d’informations (Retrieval) chargée de trouver les informations pertinentes en réponse à une requête donnée, et une étape de génération de langage qui prend ces informations comme contexte et les aggrège dans une réponse plus détaillée et plus naturelle (Generation).

En combinant ces différentes étapes, la méthode RAG permet aux chatbots d’IA générative de fournir des réponses plus pertinentes et plus précises aux requêtes des utilisateurs. Elle tire parti à la fois de la capacité des modèles de génération de langage à produire un langage naturel et de la capacité des systèmes de récupération d’informations à trouver rapidement des réponses pertinentes dans un ensemble de données. Cela se traduit par une expérience utilisateur améliorée et une plus grande capacité du chatbot à comprendre et à répondre aux besoins des utilisateurs.

3. Conditioning and training the chatbot

At Wikit, we adopt an approach called "grounding", which involves clearly defining the areas in which the chatbot excels, and limiting its interactions to the topics it has mastered. We "educate" our chatbots by conditioning them to interact on a well-defined scope, so that they only answer questions for which they have been trained. The aim is to limit the model's initial knowledge by training the algorithm with the company's "only" data. In addition, it is possible to ask the models not to provide answers if the required information is not present in the training data, and especially not to establish connections that are not explicitly present in the training data. In this way, models are much less likely to hallucinate, which considerably increases the quality of the results.

Once the data has been prepared, our approach is to test the chatbots with a limited audience of users to assess the validity of the responses generated, identify any necessary adjustments and make corrections. It's important to evaluate not only the chatbot's input queries, but also its responses. To do this, we adopt a rigorous evaluation process, using a battery of tests to assess how well the chatbot behaves in the face of inappropriate (so-called "toxic") requests, such as insults or malicious acts. It is of the utmost importance that our chatbots' instructions are as robust as possible in the face of such requests.

Then, as confidence in its capabilities grows, its scope can be extended. However, control must be maintained by clearly defining expectations and ensuring that users understand the bot's limitations.

4. Source display

At Wikit, we've chosen to focus on AI that's as transparent as possible. With this in mind, we offer users of our chatbots the opportunity toexplore the exact sources of the answers generated, to enable them to understand the levers that led to these answers. Ultimately, the aim will be to eliminate the various layers of the decision-making process, to create greater confidence in AI's capabilities and, ultimately, lead to a more efficient chatbot tailored to our customers' needs. Transparent AI does not guarantee perfect explainability, but it does ensure that every answer is not only accurate, but also verifiable and understandable.

5. Setting up a disclaimer and transferring it to a human being

Despite all these measures, generative AI chatbots are not magic, and hallucinations cannot be completely eliminated. That's why we systematically include a disclaimer on all our chatbots , inviting users to verify important information in the original documents.

On the other hand, it's inevitable that the chatbot won't be able to answer every query or question posed by users, especially in the early stages of deployment, when it may encounter limitations in its understanding or answer-generating capabilities. To avoid any frustration on the part of users, it' s essential that the chatbot clearly admits when it can't answer, and facilitates transfers to a human agent. This ensures a positive user experience, even when the chatbot fails to respond.

Conclusion

Managing hallucinations is one of the main technical challenges facing generative AI chatbots. At Wikit, we are implementing appropriate solutions to detect and limit hallucinations, and we are making daily progress towards more reliable and efficient chatbots, capable of providing accurate and relevant responses, in a variety of user contexts.

However, we feel it is essential to remember that generative AI chatbots are not magic tools. They should therefore not be seen as an alternative to humans, but rather as complementary tools that, when correctly configured and used, will greatly contribute to improving the efficiency and productivity of the companies that deploy them.

Don't miss our next resources