Introduction to Retrieval-Augmented Generation (RAG)

The era of Large Language Models (LLMs) has transformed the artificial intelligence landscape, particularly with the arrival of ChatGPT, which impressed users with its memorization and conversational capabilities. 

However, despite being trained on a massive data corpus, these models have certain limitations when it comes to answering questions about specific domains or recent events.

This is where the concept of Retrieval Augmented Generation (RAG) can help.

Fine-tuning vs. Retrieval Augmented Generation

Fine-tuning and RAG are two powerful techniques for specializing an LLM, but they operate differently. Fine-tuning directly adjusts the model's parameters to make it more specific, while RAG automatically enriches the questions asked to the LLM by adding relevant context.

The limitations of fine-tuning

• Need for large volumes of data : To achieve good performance, fine-tuning requires high-quality, diverse data in large quantities.
• Expensive infrastructure : The fine-tuning process requires powerful infrastructure, particularly GPUs.
• Data updates : It is difficult and expensive to regularly update the data used for fine-tuning.
• Hallucinations : The problem of hallucinations (incorrect or invented answers by the model) persists.

RAG, on the other hand, overcomes these limitations by automatically adding relevant context to questions.

What is RAG?

Natural RAG combines two main techniques: information retrieval and text generation. This method allows for the automatic addition of relevant context to questions asked of the LLM, thereby increasing the accuracy and relevance of the answers.

How does RAG work?

1. Retrieval : When a question is asked, the RAG system begins by searching for the most relevant documents that might contain the answer. This search is performed using similarity techniques between the question and document segments (chunks).
2. Generation : Once the relevant documents are retrieved, the LLM uses this contextual information to generate an accurate response.

Example of a response without RAG

Let's look at a concrete example to illustrate how RAG works. 

In the diagram below, the user asks a question about our company, Aqsone. Since the LLM has not been trained on data associated with the company, it is unable to answer, or it will provide an answer based on a hallucination.

‍

Example of a response with RAG

In this new diagram, RAG is used. We provide the RAG system with documents in which to search for information related to the question. For similarity between the question and the documents, it determines which document(s) contain information that could potentially answer the question (Retrieval). The question associated with these context elements are submitted to the LLM, which can then use them to answer the question (Generation).

‍

Learn more

To learn more about the technology behind RAG, we can refer to the following diagram and steps: 

• Document integration:
  • The documents are transformed into chunks, meaning they are broken down into several pieces of text
  • These chunks are transformed into vectors by an embedding model
  • The vectors are stored in a database
• Processing and answering the user's question:
  • The user's question is also transformed into a vector
  • A similarity analysis is performed between this vector and the vector database to return X chunks of context
  • The question and the text associated with the X context chunks are sent to the LLM, which can answer the question

‍

Advanced RAG techniques

RAG can be optimized using various advanced techniques, such as:

• Reranking : Reordering retrieved chunks by relevance to improve response quality.
• Embedding model fine-tuning : Specializing the embedding model for specific input texts.
• Multi-query : Formulating multiple variations of a question to increase the chances of retrieving relevant information.
• Self-query : Using automatically constructed queries to refine context retrieval.

Use cases and demonstrations

RAG (Retrieval Augmented Generation) has varied and powerful applications across many fields. Here are some concrete examples where RAG can provide significant added value:

IT support chatbots

Description : These chatbots are designed to answer frequently asked questions and resolve user issues based on a knowledge base derived from Q&As and past support tickets.

Example : An employee tries to connect to a secure network but fails. They ask the IT support chatbot what to do. The RAG system searches through past tickets and FAQs, finds a documented solution for this type of problem, and provides clear steps to resolve the issue.

Employee onboarding chatbots

Description : These chatbots assist new hires by providing the information and resources needed for their integration into the company, based on internal documentation.

Example : A new employee asks how to submit a time-off request. The chatbot retrieves the sections of the HR documentation that describe the leave request procedure and provides a detailed response with step-by-step instructions.

Discover, for example, our use case for HR assistant chatbot.

Industrial diagnostic support chatbots

Description : These chatbots support maintenance teams by providing rapid diagnostics for equipment malfunctions, based on sensor data and historical records of similar failures.

Example : A technician notices abnormal vibration on a machine and queries the chatbot, which identifies a potential bearing issue by drawing on past cases. The chatbot then provides diagnostic steps and repair recommendations, helping resolve the issue more efficiently.

Contract analysis chatbot

Description : These chatbots are designed to help users review and understand contract clauses. They can also identify potential unfair terms.

Example : A user submits a contract to the chatbot and asks if certain clauses might be unfair. The system analyzes the document, flags specific clauses such as excessive late penalties or unusual termination conditions, and presents them to the user while explaining the associated risks.

Conclusion

Retrieval-Augmented Generation represents a significant advancement in the field of AI, offering effective solutions to the limitations of traditional LLMs. By combining retrieval and generation, RAG improves the accuracy, relevance, and specialization of responses, making interactions with AI systems more useful and pertinent.