RAGFlow an Open-Source Retrieval-Augmented Generation (RAG) Engine

RAGFlow is an open-source Retrieval-Augmented Generation (RAG) engine that integrates with Large Language Models (LLMs) to provide accurate, citation-backed answers from complex data formats. The RAGFlow Developer Documentation offers comprehensive guidance for deploying, configuring, and utilizing RAGFlow effectively.

Architectural Components

The following is the architecture:

Data Ingestion and Processing:

• RAGFlow accepts diverse data formats, including PDFs, Word documents, and web pages.
• The Deep Document Understanding module processes these inputs, extracting and structuring information for effective retrieval.

Knowledge Base Management:

• Processed data is stored in a knowledge base, organized to support efficient search and retrieval operations.
• The system employs advanced indexing and chunking strategies to enhance retrieval accuracy.

Retrieval Mechanism:

• Upon receiving a user query, RAGFlow searches the knowledge base to identify relevant information.
• It utilizes multiple recall strategies combined with fused re-ranking to prioritize the most pertinent results.

Language Model Integration:

• RAGFlow integrates with configurable LLMs to generate responses based on retrieved information.
• This integration ensures that generated answers are both contextually relevant and grounded in the underlying data.

Citation and Response Generation:

• The system produces answers accompanied by citations, providing transparency and traceability to the original data sources.
• This feature enhances the reliability and trustworthiness of the responses.

User Interface and API Access:

• RAGFlow offers an intuitive user interface for direct interaction.
• Additionally, it provides APIs that facilitate seamless integration with other business applications, enabling automated workflows and extended functionalities.

Deploying RAGFlow Locally

To deploy RAGFlow locally, ensure your system meets the following prerequisites:

Hardware Requirements:

• CPU: 4 cores (x86)
• RAM: 16 GB
• Disk Space: 50 GB

Software Requirements:

• Docker: Version 24.0.0 or higher
• Docker Compose: Version 2.26.1 or higher

Before starting the server, adjust the vm.max_map_count setting to at least 262144 to ensure Elasticsearch functions correctly. This can be done using the following command:

sudo sysctl -w vm.max_map_count=262144

To make this change permanent, add vm.max_map_count=262144 to your /etc/sysctl.conf file.

With the prerequisites in place, you can start the RAGFlow server using Docker Compose:

docker compose -f docker/docker-compose.yml up -d

After the server is running, access the RAGFlow interface by navigating to http://<IP_OF_YOUR_MACHINE> in your web browser.

Configuring Language Models

RAGFlow requires integration with an LLM to provide accurate, grounded responses. It supports various models, including those from OpenAI, Ollama, and others.

To configure an LLM:

1. Click on your profile logo in the top-right corner of the RAGFlow interface.
2. Select “Model providers” from the dropdown menu.
3. Add your desired LLM by entering the necessary API keys and configuration details.

Ensure that the models you add are properly configured to interact with RAGFlow.

Creating a Knowledge Base

A knowledge base in RAGFlow is a collection of datasets derived from various documents. To create one:

1. Navigate to the “Knowledge Base” tab at the top of the page.
2. Click “Create knowledge base” and provide a name for it.
3. Configure the embedding model and chunking method suitable for your documents.
4. Upload your documents in supported formats such as PDF, DOCX, XLSX, and others.
5. After uploading, initiate the parsing process to convert documents into datasets.

RAGFlow offers multiple chunking templates tailored to different document layouts, enhancing the accuracy of information retrieval.

You can test retrieval by navigating to the “Retrieval testing” section:

In the example above, I am testing retrieval on the document that I uploaded to my “super-secret-data” knowledge base.

Creating Assistants

You can create assistants by navigating to “Chat” and then clicking on Add.

The figure above shows the “Chat Configuration” interface for creating a specialized assistant within RAGFlow. The interface allows the user to configure the assistant’s settings, including:

1. Assistant Name — “Web Application Security Expert” is the designated name for this assistant.
2. Description of Assistant — A brief description labeled “Web app security expert assistant.”
3. Assistant Avatar — An option to upload an avatar for the assistant, currently not set.
4. Empty Response — A field that appears to handle situations where the assistant may not have an answer, but it is left blank.
5. Set an Opener — A greeting message for users: “Hi Omar! 👋 I’m your Web App Security assistant, what can I do for you?”
6. Show Quote — A toggle switch is enabled, suggesting that responses may include citations or referenced text.
7. Text to Speech — This toggle switch is off, meaning the assistant will respond via text rather than voice.
8. Knowledge Bases — The assistant is linked to a knowledge base titled “super-secret-data.” (in this case, I am using the “super-secret-data” knowledge base that I created earlier).

The “Prompt Engine” tab of the Chat Configuration interface in RAGFlow allows you to create system prompts and configure other parameters.

System Prompt (Expert Definition):

• The assistant is configured to act as a Web Application Security Expert, focusing on detecting and mitigating vulnerabilities like SQL injection, XSS (Cross-Site Scripting), and CSRF (Cross-Site Request Forgery), etc.
• The description clearly outlines the assistant’s role and responsibilities.

Similarity Threshold (Slider):

• This setting controls how closely retrieved results must match the input query. A higher value increases precision by reducing less relevant results.

Keyword Similarity Weight (Slider):

• Adjusts the importance of keyword matching when retrieving relevant documents or generating responses.

Top N (Slider):

• Determines how many of the top results are considered during query processing. A higher value retrieves more potential matches.

Multi-turn Optimization (Toggle):

• This is enabled, indicating that the assistant is optimized for multi-turn conversations, allowing for follow-up questions and contextual dialogue.

Rerank Model (Dropdown):

• There’s an option to select a reranking model, but no model is currently selected.

Variable Section (Key-Value Pairs):

• A variable labeled “knowledge” is added to reference the knowledge base associated with the assistant. This can influence how the assistant processes and references external data.

Intervening in File Parsing

RAGFlow provides visibility into the parsing process, allowing manual intervention to improve data accuracy. After parsing a document:

1. Click on the parsed file to view its chunking results.
2. Review each chunk and make necessary adjustments, such as adding keywords or modifying content.
3. Save your changes to ensure the knowledge base reflects accurate information.

This feature ensures that the data used for generating responses is precise and relevant.

Creating an Agent

RAGFlow introduces an innovative agent mechanism designed to enhance the capabilities of LLMs in business applications. This feature combines a no-code workflow editor with a comprehensive graph-based task orchestration framework, enabling users to create sophisticated agents tailored to specific needs.

Agents in RAGFlow serve as intermediaries that manage complex tasks by orchestrating various search technologies, including query intent classification, conversation leading, and query rewriting. This orchestration aims to:

• Enhance Retrieval Accuracy: By refining user queries and managing interactions, agents improve the precision of information retrieval from knowledge bases.
• Handle Complex Scenarios: Agents are equipped to manage multifaceted interactions, making them suitable for diverse business applications.

Creating an Agent

1. Navigate to the “Agent” tab located at the top center of the RAGFlow interface.
2. Click on “+ Create agent” to open the agent template page.
3. Choose the “Blank” template to build an agent from the ground up, or select from existing templates tailored to common scenarios, such as a general-purpose chatbot.

Use the no-code workflow editor to design the agent’s behavior by dragging and dropping components (operators) into the workflow.

Set the input and output parameters for each component, select the appropriate knowledge base, and adjust the Large Language Model (LLM) settings as needed.

For a detailed guide, visit the Introduction to Agents page on the RAGFlow documentation site.

Advanced Configurations

RAGFlow offers various advanced settings to optimize performance:

• Docker Environment Variables: Configure services like Elasticsearch, MySQL, and Redis by adjusting variables in the .env file.
• Service Configuration: Modify the service_conf.yaml.template file to set system-level parameters for RAGFlow’s API server and task executor.

Detailed information on these configurations is available in the Configurations section of the documentation.

Upgrading RAGFlow

To upgrade RAGFlow to the latest version:

1. Pull the latest Docker image:

docker pull infiniflow/ragflow:latest

2. Update your .env file to reference the new image version.

3. Restart the RAGFlow services:

docker compose down docker compose up -d

For detailed upgrade instructions, refer to the Upgrade RAGFlow guide.

The RAGFlow Developer Documentation provides a comprehensive roadmap for deploying and utilizing RAGFlow effectively.