完整指南：构建具备自评分和网络搜索能力的高级rag系统！
检索增强生成 (RAG) 系统
检索增强生成 (RAG) 系统在构建可靠的人工智能应用程序中变得越来越重要。在本文中，我们将探讨构建一个先进的 RAG 系统，该系统结合了本地文档检索、动态网页搜索功能和使用 Pydantic.ai 和 Groq 的自评分机制。
PydanticAI
PydanticAI 是一个 Python 代理框架，旨在减少使用生成式人工智能构建生产级应用程序的痛苦。
为什么使用 PydanticAI
- 由 Pydantic 背后的团队构建（OpenAI SDK、Anthropic SDK、LangChain、LlamaIndex、AutoGPT、Transformers、CrewAI、Instructor 等的验证层）
- 模型无关 — 目前支持 OpenAI、Gemini、Anthropic 和 Groq，并且有一个简单的接口来实现对其他模型的支持。
- 类型安全
- 控制流和代理组合使用普通 Python 完成，使您能够使用在任何其他（非 AI）项目中使用的相同 Python 开发最佳实践
- 使用 Pydantic 进行结构化响应验证
- 流式响应，包括使用 Pydantic 对流式 结构化 响应的验证
- 新颖的、类型安全的依赖注入系统，对测试和基于评估的迭代开发非常有用
- Logfire 集成用于调试和监控您的大语言模型驱动应用程序的性能和一般行为
技术栈
我们的系统利用了几种现代人工智能技术：
- TensorFlow
- PyTorch
- Scikit-learn
特性
- 可扩展性
- 灵活性
- 性能
代码示例
以下是使用 TensorFlow 的简单示例：
import tensorflow as tf
## Define a constant
hello = tf.constant('Hello, TensorFlow!')
print(hello)
技术表
| 技术 | 描述 |
|---|---|
| TensorFlow | 一个开源机器学习库 |
| PyTorch | 一个深度学习框架 |
| Scikit-learn | 一个机器学习库 |
图像示例
核心组件：
- Pydantic.ai 用于类型安全的 AI 代理
- Groq 的 llama-3.3-70b-多功能模型用于 LLM 推理
- LangChain 用于文档处理
- ChromaDB 和 HuggingFace嵌入
- Tavily 用于网页搜索
支持库:
- sentence-transformers 用于嵌入
- PyPDF 用于文档加载
- nest_asyncio 用于 Jupyter 中的异步操作
系统架构
1. 文档处理和索引
loader = PyPDFLoader("path/to/document.pdf")
documents = loader.load()
split_docs = RecursiveCharacterTextSplitter(
chunk_size=500,
chunk_overlap=50
).split_documents(documents)
vectorstore = Chroma.from_documents(
documents=split_docs,
embedding=embedding,
persist_directory=persist_directory
)
2. 代理配置
系统使用一个复杂的代理设置，具有多种功能。
groq_agent = Agent(
groq_model,
deps_type=Deps,
retries=2,
result_type=str,
system_prompt="""You are a Helpful Assistant Proficient in
Answering concise, factual and to the point answers..."""
)
3. 检索工具
该系统包含一个自定义的检索工具，用于访问本地文档：
@groq_agent.tool
async def retriever_tool(ctx: RunContext[Deps], question: str) -> List[str]:
load_vectorstore = Chroma(
persist_directory=persist_directory,
embedding_function=embedding
)
docs = load_vectorstore.similarity_search(question, k=3)
return [d.page_content for d in docs]
4. 网络搜索集成
使用 Tavily 实现了一个网络搜索工具：
@groq_agent.tool_plain
async def websearch_tool(question) -> str:
tavily_client = TavilyClient()
answer = tavily_client.qna_search(query=question)
return answer
关键特性
1. 自评分机制
系统根据三个标准评估其自身的响应：
- 相关性 (0–1)
- 对上下文的忠实度 (0–1)
- 上下文质量 (0–1)
{
"Relevancy": 0.9,
"Faithfulness": 0.95,
"Context Quality": 0.85,
"Needs Web Search": false,
"Explanation": "Response directly addresses the question..."
}
2. 动态上下文增强
系统可以自动决定何时通过网络搜索增强本地知识：
if grades["Needs Web Search"]:
web_results = await websearch_tool(query)
3. 结构化依赖
使用 Pydantic 进行类型安全：
@dataclass
class Deps:
question: str | None
context: str | None
代码实现
-
安装所需的依赖
%pip -q install pydantic-ai %pip -q install nest_asyncio %pip -q install devtools %pip install 'pydantic-ai-slim[openai,groq,logfire]' %pip install tavily-python %pip install -qU langchain %pip install -qU langchain_community %pip install -qU sentence_transformers %pip install -qU langchain_huggingface %pip install pypdf -
在 Google Colab 中设置 API 密钥
from google.colab import userdata import os os.environ["OPENAI_API_KEY"] = userdata.get('OPENAI_API_KEY') os.environ["GROQ_API_KEY"] = userdata.get('GROQ_API_KEY') -
实例化大语言模型 (LLM)
from pydantic_ai import Agent from pydantic_ai.models.openai import OpenAIModel from pydantic_ai.models.groq import GroqModel openai_model = OpenAIModel('gpt-4o-mini') groq_model = GroqModel("llama-3.3-70b-versatile") -
加载所需文档并构建索引
from langchain_community.vectorstores import Chroma from langchain_community.embeddings import HuggingFaceEmbeddings from langchain.document_loaders import PyPDFLoader from langchain.text_splitter import RecursiveCharacterTextSplitter loader = PyPDFLoader("/content/data/Fibromyalgia_Final.pdf") documents = loader.load() split_docs = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=50).split_documents(documents) embedding = HuggingFaceEmbeddings(model_name="sentence-transformers/all-MiniLM-L6-v2") vectorstore = Chroma.from_documents( documents=split_docs, embedding=embedding, persist_directory=persist_directory, collection_name="fibromyalgia" ) -
定义依赖
PydanticAI 使用依赖注入系统为您的代理提供数据和服务。依赖可以是任何 Python 类型。在简单的情况下，您可能能够将单个对象作为依赖项传递（例如，HTTP 连接），但当您的依赖项包含多个对象时，数据类 通常是一个方便的容器。
from dataclasses import dataclass @dataclass class Deps: question: str | None context: str | None -
实例化 PydanticAI 代理
代理是 PydanticAI 与大语言模型 (LLM) 交互的主要接口。
在某些用例中，单个代理将控制整个应用程序或组件，但多个代理也可以相互交互，以体现更复杂的工作流。
代理类具有完整的 API 文档，但从概念上讲，您可以将代理视为一个容器，包含：
- 系统提示 — 开发人员为 LLM 编写的一组指令
- 一个或多个功能工具 — LLM 可能调用的函数，以获取生成响应时所需的信息
- 可选的结构化结果类型 — LLM 在运行结束时必须返回的结构化数据类型
- 依赖类型约束 — 系统提示函数、工具和结果验证器在运行时可能都会使用依赖项
- 代理还可以选择关联一个默认的 LLM 模型；在运行代理时也可以指定要使用的模型
import nest_asyncio nest_asyncio.apply() groq_agent = Agent(groq_model, deps_type=Deps, retries=2, result_type=str, system_prompt=("You are a Helpful Assistant Proficient in Answering concise, factful and to the point answers for questions asked based on the Context provided." "You have to Use the `retriever_tool' to get relevant context and generate response based on the context retrieved." """You are a grading assistant. Evaluate the response based on: 1. Relevancy to the question 2. Faithfulness to the context 3. Context quality and completeness Please grade the following response based on: 1. Relevancy (0-1): How well does it answer the question? 2. Faithfulness (0-1): How well does it stick to the provided context? 3. Context Quality (0-1): How complete and relevant is the provided context? Question: {ctx.deps.query} Context: {ctx.deps.context} Response: {ctx.deps.response} Also determine if web search is needed to augment the context. Provide the grades and explanation in the JSON format with key attributes 'Relevancy', 'Faithfulness', 'Context Quality', 'Needs Web Search': {"Relevancy": <score>, "Faithfulness": <score>, "Context Quality": <score>, "Needs Web Search": <true/false>, "Explanation": <explanation>, "Answer":<provide response based on the context from the `retriever_tool' if 'Need Web Search' value is 'false' otherwise Use the `websearch_tool` function to generate the final response}""" ), ) -
功能工具
功能工具提供了一种机制，使模型能够检索额外信息，以帮助生成响应。
当将代理可能需要的所有上下文放入系统提示中不切实际或不可能时，它们非常有用，或者当您希望通过将生成响应所需的一些逻辑推迟到另一个（不一定是 AI 驱动的）工具来使代理的行为更具确定性或可靠性时。
功能工具基本上是 RAG（检索增强生成）的“R” — 通过让模型请求额外信息来增强模型的能力。
PydanticAI 工具与 RAG 之间的主要语义差异在于 RAG 与向量搜索同义，而 PydanticAI 工具则更通用。
有多种方法可以将工具注册到代理中：
- 通过
@agent.tool装饰器 — 用于需要访问代理 上下文 的工具 - 通过
@agent.tool_plain装饰器 — 用于不需要访问代理 上下文 的工具 - 通过
tools关键字参数传递给Agent，可以接受普通函数或Tool的实例
- 通过
-
创建网络搜索工具
@groq_agent.tool_plain async def websearch_tool(question) -> str: """check if the square is a winner""" tavily_client = TavilyClient() answer = tavily_client.qna_search(query=question) print(f"WEB SEARCH:{answer}") return answer -
创建检索工具
@groq_agent.tool async def retriever_tool(ctx: RunContext[Deps], question: str) -> List[str]: load_vectorstore = Chroma(persist_directory=persist_directory, embedding_function=embedding, collection_name="fibromyalgia") docs = load_vectorstore.similarity_search(question, k=3) documents = [d.page_content for d in docs] print(f"RAG Retrieval:{documents}") return documents -
结果 — 调用代理
结果是从 运行代理 返回的最终值。结果值被包装在
RunResult和StreamedRunResult中，因此您可以访问其他数据，如 运行成本 和 消息历史。当收到普通文本响应或模型调用与某个结构化结果类型相关联的工具时，运行将结束。问题1
query = "What is Fibromyalgia?" response = groq_agent.run_sync(query) print(response)
RAG 检索
/usr/local/lib/python3.10/dist-packages/logfire/_internal/stack_info.py:107: LogfireNotConfiguredWarning: No logs or spans will be created until `logfire.configure()` has been called. Set the environment variable LOGFIRE_IGNORE_NO_CONFIG=1 or add ignore_no_config=true in pyproject.toml to suppress this warning.
warnings.warn(msg, stacklevel=stacklevel, category=category)
获取响应
在这里，我们可以看到代理首先调用了检索代理，然后评估了响应，发现答案是相关的，并且不需要进行网络搜索。
response.data
{
"Relevancy": 0.8,
"Faithfulness": 0.9,
"Context Quality": 0.7,
"Needs Web Search": false,
"Explanation": "The provided context from the `retrievre_tool` gives a good overview of what Fibromyalgia is, including its definition, epidemiology, symptoms, and treatment options. However, the context could be more comprehensive and detailed, especially regarding the latest research and management strategies.",
"Answer": "Fibromyalgia is a chronic, diffuse musculoskeletal pain syndrome characterized by tenderness, sleep disturbances, fatigue, and affective dysfunction, with a number of associated symptoms and comorbidities. It affects 2-10% of the general population and is often treated with low-dose tricyclic antidepressants, dual serotonin-norepinephrine reuptake inhibitors, selective serotonin reuptake inhibitors, and antiepileptics, along with regular monitoring and follow-up."
}
涉及的令牌数量
response.cost()
成本(request_tokens=1655, response_tokens=229, total_tokens=1884, details=None)
问题 2
query = "What is Fibromyalgia and what are its causes?"
response = groq_agent.run_sync(query)
print(response)
RAG 检索
['Fibromyalgia \\n \\nIntroduction \\nFibromyalgia syndrome (FM) is defined as a common rheumatological syndrome characterized by chronic, diffuse \\nmusculoskeletal pain and tenderness with a number of associated symptoms, among which sleep disturbances, \\nfatigue, and affective dysfunction are particularly frequent. \\n \\nEpidemiology and Economics \\n• Affects 2–10% of the general population, in all ages, ethnic groups, and cultures.',
'i. low-dose tricyclic antidepressants (amitriptyline) \\nii. dual serotonin-norepinephrine reuptake inhibitors \\niii. selective serotonin reuptake inhibitors \\niv. antiepileptics (gabapentin, pregabalin) \\no regular monitoring and follow-up \\n \\nReferences \\n1. Goldenberg DL, Burckhardt C, Crofford L. Management of fibromyalgia syndrome. JAMA 2004;292:2388–95. \\n2. Russell IJ, Bieber CS. Myofascial pain and fibromyalgia syndrome. In: McMahon SB, Koltzenburg M, editors. Melzack and',
'• The pain is described as a persistent, diffuse, deep, aching, throbbing, sensation in muscles and is most \\noften continuous \\n• Clinical symptoms associated with FM are affective dysfunction, cognitive deficits, short-term memory \\nloss, headache, nonrestorative sleep, and daytime tiredness resembling physical fatigue \\n• A number of clinical conditions occur more frequently in FM than in the general population \\n(comorbidities): \\no depression \\no anxiety \\no irritable bowel syndrome (IBS)']
运行结果
RunResult(_all_messages=[
SystemPrompt(content='You are a Helpful Assistant Proficient in Answering concise, factual and to the point answers for questions asked based on the Context provided. You have to Use the `retrievre_tool` to get relevant context and generate response based on the context retrieved. You are a grading assistant. Evaluate the response based on:\\n 1. Relevancy to the question\\n 2. Faithfulness to the context\\n 3. Context quality and completeness\\n \\n Please grade the following response based on:\\n 1. Relevancy (0-1): How well does it answer the question?\\n 2. Faithfulness (0-1): How well does it stick to the provided context?\\n 3. Context Quality (0-1): How complete and relevant is the provided context?\\n \\n Question: {ctx.deps.query}\\n Context: {ctx.deps.context}\\n Response: {ctx.deps.response}\\n \\n Also determine if web search is needed to augment the context.\\n \\n Provide the grades and explanation in the JSON format with key attributes 'Relevancy', 'Faithfulness', 'Context Quality', 'Needs Web Search':\\n {"Relevancy": <score>,\\n "Faithfulness": <score>,\\n "Context Quality": <score>,\\n "Needs Web Search": <true/false>,\\n "Explanation": <explanation>,\\n "Answer":<provide response based on the context from the `retrievre_tool` if 'Need Web Search' value is 'false' otherwise Use the `websearch_tool` function to generate the final response}',
role='system'),
UserPrompt(content="What is Fibromyalgia and what are its causes?", timestamp=datetime.datetime(2024, 12, 14, 15, 19, 48, 480034, tzinfo=datetime.timezone.utc), role='user'),
ModelStructuredResponse(calls=[ToolCall(tool_name='rertiever_tool', args=ArgsJson(args_json='{"question": "Fibromyalgia causes"}'), tool_id='call_ej82')], timestamp=datetime.datetime(2024, 12, 14, 15, 19, 48, tzinfo=datetime.timezone.utc), role='model-structured-response'),
ToolReturn(tool_name='rertiever_tool', content=['Fibromyalgia \\n \\nIntroduction \\nFibromyalgia syndrome (FM) is defined as a common rheumatological syndrome characterized by chronic, diffuse \\nmusculoskeletal pain and tenderness with a number of associated symptoms, among which sleep disturbances, \\nfatigue, and affective dysfunction are particularly frequent. \\n \\nEpidemiology and Economics \\n• Affects 2–10% of the general population, in all ages, ethnic groups, and cultures.', 'i. low-dose tricyclic antidepressants (amitriptyline) \\nii. dual serotonin-norepinephrine reuptake inhibitors \\niii. selective serotonin reuptake inhibitors \\niv. antiepileptics (gabapentin, pregabalin) \\no regular monitoring and follow-up \\n \\nReferences \\n1. Goldenberg DL, Burckhardt C, Crofford L. Management of fibromyalgia syndrome. JAMA 2004;292:2388–95. \\n2. Russell IJ, Bieber CS. Myofascial pain and fibromyalgia syndrome. In: McMahon SB, Koltzenburg M, editors. Melzack and', '• The pain is described as a persistent, diffuse, deep, aching, throbbing, sensation in muscles and is most \\noften continuous \\n• Clinical symptoms associated with FM are affective dysfunction, cognitive deficits, short-term memory \\nloss, headache, nonrestorative sleep, and daytime tiredness resembling physical fatigue \\n• A number of clinical conditions occur more frequently in FM than in the general population \\n(comorbidities): \\no depression \\no anxiety \\no irritable bowel syndrome (IBS)'], tool_id='call_ej82', timestamp=datetime.datetime(2024, 12, 14, 15, 19, 48, 901525, tzinfo=datetime.timezone.utc), role='tool-return'),
ModelTextResponse(content='{"Relevancy": 0.8, \\n"Faithfulness": 0.9, \\n"Context Quality": 0.7, \\n"Needs Web Search": false, \\n"Explanation": "The provided context from the retriever tool provides a good overview of Fibromyalgia, its symptoms, and associated conditions. However, it does not explicitly state the causes of Fibromyalgia. The context mentions that Fibromyalgia is a common rheumatological syndrome characterized by chronic, diffuse musculoskeletal pain and tenderness, but it does not provide a clear explanation of what triggers this condition.", \\n"Answer": "Fibromyalgia is a common rheumatological syndrome characterized by chronic, diffuse musculoskeletal pain and tenderness, often accompanied by sleep disturbances, fatigue, and affective dysfunction. The exact causes of Fibromyalgia are not explicitly stated in the provided context, but it is often associated with a number of clinical conditions, including depression, anxiety, and irritable bowel syndrome."}', timestamp=datetime.datetime(2024, 12, 14, 15, 19, 49, tzinfo=datetime.timezone.utc), role='model-text-response')
], _new_message_index=1, data='{"Relevancy": 0.8, \\n"Faithfulness": 0.9, \\n"Context Quality": 0.7, \\n"Needs Web Search": false, \\
### 纤维肌痛综合症概述
纤维肌痛综合症是一种慢性疾病，其特征是广泛的肌肉骨骼疼痛、疲劳、睡眠障碍和认知功能障碍。纤维肌痛综合症的确切原因尚不完全清楚，但被认为涉及遗传、环境和激素因素的组合。它通常与其他疾病相关，如抑郁症、焦虑症和肠易激综合症。
#### 成本信息
Cost(request_tokens=1669, response_tokens=232, total_tokens=1901, details=None)
#### 问题 3
```python
query = "What is the life expectancy of people suffering with fibromyalgia?"
response = groq_agent.run_sync(query)
print(response.data)
parser = JsonOutputParser()
print(parser.parse(response.data))
print(parser.parse(response.data)['Answer'])
print(response.cost())
纤维肌痛症
介绍
纤维肌痛症综合症（FM）被定义为一种常见的风湿病综合症，其特征是慢性、弥漫性的肌肉骨骼疼痛和压痛，并伴随多种相关症状，其中睡眠障碍、疲劳和情感功能障碍尤为常见。
流行病学和经济学
- 影响2-10%的普通人群，涵盖所有年龄、种族和文化。
- 疼痛被描述为一种持续的、弥漫的、深沉的、疼痛的、搏动的感觉，通常是连续性的。
- 与FM相关的临床症状包括情感功能障碍、认知缺陷、短期记忆丧失、头痛、非恢复性睡眠和类似于身体疲劳的白天疲惫。
- 一些临床情况在FM患者中发生的频率高于普通人群（共病）：
- 抑郁症
- 焦虑症
- 肠易激综合症（IBS）
治疗选择
i. 低剂量三环类抗抑郁药（阿米替林）
ii. 双重血清素-去甲肾上腺素再摄取抑制剂
iii. 选择性血清素再摄取抑制剂
iv. 抗癫痫药（加巴喷丁、普瑞巴林）
- 定期监测和随访
参考文献
- Goldenberg DL, Burckhardt C, Crofford L. 纤维肌痛症综合症的管理. JAMA 2004;292:2388–95.
- Russell IJ, Bieber CS. 骨骼肌痛和纤维肌痛症综合症. In: McMahon SB, Koltzenburg M, editors. Melzack and
纤维肌痛症患者的预期寿命
纤维肌痛症患者的预期寿命与普通人群并没有显著差异。根据一些研究，纤维肌痛症患者由于自杀、心血管疾病和其他合并症的风险增加，可能会有稍低的预期寿命。然而，通过适当的治疗和管理，许多纤维肌痛症患者可以过上积极和充实的生活。需要注意的是，纤维肌痛症是一种慢性病，其对预期寿命的影响可能因个人情况而异。
评估场景
在上述场景中，我们可以看到代理评估了响应，并发现答案与问题不相关，相关性得分为 0.2，并评估为了丰富答案需要进行网络搜索。我们可以看到代理调用了 websearch_tool 以获取关于所提问题的见解并生成了响应。
结论
这个 RAG 系统展示了如何构建一个复杂的人工智能应用程序，该应用程序：
- 结合了本地和基于网络的知识
- 实现自我评估
- 维持高响应质量
- 使用类型安全的组件
Pydantic.ai、Groq 和 LangChain 的结合为构建生产就绪的人工智能应用程序创造了坚实的基础。
未来改进
-
增强评分：
- 多维评分
- 来源可信度评估
- 信心评分
-
性能优化：
- 缓存机制
- 并行处理
- 批量操作
-
额外功能：
- 多模态支持
- 对话记忆
- 自定义知识注入
参考文献
学术论文和文档
- RAG系统: 检索增强生成 (RAG) 用于知识密集型 NLP 任务
- 语言模型和嵌入: BGE: BAAI 通用嵌入
- LangChain 文档
