min-hol-repo/langchain_voyage_mdb

# 实战实验室 - MongoDB 故障排查 RAG Chatbot 一个基于 **Hybrid Search + RRF** 的 RAG (Retrieval-Augmented Generation) pipeline， 使用 LangChain + Voyage AI + MongoDB Atlas + OpenAI 构建。 ## 架构 ``` User Question │ ├─── [Vector Search] voyage-4 → FAISS (local, free) → Ranked List A │ ├─── [Full-Text Search] $search (Atlas Search, works on M0) → Ranked List B │ └─── [RRF Fusion] 1/(k+rank_A) + 1/(k+rank_B) → Final Ranking │ └─── Top Documents → GPT-4o-mini → Final Answer ``` ## 技术栈 | 角色 | 技术 | |------|------------| | Embedding | [Voyage AI](https://www.voyageai.com/) `voyage-4` (1024 维) | | Vector Search | **[FAISS](https://github.com/facebookresearch/faiss)** (本地，免费 — 无需 M10+) | | Full-Text Search | MongoDB Atlas Search (`$search`，支持 **Free Tier M0**) | | 文档存储 | [MongoDB Atlas](https://www.mongodb.com/atlas) (Free Tier M0) | | 搜索融合 | RRF (Reciprocal Rank Fusion) | | 答案生成 | [OpenAI](https://platform.openai.com/) `gpt-4o-mini` | | RAG Pipeline | [LangChain](https://www.langchain.com/) LCEL | ## 什么是 RRF (Reciprocal Rank Fusion)？ 一种使用基于排名的评分来合并多个检索系统结果的算法。 ``` \text{RRF\_score}(d) = \sum_{i \in \text{systems}} \frac{1}{k + \text{rank}_i(d)} ``` - **Vector Search** (语义)：查找具有相似含义的文档 - **Full-Text Search** (关键词)：查找包含确切关键词的文档 - **RRF 融合**：按排名合并两者的结果，以提高整体检索质量 ## 文件结构 ``` ├── mongodb_rag.py # Main script (function-based modular structure) ├── mongodb_rag.ipynb # Jupyter Notebook (step-by-step tutorial) ├── requirements.txt # Dependencies ├── .env.example # Environment variable template └── .gitignore ``` ## 快速开始 ### 1. 克隆仓库 ``` git clone https://github.com/min-hol-repo/langchain_voyage_mdb.git cd langchain_voyage_mdb ``` ### 2. 安装包 ``` pip install -r requirements.txt ``` ### 3. 设置环境变量 ``` cp .env.example .env ``` 打开 `.env` 文件并填写以下三个 key： ``` # MongoDB Atlas 连接字符串 # 在以下位置查找：Atlas UI → Database → Connect → Drivers MONGODB_URI=mongodb+srv://:@.mongodb.net/ # OpenAI API key # https://platform.openai.com/api-keys OPENAI_API_KEY=sk-... # Voyage AI API key # https://dash.voyageai.com/api-keys VOYAGE_API_KEY=pa-... ``` ### 4. 准备 MongoDB Atlas Cluster - 推荐使用 **M10 或更高**的 cluster（完全支持 Vector Search + Atlas Search） - M0 Free Tier 的 Atlas Search 功能受限 ### 5. 运行 ``` # 运行 Python 脚本（索引会自动创建） python mongodb_rag.py ```  或者使用 Jupyter Notebook 逐步运行： ``` jupyter notebook mongodb_rag.ipynb ``` ## 核心功能 ### 自动创建索引 无需在 Atlas UI 中手动创建索引 — 它们会在运行时自动创建。 ``` from pymongo.operations import SearchIndexModel # Vector Search 索引（用于语义搜索） SearchIndexModel( definition={"fields": [{"type": "vector", "path": "embedding", "numDimensions": 1024, "similarity": "cosine"}]}, name="vector_index", type="vectorSearch", ) # Atlas Search 索引（用于关键词搜索） SearchIndexModel( definition={"mappings": {"dynamic": False, "fields": {"text": {"type": "string"}}}}, name="search_index", type="search", ) ``` ### Hybrid Search ``` results = hybrid_search( collection=collection, query="MongoDB connection pool exhaustion issue", embeddings=embeddings, k=10, # Number of results per search rrf_k=60, # RRF constant (higher = less rank-difference effect) ) ``` ### RRF 输出示例 ``` RRF (Reciprocal Rank Fusion) Search Results =========================================================================== Rank Document Title VecRank TxtRank RRF Score Category --------------------------------------------------------------------------- 1 Connection Pool Exhaustion 1 1 0.032787 connection 2 Slow Queries - Missing Index 3 2 0.031185 performance 3 Replication Lag 2 - 0.016129 replication =========================================================================== ``` ### 知识库 包含 10 个 MongoDB 故障排查场景： | 类别 | 内容 | |----------|---------| | `connection` | 连接池耗尽 | | `performance` | 缺少索引导致的慢查询 | | `replication` | 复制延迟，Primary 选举失败 | | `memory` | WiredTiger Cache 不足，OOM Killer | | `storage` | 磁盘空间不足 | | `locking` | 锁争用 | | `search` | Atlas Vector Search 索引错误 | | `backup` | Mongodump / Mongorestore | ## 尝试您自己的问题 ``` from mongodb_rag import ask_mongodb_question answer = ask_mongodb_question( question="MongoDB server crashed unexpectedly. What are the causes and solutions?", collection=collection, embeddings=embeddings, llm=llm, ) ``` ## 参考 - [MongoDB Atlas Vector Search 文档](https://www.mongodb.com/docs/atlas/atlas-vector-search/) - [MongoDB Atlas Search 文档](https://www.mongodb.com/docs/atlas/atlas-search/) - [Voyage AI 模型文档](https://docs.voyageai.com/docs/embeddings) - [LangChain MongoDB 集成](https://python.langchain.com/docs/integrations/vectorstores/mongodb_atlas/) - [RRF 论文 (Cormack et al., 2009)](https://plg.uwaterloo.ca/~gvcormac/cormacketal09-rrf.pdf) ## 许可证 MIT License

标签：AI, LangChain, MongoDB, NoSQL, RAG, 向量检索, 混合检索, 自动化代码审查, 轻量级, 运维辅助, 逆向工具