fix: 修复 CORS allow_credentials 与 allow_origins=* 冲突

This commit is contained in:
2026-07-10 15:18:39 +08:00
parent 3b8b585f31
commit 284a4b9e09
6 changed files with 3663 additions and 5 deletions
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,142 @@
# 多格式文档支持 — 设计文档
**日期**: 2026-07-10
**版本**: 1.0
## 目标
将 md-vector-db 从"仅 Markdown"扩展为支持 `.txt``.pdf``.html` 的多格式文档向量数据库。
## 非目标
- 不支持 `.docx``.pptx``.epub` 等 Office/电子书格式(留待后续扩展)
- 不改变现有的嵌入和检索流程
- 不改变 HTTP API 的请求/响应模型
## 架构
### 模块结构
```
src/core/
├── splitters/ # 新建目录
│ ├── __init__.py # 导出 registry + 所有 Splitter
│ ├── base.py # Splitter Protocol + BaseTextSplitter (ABC)
│ ├── markdown.py # MarkdownSplitter (从 ingest.py 移入)
│ ├── text.py # TextSplitter (纯文本按段落+标点硬切)
│ ├── pdf.py # PDFSplitter (pymupdf → TextSplitter)
│ ├── html.py # HTMLSplitter (bs4 → TextSplitter)
│ └── registry.py # 工厂 + 扩展名→Splitter 映射表
├── ingest.py # 精简,用 registry.get_splitter() 自动选择
```
### 类继承
```
Splitter (Protocol)
└── BaseTextSplitter (ABC) # max_size, overlap, _split_single_paragraph
├── MarkdownSplitter # 已有,从 ingest.py 移入
├── TextSplitter # 纯文本:按 \n\n 切段落,超长按标点硬切
├── PDFSplitter # 读取 PDF → extract_text() → 委托 TextSplitter
└── HTMLSplitter # 读取 HTML → get_text() → 委托 TextSplitter
```
`PDFSplitter``HTMLSplitter` **不继承** `BaseTextSplitter`,而是组合一个 `TextSplitter` 实例。它们实现 `Splitter` Protocol,在 `split()` 中:提取纯文本 → 委托 `TextSplitter.split()`
### 注册表
`registry.py` 维护默认扩展名映射:
```python
_DEFAULT_MAP = {
".md": "markdown",
".markdown": "markdown",
".txt": "text",
".pdf": "pdf",
".html": "html",
".htm": "html",
}
def get_splitter(file_path: str, **config) -> Splitter:
"""根据扩展名自动选择 Splitter,未匹配回退到 TextSplitter."""
```
## 依赖策略
- `pymupdf``beautifulsoup4` 作为**可选依赖**
- 首次使用 PDF/HTML 格式时才 `import`,库缺失时抛 `ImportError` 带安装提示
- 安装方式:
```bash
uv sync --extra pdf # PDF 支持
uv sync --extra html # HTML 支持
uv sync --extra all # 全部可选依赖
```
## ingest.py 改动
### DocumentIngestor.__init__
已有 `splitter` 可选参数,**保持不变**。显式传入的 splitter 覆盖自动选择。
### ingest_file()
改为使用 `get_splitter(file_path, ...)` 自动选择 splitter
```python
def ingest_file(self, file_path: str) -> int:
splitter = self.splitter or get_splitter(file_path,
max_size=..., overlap=...)
...
```
### ingest_directory()
`rglob("*.md")` 改为遍历所有支持格式:
```python
_SUPPORTED_SUFFIXES = {".md", ".markdown", ".txt", ".pdf", ".html", ".htm"}
def ingest_directory(self, dir_path: str) -> dict[str, int]:
for f in Path(dir_path).rglob("*"):
if f.suffix.lower() in _SUPPORTED_SUFFIXES:
...
```
## 数据流
```
文件路径 → get_splitter(path)
├── .md → MarkdownSplitter.split(text)
├── .txt → TextSplitter.split(text)
├── .pdf → PDFSplitter.split(binary)
│ └── pymupdf 提取文字 → TextSplitter.split(text)
└── .html → HTMLSplitter.split(html)
└── bs4 去标签 → TextSplitter.split(text)
batch_embed(chunks)
ChromaDB.add()
```
## 测试策略
- `tests/test_splitters_text.py` — TextSplitter 段落切分、硬切、overlap
- `tests/test_splitters_registry.py` — 扩展名映射、回退逻辑、自定义注册
- `tests/test_splitters_pdf.py` — PDF 提取文字 + 分块(需 pymupdf)
- `tests/test_splitters_html.py` — HTML 去标签 + 分块(需 bs4
- `tests/test_ingest.py` — 补 `ingest_directory` 多格式遍历测试
## CLI/API 影响
- **CLI**:零改动。`ingest``ingest-dir` 自动获得多格式能力
- **API**:零改动。`POST /api/v1/ingest``file_path` 自动支持
- **`ingest_content()`**:仍默认使用 `MarkdownSplitter`(处理 Markdown 字符串),显式传入 content 时不变
## 风险
| 风险 | 缓解 |
| -------------------------------------------------------- | --------------------------------------------------------------------------------------------- |
| PDF 提取失败(扫描件/图片 PDF) | 抛明确异常,跳过该文件 |
| HTML 标签复杂导致去标签不干净 | 用`soup.get_text(separator="\n")` 保留段落结构 |
| `ingest.py` 迁移 MarkdownSplitter 后旧 import 路径失效 | 在`ingest.py` 保留兼容 import`from src.core.splitters.markdown import MarkdownSplitter` |
@@ -0,0 +1,369 @@
# md-vector-db 全面安全与质量审计报告
> **审计日期**: 2026-07-10
> **审计范围**: 全部源代码(22 个源文件)、13 个测试文件、配置、文档
> **测试基线**: 90 个测试全部通过
> **项目版本**: commit `3b8b585`
---
## 一、审计概览
本次审计由三个并行代理分别审查 **核心模块**config/db/embedder/ingest/search/security/splitters)、**服务层与 CLI**app/auth/deps/main.py/scripts)、**测试与配置**tests/pyproject.toml/config.yaml/文档),最终整合为本报告。
### 问题统计
| 严重级别 | 数量 | 含义 |
|----------|------|------|
| **CRITICAL** | **1** | 阻断 — 数据重复处理浪费 GPU 资源 |
| **HIGH** | **13** | 警告 — 合并前应修复 |
| **MEDIUM** | **11** | 信息 — 应考虑修复 |
| **LOW** | **10** | 注意 — 可选修复 |
| **总计** | **35** | |
### 覆盖评估
| 模块 | 覆盖程度 |
|------|----------|
| `core/security.py` | ✅ 优秀 — 所有路径穿越场景已覆盖 |
| `core/config.py` | ✅ 良好 — YAML 加载/默认值/环境变量 |
| `core/embedder.py` | ✅ 良好 — 所有 provider 初始化和 batch_embed |
| `core/splitters/*` (除 markdown) | ✅ 良好 |
| `core/db.py` | ⚠️ 基本 — 缺少 list_collections、write_guard、并发测试 |
| `core/ingest.py` | ❌ 不足 — 缺少 ingest_file、ingest_directory |
| `core/search.py` | ❌ 不足 — 缺少 3/4 方法测试 |
| `server/deps.py` / `app.py` | ⚠️ 基本 — 缺少认证/速率限制/CORS 配置测试 |
| `server/auth.py` | ❌ 零覆盖 |
| `cli/main.py` | ❌ 零覆盖 |
| **整体估算** | **~55-60%**(低于 80% 目标) |
---
## 二、CRITICAL 级别(1 个)
### [CRIT-1] `ingest_obsidian.py` 顶层 .md 文件被重复入库
- **文件**: `scripts/ingest_obsidian.py`,第 54-77 行
- **根因**: 第一轮用 `rglob("*.md")` 递归扫描所有文件,第二轮用 `glob("*.md")` 再次收集顶层文件。由于 `rglob` 已经包含顶层文件,每个顶层 `.md` 被处理两次。
- **影响**: 嵌入计算浪费(GPU 资源),入库时间翻倍于顶层文件数量。第二次入库虽然会覆盖第一次(file_name 相同,先删后加),但嵌入已完成,白白消耗了 GPU。
- **修复**: 删除第二轮循环(第 67-77 行),`rglob` 已覆盖所有文件。
---
## 三、HIGH 级别(13 个)
### 安全
#### [HIGH-1] CORS `allow_credentials=True` 与 `allow_origins="*"` 冲突
- **文件**: `src/server/app.py`,第 66-72 行
- **根因**: CORS 规范禁止 `Access-Control-Allow-Origin: *``Access-Control-Allow-Credentials: true` 同时使用。浏览器会拒绝此配置。当前默认 `CORS_ORIGINS="*"``allow_credentials=True` 直接违反规范。
- **此外**: `*` 允许任意来源跨域访问,安全上过于宽松。
- **修复**:
```python
app.add_middleware(
CORSMiddleware,
allow_origins=os.environ.get("CORS_ORIGINS", "http://localhost:3000").split(","),
allow_credentials=False, # 默认关闭
...
)
```
#### [HIGH-2] `search_documents` 端点缺少异常处理
- **文件**: `src/server/app.py`,第 147-155 行
- **根因**: `ingest_document` 有完整的 `try/except` 包裹,但 `search_documents` 完全没有。如果嵌入模型加载失败或 ChromaDB 查询异常,会返回 FastAPI 默认的原始 traceback,泄露内部路径和调用栈。
- **修复**: 添加异常捕获:
```python
try:
searcher = state.get_searcher(req.collection)
results = searcher.search(req.query, top_k=req.top_k)
return {"results": results, "collection": searcher.collection_name}
except Exception:
logger.exception("检索失败")
raise HTTPException(status_code=500, detail="服务器内部错误")
```
#### [HIGH-3] `delete_document` 的 `collection` 查询参数缺少输入校验
- **文件**: `src/server/app.py`,第 163 行
- **根因**: `IngestRequest.collection` 和 `SearchRequest.collection` 都使用了 `pattern=r"^[a-zA-Z0-9_-]+$"` 校验,但 `delete_document` 的 `collection` 查询参数没有任何正则校验。ChromaDB collection 名称允许的字符集有限,传入非法字符可能导致未预期的 ChromaDB 内部错误。
- **修复**: 添加 `Query(default=None, max_length=128, pattern=r"^[a-zA-Z0-9_-]+$")`。
#### [HIGH-4] CLI 与 API 的路径安全检查逻辑不一致
- **文件**: `src/cli/main.py` 第 85 行 vs `src/server/app.py` 第 125-130 行
- **根因**: CLI 仅用 `is_safe_path()` 做基础检查(拒绝 `..` + 绝对路径),而 API 还额外做了 `os.path.commonpath` 绑定当前工作目录的检查。两处策略不同增加了安全漏洞风险。
- **修复**: 提取统一的 `is_path_within_workspace()` 函数到 `security.py`,两处共用:
```python
def is_path_within_workspace(path_str: str) -> bool:
"""检查路径是否在当前工作目录内(防路径穿越)。"""
if not is_safe_path(path_str):
return False
path = Path(path_str).resolve()
cwd = Path.cwd().resolve()
try:
return Path(os.path.commonpath([str(path), str(cwd)])) == cwd
except ValueError:
return False
```
### 功能
#### [HIGH-5] chunk 配置项被硬编码覆盖 — YAML 中的 chunk 设置完全无效
- **文件**: `src/core/ingest.py`,第 46 行
- **根因**: `get_splitter(file_path, max_size=1000, overlap=100)` 写死了值。用户无论怎么修改 `config.yaml` 里的 `chunk.max_size` 和 `chunk.overlap`,都完全不会生效。
- **影响**: 所有通过 API/CLI 正常流程的入库都使用硬编码值。`AppConfig.chunk` 数据类从未在正常入库链路中被读取。
- **修复**: `DocumentIngestor` 构造函数接受 `ChunkConfig` 参数,`deps.py` 的 `AppState.get_ingestor` 传入 `self.config.chunk`。
#### [HIGH-6] CLI `serve` 命令双重加载嵌入模型
- **文件**: `src/cli/main.py` 第 174 行 + `src/server/deps.py`
- **根因**: `serve` 命令调用 `get_state().config`,触发 `AppState.__init__` → `create_embedder()` 加载模型(第一次)。uvicorn 子进程启动后,`src.server.app:app` 生命周期内再次调用 `get_state()` → 加载模型(第二次)。对于 `serve` 命令,只需要服务配置(host, port, SSL),不需要加载 ML 模型。
- **影响**: 模型被加载两次,浪费几 GB 内存和数秒启动时间。
- **修复**: 将配置加载与模型加载解耦——提供一个不初始化模型的轻量配置读取函数给 CLI `serve` 命令使用。
### 代码质量
#### [HIGH-7] PDF 文件句柄泄漏 — fitz.Document 在异常路径下未关闭
- **文件**: `src/core/splitters/pdf.py`,第 34-43 行
- **根因**: `doc.close()` 在 `try` 块内,如果在 `for page in doc` 或 `page.get_text()` 期间抛出异常,`close()` 永远不会被调用。
- **修复**: 使用 `with fitz.open(pdf_path) as doc:`(pymupdf 支持上下文管理器协议)。
#### [HIGH-8] embedder.py 中环境变量操作不是线程安全的
- **文件**: `src/core/embedder.py`,第 80-90 行
- **根因**: `os.environ["HF_ENDPOINT"]` 的读写没有锁保护。如果多个线程同时创建 `LocalEmbedder` 实例,可能产生竞态——线程 A 设置的值被 B 覆盖,A 的 `finally` 恢复时读到 B 的值。
- **修复**: 使用线程锁保护该区域,或通过 `huggingface_hub` 的 API 在进程启动时统一设置而非按实例设置。
#### [HIGH-9] `ingest_obsidian.py` 初始化阶段无异常处理
- **文件**: `scripts/ingest_obsidian.py`,第 32-35 行
- **根因**: `load_config()` / `VectorDB()` / `create_embedder()` / `DocumentIngestor()` 任一失败都会直接崩溃,没有友好的错误信息或恢复路径。
- **修复**: 添加顶层 `try/except` 包裹,捕获异常后打印 traceback 并以 `sys.exit(1)` 退出。
### 测试缺口
#### [HIGH-10] CLI 模块零测试覆盖
- **文件**: `src/cli/main.py`216 行,5 个命令)
- **影响**: 通配符展开、stdin 输入、JSON 格式化输出、路径安全检查等逻辑无自动化验证。
- **修复**: 创建 `tests/test_cli.py`,使用 `typer.testing.CliRunner` 覆盖每个命令。
#### [HIGH-11] 认证与速率限制零单元测试
- **文件**: `src/server/auth.py`58 行)
- **影响**: `verify_api_key()` 和 `RateLimiter` 两个安全关键组件完全没有独立测试。所有 API 测试因未设置 `MD_VECTOR_API_KEY` 而跳过认证验证。
- **修复**: 创建 `tests/test_auth.py`,覆盖密钥匹配/不匹配/未提供、速率限制正常/超限/过期恢复、并发线程安全。
#### [HIGH-12] `ingest_file` / `ingest_directory` 零测试
- **文件**: `src/core/ingest.py`121 行)
- **影响**: 包含扩展名检测、路径哈希、PDF/EPUB 二进制特殊处理、目录递归扫描和过滤逻辑,全部未测试。
- **修复**: 在 `tests/test_ingest.py` 中补充文件入库和多格式目录入库的集成测试。
#### [HIGH-13] `Searcher` 三个方法零测试
- **文件**: `src/core/search.py`97 行)
- **影响**: `list_sources()`、`delete_by_source()`、`get_collection_info()` 以及 `search(source_file=...)` 过滤均无测试。
- **修复**: 在 `tests/test_search.py` 中补充。
---
## 四、MEDIUM 级别(11 个)
### [MED-1] `content` 模式默认 file_name 为 `"untitled.md"` 导致数据竞态覆盖
- **文件**: `src/server/app.py`,第 137 行
- **根因**: 多个并发请求不提供 `file_name` 时都写入 `"untitled.md"`。`ingest_content` 先 `_remove_by_source(file_name)` 再 `_add_chunks()`,且两次操作之间锁释放,存在竞态——后完成的请求覆盖先完成的请求数据。
- **修复**: 自动生成 UUID 唯一名称:`file_name = req.file_name or f"untitled_{uuid.uuid4().hex[:8]}.md"`
### [MED-2] `X-XSS-Protection` 头已过时且可能引入安全问题
- **文件**: `src/server/app.py`,第 81 行
- **根因**: 此响应头已被所有现代浏览器废弃。旧版 IE 的 XSS Auditor 自身存在安全漏洞。OWASP 等安全组织明确建议不要设置此头。
- **修复**: 替换为 `Content-Security-Policy: default-src 'self'`
### [MED-3] `EXPECTED_API_KEY` 在模块导入时求值,存在加载顺序隐患
- **文件**: `src/server/auth.py`,第 14 行
- **根因**: 该常量在模块被 import 时求值。虽然目前 `config.py` 先调用 `load_dotenv()` 再触发导入链,但这是一个隐式依赖——如果有人调整导入顺序或在测试中直接 import `auth.py``EXPECTED_API_KEY` 可能为空。
- **修复**: 改为惰性求值函数:`def _get_api_key() -> str: return os.environ.get("MD_VECTOR_API_KEY", "")`
### [MED-4] ingest.py 与 search.py 存在去重删除逻辑重复
- **文件**: `ingest.py:108-120` 和 `search.py:83-97`
- **根因**: 两处 `_remove_by_source` / `delete_by_source` 逻辑几乎一模一样:获取匹配文档、删除 IDs、捕获 `ValueError`。
- **修复**: 统一放到 `VectorDB.delete_by_metadata(where)` 方法中。
### [MED-5] `list_collections_with_stats` 静默吞掉所有异常
- **文件**: `src/server/deps.py`,第 54-59 行
- **根因**: 当 ChromaDB 不可用时,返回空列表而不是错误。调用方向用户报告"没有任何集合",而非"数据库连接失败"。错误语义被扭曲。
- **修复**: 让异常向上传播,由 API 层统一处理并返回适当的 HTTP 错误。
### [MED-6] EPUB 缺失依赖测试设计有误
- **文件**: `tests/test_splitters_epub.py`,第 51-56 行
- **根因**: `test_epub_missing_dependency_message` 声称测试"未安装 ebooklib 时应给出明确提示",但模块顶部有 `pytest.importorskip("ebooklib", ...)`,意味着当 ebooklib 未安装时整个文件被 skip,该测试永远不会在目标场景下执行。
- **修复**: 将此测试移到独立文件中(不使用 `importorskip`),或使用 `monkeypatch` 模拟 `ImportError`。
### [MED-7] 环境变量测试使用 `importlib.reload` 过于脆弱
- **文件**: `tests/test_config.py`,第 89-106 行
- **根因**: `importlib.reload(config)` 有全局副作用,可能影响后续测试。`finally` 块中的清理逻辑也不完整。
- **修复**: 使用 `monkeypatch.setenv` + 重新实例化代替 `reload`。
### [MED-8] 速率限制器的配置值硬编码
- **文件**: `src/server/auth.py`,第 58 行
- **根因**: `RateLimiter(max_requests=30, window_seconds=60)` 无法通过环境变量或配置文件调整。
- **修复**: 从环境变量读取:`max_requests=int(os.environ.get("RATE_LIMIT_MAX", "30"))`
### [MED-9] `ingest_obsidian.py` 存在硬编码的绝对回退路径
- **文件**: `scripts/ingest_obsidian.py`,第 47-51 行
- **根因**: `targets` 中的三个路径仅在本机有效,在其他开发者的机器上毫无意义。如果恰好存在同名目录,会意外入库其他数据。
- **修复**: 移除硬编码回退路径,直接报错提示用户提供参数。
### [MED-10] MarkdownSplitter 测试组织不清晰
- **文件**: 分散在 `tests/test_ingest.py` 中
- **根因**: Markdown 分块测试与入仓器测试混在一起,缺少 Markdown 特有的边界测试:无标题文档、深层标题、代码块内含 `#` 号被误识别为标题等。
- **修复**: 提取到 `tests/test_splitters_markdown.py`,补充边界测试。
### [MED-11] `splitters/__init__.py` 未导出 HTMLSplitter
- **文件**: `src/core/splitters/__init__.py`,第 10-20 行
- **根因**: `HTMLSplitter` 在第 7 行被 import 但未加入 `__all__` 列表,而 `PDFSplitter` 和 `EPUBSplitter` 都在列表中。
- **修复**: 在 `__all__` 中加入 `"HTMLSplitter"`。
---
## 五、LOW 级别(10 个)
| # | 文件 | 问题 | 建议 |
|---|------|------|------|
| L-1 | `src/core/splitters/epub.py:44` | EPUB 硬编码 UTF-8 解码,非 UTF-8 文件会丢失章节 | 添加编码检测回退(先 UTF-8,失败后 chardet |
| L-2 | `src/core/embedder.py:24` | 模块级 `import requests` 依赖传递性依赖项 | 改为惰性导入(在 DashscopeEmbedder.embed 内部) |
| L-3 | `src/core/embedder.py:176` | `list.sort()` 原地修改 API 响应 | 改为 `sorted(embeddings_raw, key=...)` |
| L-4 | `src/core/config.py:16` | `DEFAULT_CONFIG_PATH` 使用相对路径 | 统一使用 `load_config` 的路径解析逻辑 |
| L-5 | `src/core/splitters/registry.py:20` | `_custom_registry` 模块级字典无线程锁保护 | 添加 `threading.Lock` 或文档说明仅限启动时调用 |
| L-6 | `pyproject.toml:21` | `dev` 依赖缺少 `pytest-cov` 和 `ruff`/`mypy` | 添加 `pytest-cov>=5.0`、`ruff>=0.8.0`、`mypy>=1.13` 及相关 tool 配置节 |
| L-7 | `config.yaml` | `host: 0.0.0.0` 与代码默认值 `127.0.0.1` 不一致 | 统一默认值或在配置文件中添加注释说明差异 |
| L-8 | `config.yaml` | SSL 字段(ssl_keyfile/ssl_certfile)未在配置文件中暴露 | 添加注释说明可用配置 |
| L-9 | `scripts/serve.py` | `DeprecationWarning` 被 Python 默认过滤,多数用户看不到 | 改为 `print(..., file=sys.stderr)` 或 `FutureWarning` |
| L-10 | `CLAUDE.md` / `README.md` | 文档中测试数量写"46 个",实际 90 个;数据流未反映多格式支持 | 更新为实际数量,数据流加入 `get_splitter()` 描述 |
---
## 六、架构与设计评估
### 优点
1. **策略模式设计良好** — `Embedder(Protocol)` 和 `Splitter(Protocol)` 接口清晰,新增 provider/格式只需实现接口并注册
2. **扩展名自动路由** — `registry.py` 的 `get_splitter()` 工厂 + 回退到 TextSplitter 的设计优雅
3. **线程安全基础扎实** — `VectorDB._write_lock` + `deps.py` 的 `_state_lock`/`_cache_lock` 构成多层保护
4. **安全防御分层** — API Key 认证 → 速率限制 → 路径穿越防护,防御纵深合理
5. **配置热加载友好** — `load_config()` 支持文件变更检测,适合长期运行的服务
### 需要改进的架构问题
1. **配置与模型加载未解耦** — `AppState.__init__` 做了太多事(配置+模型+数据库),导致 CLI `serve` 双重加载。建议拆分为 `load_config_only()` 和 `init_full_state()`
2. **chunk 配置未贯传整个链路** — `config.yaml → AppConfig.chunk` 存在但被 `ingest.py` 硬编码覆盖。应该在 `DocumentIngestor` 构造函数中接受 `ChunkConfig`
3. **路径安全检查逻辑分散** — CLI 和 API 各有一套检查,应该提取为统一的 `is_path_within_workspace()`
4. **去重删除逻辑重复** — `ingest.py` 和 `search.py` 中几乎相同的代码应合并到 `VectorDB`
---
## 七、测试策略建议
### 当前测试分布
```
tests/
├── test_api.py # 14 个 — API 端点基本功能
├── test_config.py # 7 个 — 配置加载
├── test_db.py # 4 个 — 数据库 CRUD
├── test_deps.py # 11 个 — AppState 生命周期
├── test_embedder.py # 8 个 — 嵌入 provider
├── test_ingest.py # 11 个 — 入仓(偏 MarkdownSplitter
├── test_search.py # 8 个 — 基本搜索
├── test_security.py # 11 个 — 路径安全
├── test_splitters.py # 10 个 — TextSplitter + 注册表
├── test_splitters_html.py # 4 个
├── test_splitters_pdf.py # 3 个
├── test_splitters_epub.py # 3 个
└── (缺失)
├── test_auth.py ❌ 认证与速率限制
├── test_cli.py ❌ CLI 命令
└── test_splitters_markdown.py ❌ Markdown 边界
```
### 建议新增测试
1. **`tests/test_cli.py`** — CLI 5 个命令(使用 `CliRunner`
2. **`tests/test_auth.py`** — `verify_api_key` + `RateLimiter`
3. **`tests/test_splitters_markdown.py`** — 标题边界、代码块内 `#` 号、空文档
4. **补充 `tests/test_ingest.py`** — `ingest_file` / `ingest_directory`
5. **补充 `tests/test_search.py`** — `list_sources` / `delete_by_source` / `get_collection_info`
---
## 八、修复优先级路线图
### 第一阶段(立即修复 — 1 个 CRITICAL
| 问题 | 文件 | 工作量 |
|------|------|--------|
| CRIT-1: 顶层 .md 文件重复入库 | `scripts/ingest_obsidian.py` | 5 分钟 |
### 第二阶段(尽快修复 — 13 个 HIGH)
| 问题 | 领域 | 工作量 |
|------|------|--------|
| HIGH-5: chunk 配置被子覆盖 | 功能 | 30 分钟 |
| HIGH-7: PDF 文件句柄泄漏 | 资源 | 5 分钟 |
| HIGH-1: CORS 配置冲突 | 安全 | 5 分钟 |
| HIGH-2: search 端点无异常处理 | 安全 | 5 分钟 |
| HIGH-3: delete collection 缺少校验 | 安全 | 5 分钟 |
| HIGH-4: 路径安全检查不一致 | 安全 | 20 分钟 |
| HIGH-6: serve 双重加载模型 | 性能 | 30 分钟 |
| HIGH-8: HF_ENDPOINT 非线程安全 | 稳定性 | 15 分钟 |
| HIGH-9: ingest_obsidian 无异常处理 | 稳定性 | 10 分钟 |
| HIGH-10: CLI 零测试 | 测试 | 1-2 小时 |
| HIGH-11: auth 零测试 | 测试 | 1 小时 |
| HIGH-12: ingest_file/dir 零测试 | 测试 | 1 小时 |
| HIGH-13: Searcher 方法零测试 | 测试 | 30 分钟 |
### 第三阶段(计划修复 — 11 个 MEDIUM)
涉及竞态覆盖、过时安全头、重复代码、错误语义、测试设计等。
### 第四阶段(择机修复 — 10 个 LOW)
涉及文档过时、编码回退、线程锁、工具链配置等。
---
## 九、合规性检查
对照用户全局安全规则(`~/.claude/rules/ecc/zh/security.md`)进行逐项检查:
| 检查项 | 状态 | 说明 |
|--------|------|------|
| 无硬编码密钥 | ✅ 通过 | API Key 均从环境变量读取 |
| 所有用户输入已验证 | ⚠️ 部分 | `delete_document` 的 collection 参数缺少正则校验 |
| SQL 注入防护 | N/A | 无 SQL 数据库 |
| XSS 防护 | ⚠️ 部分 | `X-XSS-Protection` 已过时,应换为 CSP |
| CSRF 保护 | ⚠️ 部分 | CORS 配置存在规范冲突 |
| 认证/授权已验证 | ✅ 通过 | API Key 认证 + hmac.compare_digest |
| 所有端点启用速率限制 | ✅ 通过 | RateLimiter 中间件 |
| 错误消息不泄露敏感数据 | ⚠️ 部分 | `search_documents` 缺少异常处理会泄露 traceback |
---
*审计人: Claude Code(多代理协作审查)*
*下次审计建议: 在修复 CRITICAL 和 HIGH 问题后进行复验*
+4 -4
View File
@@ -65,10 +65,10 @@ app = FastAPI(title="md-vector-db", version="0.1.0", lifespan=lifespan)
# CORS 中间件
app.add_middleware(
CORSMiddleware,
allow_origins=os.environ.get("CORS_ORIGINS", "*").split(","),
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
allow_origins=os.environ.get("CORS_ORIGINS", "http://localhost:3000").split(","),
allow_credentials=False,
allow_methods=["GET", "POST", "DELETE", "OPTIONS"],
allow_headers=["Content-Type", "Authorization", "X-API-Key"],
)
Generated
+224 -1
View File
@@ -214,6 +214,19 @@ wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/27/44/d2ef5e87509158ad2187f4dd0852df80695bb1ee0cfe0a684727b01a69e0/bcrypt-5.0.0-cp39-abi3-win_arm64.whl", hash = "sha256:f2347d3534e76bf50bca5500989d6c1d05ed64b440408057a37673282c654927", size = 144953, upload-time = "2025-09-25T19:50:37.32Z" },
]
[[package]]
name = "beautifulsoup4"
version = "4.15.0"
source = { registry = "https://pypi.tuna.tsinghua.edu.cn/simple/" }
dependencies = [
{ name = "soupsieve" },
{ name = "typing-extensions" },
]
sdist = { url = "https://pypi.tuna.tsinghua.edu.cn/packages/43/65/318323f98dbee45d42dff61d8f047181bc6f2268a9068cfad035a46be5af/beautifulsoup4-4.15.0.tar.gz", hash = "sha256:288e3ca7d54b06f2ac191970bc275c1939cb46d450b255bf6718b04aa37ab4f7", size = 632571, upload-time = "2026-06-07T16:44:20.453Z" }
wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/88/c6/92fcd42f1ba33e1184263f25bfabf3d27c383410470f169e4b8163bf9c17/beautifulsoup4-4.15.0-py3-none-any.whl", hash = "sha256:d6f88de62e1d4e38ecb1077eb9724cd0eff29d2a08ca16a401e9b9e93f117cf9", size = 109924, upload-time = "2026-06-07T16:44:21.566Z" },
]
[[package]]
name = "build"
version = "1.5.0"
@@ -357,6 +370,15 @@ wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" },
]
[[package]]
name = "distro"
version = "1.9.0"
source = { registry = "https://pypi.tuna.tsinghua.edu.cn/simple/" }
sdist = { url = "https://pypi.tuna.tsinghua.edu.cn/packages/fc/f8/98eea607f65de6527f8a2e8885fc8015d3e6f5775df186e443e0964a11c3/distro-1.9.0.tar.gz", hash = "sha256:2fa77c6fd8940f116ee1d6b94a2f90b13b5ea8d019b98bc8bafdcabcdd9bdbed", size = 60722, upload-time = "2023-12-24T09:54:32.31Z" }
wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/12/b3/231ffd4ab1fc9d679809f356cebee130ac7daa00d6d6f3206dd4fd137e9e/distro-1.9.0-py3-none-any.whl", hash = "sha256:7bffd925d65168f85027d8da9af6bddab658135b840670a223589bc0c8ef02b2", size = 20277, upload-time = "2023-12-24T09:54:30.421Z" },
]
[[package]]
name = "durationpy"
version = "0.10"
@@ -366,6 +388,19 @@ wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/b0/0d/9feae160378a3553fa9a339b0e9c1a048e147a4127210e286ef18b730f03/durationpy-0.10-py3-none-any.whl", hash = "sha256:3b41e1b601234296b4fb368338fdcd3e13e0b4fb5b67345948f4f2bf9868b286", size = 3922, upload-time = "2025-05-17T13:52:36.463Z" },
]
[[package]]
name = "ebooklib"
version = "0.20"
source = { registry = "https://pypi.tuna.tsinghua.edu.cn/simple/" }
dependencies = [
{ name = "lxml" },
{ name = "six" },
]
sdist = { url = "https://pypi.tuna.tsinghua.edu.cn/packages/77/85/322e8882a582d4b707220d1929cfb74c125f2ba513991edbce40dbc462de/ebooklib-0.20.tar.gz", hash = "sha256:35e2f9d7d39907be8d39ae2deb261b19848945903ae3dbb6577b187ead69e985", size = 127066, upload-time = "2025-10-26T20:56:20.968Z" }
wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/bf/ee/aa015c5de8b0dc42a8e507eae8c2de5d1c0e068c896858fec6d502402ed6/ebooklib-0.20-py3-none-any.whl", hash = "sha256:fff5322517a37e31c972d27be7d982cc3928c16b3dcc5fd7e8f7c0f5d7bcf42b", size = 40995, upload-time = "2025-10-26T20:56:19.104Z" },
]
[[package]]
name = "fastapi"
version = "0.139.0"
@@ -681,6 +716,56 @@ wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/62/a1/3d680cbfd5f4b8f15abc1d571870c5fc3e594bb582bc3b64ea099db13e56/jinja2-3.1.6-py3-none-any.whl", hash = "sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67", size = 134899, upload-time = "2025-03-05T20:05:00.369Z" },
]
[[package]]
name = "jiter"
version = "0.16.0"
source = { registry = "https://pypi.tuna.tsinghua.edu.cn/simple/" }
sdist = { url = "https://pypi.tuna.tsinghua.edu.cn/packages/1d/1f/10936e16d8860c70698a1aa939a46aa0224813b782bce4e000e637da0b2d/jiter-0.16.0.tar.gz", hash = "sha256:7b24c3492c5f4f84a37946ad9cf504910cf6a782d6a4e0689b6673c5894b4a1c", size = 176431, upload-time = "2026-06-29T13:05:13.657Z" }
wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/91/c0/555fc60473d30d66894ba825e63615e3be7524fac23858356afa7a38906c/jiter-0.16.0-cp313-cp313-macosx_10_12_x86_64.whl", hash = "sha256:41977aa5654023948c2dae2a81cbf9c43343954bef1cd59a154dd15a4d84c195", size = 306203, upload-time = "2026-06-29T13:03:36.243Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/d0/2b/c3eaf16f5d7c9bad66ea32f40a95bd169b29a91217fcc7f081375157e99c/jiter-0.16.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:d28bb3c26762358dadf3e5bf0bccd29ae987d65e6988d2e6f49829c76b003c09", size = 306489, upload-time = "2026-06-29T13:03:37.846Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/96/3f/02fdfc6705cad96127d883af5c34e4867f554f29ec7705ec1a46156400a9/jiter-0.16.0-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:0542a7189c26920778658fc8fcf2af8bae05bae9924577f71804acef37996536", size = 335453, upload-time = "2026-06-29T13:03:39.221Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/b2/a6/e4bda5920d4b0d7c5dfb7174ce4a6b2e4d3e11c9162c452ef0eab4cdbdbd/jiter-0.16.0-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:8fb8de1e23a0cb2a7f53c335049c7b72b6db41aa6227cdcc0972a1de5cb39450", size = 361625, upload-time = "2026-06-29T13:03:40.597Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/b7/97/4e6b59b2c6e55cbb3e183595f81ad65dcfb21c915fee5e19e335df21bc55/jiter-0.16.0-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:b72d0b2990ca754a9102779ac98d8597b7cb31678958562214a007f909eab78e", size = 456958, upload-time = "2026-06-29T13:03:42.074Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/15/e0/97e9557686d2f94f4b93786eccb7eed28e9228ad132ea8237f44727314a7/jiter-0.16.0-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:d5f91b1c27fc22a57993d5a5cb8a627cb8ed4b10502716fac1ffbfe1d19d84e8", size = 372017, upload-time = "2026-06-29T13:03:43.658Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/0f/94/db768b6938e0df35c86beeba3dfbbb025c9ee5c19e1aa271f2396e50864d/jiter-0.16.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:c682bea068a90b764577bdb78a60a4c1d1606daf9cd4c893832a37c7cc9d9026", size = 343320, upload-time = "2026-06-29T13:03:45.226Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/c1/d6/5a59d938244a30735fe62d9433fd325f9021ea29d89780ea4596ea93bc89/jiter-0.16.0-cp313-cp313-manylinux_2_31_riscv64.whl", hash = "sha256:8d031aabecc4f1b6276adfb42e3aabb77c89d468bf616600e8d3a11328929053", size = 350520, upload-time = "2026-06-29T13:03:46.671Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/67/f8/c4a857f49c9af125f6bbcac7e3eee7f7978ed89682833062e2dbf62576b1/jiter-0.16.0-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:eab2cd170150e70153de16896a1774e3a1dca80154c56b54d7a812c479a7165e", size = 387550, upload-time = "2026-06-29T13:03:48.361Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/8b/d6/5fbc2f7d6b67b754caa61a993a2e626e815dec47ffc2f9e35f01adfebec7/jiter-0.16.0-cp313-cp313-musllinux_1_1_aarch64.whl", hash = "sha256:6edb63a46e65a82c26800a868e49b2cac30dd5a4218b88d74bc2c848c8ad60bb", size = 515424, upload-time = "2026-06-29T13:03:49.881Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/ed/54/284f0164b64a5fed915fea6ba7e9ba9b3d8d37c67d59cf2e3bb99d45cdfe/jiter-0.16.0-cp313-cp313-musllinux_1_1_x86_64.whl", hash = "sha256:659039cc50b5addcc35fcc87ae2c1833b7c0a8e5326ef631a75e4478447bcf84", size = 546981, upload-time = "2026-06-29T13:03:51.363Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/13/c5/2a467585a576594384e1d2c43e1224deaafc085f24e243529cf98beef8e1/jiter-0.16.0-cp313-cp313-win32.whl", hash = "sha256:c9c53be232c2e206ef9cdbad81a48bfa74c3d3f08bcf8124630a8a748aad993e", size = 202853, upload-time = "2026-06-29T13:03:53.015Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/88/6a/de61d04b9eec69c71719968d2f716532a3bc121170c44a39e14979c6be81/jiter-0.16.0-cp313-cp313-win_amd64.whl", hash = "sha256:baad945ed47f163ad833314f8e3288c396118934f94e7bbb9e243ce4b341a4fd", size = 196160, upload-time = "2026-06-29T13:03:54.447Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/19/4b/b390ed59bafb3f31d008d1218578f10327714484b334439947f7e5b11e7f/jiter-0.16.0-cp313-cp313-win_arm64.whl", hash = "sha256:3c1fd2dbe1b0af19e987f03fe66c5f5bd105a2229c1aff4ab14890b24f41d21a", size = 189862, upload-time = "2026-06-29T13:03:55.754Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/a7/89/bc4f1b57d5da938fd344a466396541e586d161320d70bffd929aaafcd8f4/jiter-0.16.0-cp314-cp314-macosx_10_12_x86_64.whl", hash = "sha256:b2c61484666ad42726029af0c00ef4541f0f3b5cdc550221f56c2343208018ee", size = 308239, upload-time = "2026-06-29T13:03:57.205Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/65/7a/c415453e5213001bf3b411ff65dec3d303b0e76a4a2cfea9768cd4960994/jiter-0.16.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:63efadc657488f45db1c676d81e704cac2abf3fdb892def1faea61db053127e2", size = 308928, upload-time = "2026-06-29T13:03:58.643Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/11/fc/1f4fb7ebf9a724c7741994f4aae18fba1e2f3133df14521a79194952c34a/jiter-0.16.0-cp314-cp314-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:cf0d73f50e7b6935677854f6e8e31d499ca7064dd24734f703e060f5b237d883", size = 336998, upload-time = "2026-06-29T13:04:00.071Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/a0/8d/72cadaac05ccfa7cc3a0a2232862e6c72443ca40cf300ba8b57f9f18b69b/jiter-0.16.0-cp314-cp314-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:bf3ea07d9bc8e7d03a9fbc051295462e6dbc295b894fd72457c3136e3e43d898", size = 362112, upload-time = "2026-06-29T13:04:01.52Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/58/4a/c4b0d5f651fda90a24ffce9f8d56cde462a2e09d31ae3de3c68cef34c04e/jiter-0.16.0-cp314-cp314-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:26798522707abb47d767db536e4148ceac1b14446bf028ee85e579a2e043cfe5", size = 459807, upload-time = "2026-06-29T13:04:03.214Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/80/58/ef77879ea9aa56b50824edc5a445e226422c7a8d211f3fd2a56bcb9493cf/jiter-0.16.0-cp314-cp314-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:bc837c1b9631be10abfe0191537fe8009838204cec7e44827401ace390ddb567", size = 373181, upload-time = "2026-06-29T13:04:04.629Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/49/2e/ffbc3f254e4d8a66da3062c624a7df4b7c2b2cf9e1fe43cf394b3e104041/jiter-0.16.0-cp314-cp314-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:49060fd70737fad59d33ba9dcc0d83247dc9e77187de26053a19c16c9f32bd69", size = 344927, upload-time = "2026-06-29T13:04:06.067Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/9a/f6/0be5dc6d64a89f80aa8fec984f94dedb2973e251edcae55841d60786d578/jiter-0.16.0-cp314-cp314-manylinux_2_31_riscv64.whl", hash = "sha256:adbb8edeadd431bc4477879d5d371ece7cb1334486584e0f252656dd7ffada29", size = 352754, upload-time = "2026-06-29T13:04:07.477Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/da/6e/7d31243b3b91cd261dd19e9d3557fc3251a80883d3d8049c86174e7ab7af/jiter-0.16.0-cp314-cp314-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:31aaee5b80f672c1dc21272bcfb9cbdcfc1ea04ff50f00ed5af500b80c44fa93", size = 390553, upload-time = "2026-06-29T13:04:08.92Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/25/33/51ae371fde3c88897520f62b4d5f8b27ad7103e2bb10812ff52195609853/jiter-0.16.0-cp314-cp314-musllinux_1_1_aarch64.whl", hash = "sha256:6722bcef4ffc86c835574b1b2fac6b33b9fb4a889c781e67950e891591f3c55a", size = 516900, upload-time = "2026-06-29T13:04:10.407Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/a0/45/6449b3d123ea439ba79507c657288f461d55049e7bcbdc2cf8eb8210f491/jiter-0.16.0-cp314-cp314-musllinux_1_1_x86_64.whl", hash = "sha256:5ab4f50ff971b611d656554ea10b75f80097392c827bc32923c6eeb6386c8b00", size = 548754, upload-time = "2026-06-29T13:04:12.046Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/9b/e7/fd2fb11ae3e2649333da3aa170d04d7b3000bbdc3b270f6513382fdf4e04/jiter-0.16.0-cp314-cp314-pyemscripten_2026_0_wasm32.whl", hash = "sha256:710cc51d4ebdcd3c1f70b232c1db1ea1344a075770422bbd4bede5708335acbe", size = 122381, upload-time = "2026-06-29T13:04:13.413Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/26/80/f0b147a62c315a164ed2168908286ca302310824c218d3aae52b06c0c9a9/jiter-0.16.0-cp314-cp314-win32.whl", hash = "sha256:57b37fc887a32d44798e4d8ebfa7c9683ff3da1d5bf38f08d1bb3573ccb39106", size = 204578, upload-time = "2026-06-29T13:04:14.813Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/5e/e6/4758a14304b4523a6f5adb2419340086aa3593bd4327c2b25b5948a90548/jiter-0.16.0-cp314-cp314-win_amd64.whl", hash = "sha256:cbd18dd5e2df96b580487b5745adf57ef64ad89ba2d9662fc3c19386acce7db8", size = 198154, upload-time = "2026-06-29T13:04:16.272Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/26/be/41fa54a2e7ea41d6c99f1dc5b1f0fd4cb474680304b5d268dd518e81da3a/jiter-0.16.0-cp314-cp314-win_arm64.whl", hash = "sha256:a32d2027a9fa67f109ff245a3252ece3ccc32cc56703e1deab6cc846a59e0585", size = 191458, upload-time = "2026-06-29T13:04:17.707Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/81/6b/59127338b86d9fe4d99418f5a15118bea778103ee0fe9d9dd7e0af174e95/jiter-0.16.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:2577196f4474ef3fc4779a088a23b0897bbf86f9ea3679c372d45b8383b43207", size = 316739, upload-time = "2026-06-29T13:04:19.663Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/2d/95/49461034d5388196d3dabf98748935f017b7785d8f3f5349f834bcc4ed0d/jiter-0.16.0-cp314-cp314t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:616e89e008a93c01104161c75b4988e58716b01d62307ebfe161e52a56d2a818", size = 340911, upload-time = "2026-06-29T13:04:21.257Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/cd/97/a4369f2fb82cb3dda13b98622f31249b2e014b223fe64ee534413ad72294/jiter-0.16.0-cp314-cp314t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:0e2e9efbe042210df657bade597f66d6d75723e3d8f45a12ea6d8167ff8bbce3", size = 361747, upload-time = "2026-06-29T13:04:22.677Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/28/51/49b6ed456261646e1906016a6760367a28aacd3c24805e4e5fe64116c1db/jiter-0.16.0-cp314-cp314t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:3f4d9e473a5ce7d27fef8b848df4dc16e283893d3f53b4a585e72c9595f3c284", size = 460225, upload-time = "2026-06-29T13:04:24.441Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/33/b5/5689aff4f66c5b60be63106e591dbfcba2190df97d2c9c7cf052361ddb98/jiter-0.16.0-cp314-cp314t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:8d30a4a1c87713060c8d1cc59a7b6c8fb6b8ef0a6900368014c76c87922a2929", size = 373169, upload-time = "2026-06-29T13:04:25.884Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/a2/96/3ae1b85ee0d6d6cab254fb7f8da018272b932bbf2d69b07e98aa2a96c746/jiter-0.16.0-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:bae96332410f866e5900d809298b1ed82735932986c672495f9701daacd80620", size = 350332, upload-time = "2026-06-29T13:04:27.302Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/15/32/c99d7bafd78986556c95bf60ce84c6cc98786eac56066c12d7f828bb6747/jiter-0.16.0-cp314-cp314t-manylinux_2_31_riscv64.whl", hash = "sha256:da3d7ec75dc83bb18bca888b5edfae0656a26849056c59e05a7728badd17e7af", size = 353377, upload-time = "2026-06-29T13:04:28.731Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/0e/4b/f99a8e571287c3dec766bcc18528bbe8e8fb5365522ab5e6d64c93e87066/jiter-0.16.0-cp314-cp314t-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:ee6162b77d49a9939229df666dfa8af3e656b6701b54c4c84966d740e189264e", size = 387746, upload-time = "2026-06-29T13:04:30.319Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/75/69/c78a5b3f71040e34eb5917df26fb7ae9a2174cad1ccbf277512507c53a6e/jiter-0.16.0-cp314-cp314t-musllinux_1_1_aarch64.whl", hash = "sha256:63ffdbdae7d4499f4cda14eadc12ddcabef0fc0c081191bdc2247489cb698077", size = 517292, upload-time = "2026-06-29T13:04:31.709Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/c2/f7/095b38eda4c70d03651c403f29a5590f16d12ddc5d544aac9f9cddf72277/jiter-0.16.0-cp314-cp314t-musllinux_1_1_x86_64.whl", hash = "sha256:a111256a7193bea0759267b10385e5870949c239ed7b6ddbaaf57573edb38734", size = 549259, upload-time = "2026-06-29T13:04:33.721Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/2e/c5/6a0207d90e5f656d95af98ebd0934f382d37674416f215aeda2ff8063e51/jiter-0.16.0-cp314-cp314t-win32.whl", hash = "sha256:de5ba8763e56b793561f43bed197c9ea55776daa5e9a6b91eed68a909bc9cdbf", size = 206523, upload-time = "2026-06-29T13:04:35.068Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/a5/31/c757d5f30a8980fd945ce7b98be10be9e4ff59c7c42f5fd86804c2e87db8/jiter-0.16.0-cp314-cp314t-win_amd64.whl", hash = "sha256:b8a3f9a6008048fe9def7bf465180564a6e458047d2ce499149cfbe73c3ae9db", size = 200366, upload-time = "2026-06-29T13:04:36.61Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/7c/a2/d88de6d313d734a544a7901353ad5db67cb38dcfcd91713b7979dafc345d/jiter-0.16.0-cp314-cp314t-win_arm64.whl", hash = "sha256:0fa25b09b13075c46f5bc174f2690525a925a4fc2f7c82969a2bbabff22386ce", size = 190516, upload-time = "2026-06-29T13:04:38.004Z" },
]
[[package]]
name = "joblib"
version = "1.5.3"
@@ -738,6 +823,68 @@ wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/46/2c/5c160dbdef7123f8cc97fd8ece7e0198627a426a2a49614845e9086feb8d/kubernetes-36.0.2-py2.py3-none-any.whl", hash = "sha256:faf9b5241b58de0c4a5069f2a0ffc8ac06fece7215156cd3d3ba081a78a858b6", size = 4617568, upload-time = "2026-06-01T18:20:28.737Z" },
]
[[package]]
name = "lxml"
version = "6.1.1"
source = { registry = "https://pypi.tuna.tsinghua.edu.cn/simple/" }
sdist = { url = "https://pypi.tuna.tsinghua.edu.cn/packages/05/3b/aab6728cae887456f409b4d75e8a01856e4f04bd510de38052a47768b680/lxml-6.1.1.tar.gz", hash = "sha256:ba96ae44888e0185281e937633a743ea90d5a196c6000f82565ebb0580012d40", size = 4197430, upload-time = "2026-05-18T19:19:06.424Z" }
wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/a5/eb/7e6f37c5584ccbb2ff267f56fd0339016938c1c8684cfefab9b33ffc2f36/lxml-6.1.1-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:68a9198d0fc122d14bb76837de9aa80cf84caed990b5b237f532ed87d3706736", size = 8559780, upload-time = "2026-05-18T19:17:57.661Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/a1/36/587c2521cf23a2cd6c9c22108aa7528f683a1f195ed7ccd23a4b1786ad36/lxml-6.1.1-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:7d47866cb32fb503450b6edc9df355d10dc49836af2e89901bd6ac6b0896d9d9", size = 4618006, upload-time = "2026-05-18T19:18:04.452Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/6e/ca/ab7bfe2bf4c972af5e7878262845ead3a24a929a9b04bc11c7c1ece6c82a/lxml-6.1.1-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:eb7c9811bfaa8b1ed5ed319f5d370dfbcaa59d52ea64be2a5a85e18195930354", size = 4924139, upload-time = "2026-05-18T19:19:04.873Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/6b/55/a0c72851dfee5ecc689f949723a73dea457758912542cb955b108eaf0d8f/lxml-6.1.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:762ff394d5bd56da0cf034a23dcce4e13923f15321a2adfa2ac00201dc6d3fca", size = 5082329, upload-time = "2026-05-18T19:19:09.728Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/f0/b6/0608f7d61a3b96cc67e5648a3d906e31a5082093e10e7be65b3886289938/lxml-6.1.1-cp313-cp313-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a088f287f7d8275a33c07f2cac6c50b9319309a0200a39e7e75d80c707723099", size = 4993564, upload-time = "2026-05-18T19:19:13.608Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/4c/66/ae227524b066d29d55bf0b453d93d2d793c40218657d643dcbbca13b8faf/lxml-6.1.1-cp313-cp313-manylinux_2_26_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:e902da4b04e6b52e5893900d4b8ab46068f75f3561f01bf1080957f9fd932ed6", size = 5613467, upload-time = "2026-05-18T19:19:16.228Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/a6/76/dbe4a00b50385e40194231dcfe5a12c059de7cf90e89c83407d2b085b719/lxml-6.1.1-cp313-cp313-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1d4962d4c66bf830a7e59ed6cfc17d148149898a3aefa8ec6e59763e6e3ed085", size = 5228304, upload-time = "2026-05-18T19:19:19.354Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/1c/01/00b1b8442ed2041793336868ba0b9ea4b13d7da7c085c6404c207a63bf79/lxml-6.1.1-cp313-cp313-manylinux_2_28_i686.whl", hash = "sha256:581d4c8ae690a6609e64862dd6b7c2489635c2d13907fc2b20f2bc200ff1d21e", size = 5341607, upload-time = "2026-05-18T19:19:22.297Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/63/36/1ad29931e9a4638bb707869f01d423a6c815f82152138d1a40dfcfde2b95/lxml-6.1.1-cp313-cp313-manylinux_2_31_armv7l.whl", hash = "sha256:876e1ff5930ed8bf295ec5ef9a8155e9b6b1876bbf1deed8b3a8069311875a8f", size = 4700168, upload-time = "2026-05-18T19:19:25.133Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/3c/d1/a9536cecf9be18a0dc72d32bead283a2332d1ffebd2dd3ac70ce444686e5/lxml-6.1.1-cp313-cp313-manylinux_2_38_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:9eb9b5a968f6e0f6d640092a567e14529ff8cea2e29d00da6f78a79fa49f013c", size = 5232487, upload-time = "2026-05-18T19:19:28.603Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/0e/77/b4fb1e03bf5d130e879214d3100092e386418807fb74dd0adc4b0a48f351/lxml-6.1.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:aa49e06d94aba782c6a02eecb7e507969e7e7a41b267f1b359bb35585f295d5b", size = 5044231, upload-time = "2026-05-18T19:18:42.246Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/26/4c/d00daeeb0a5530c4028a9232aa1b93db3ef4ed2158c116ea73c79a9765b3/lxml-6.1.1-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:70cdfd80589d59e43e18005dd7244e8895e93db8ab6a620b7e23df5445a4e3d2", size = 4769450, upload-time = "2026-05-18T19:18:48.013Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/ed/6a/715a3a8d156ce42f29cf014706f5410c2ff3b02267774110fc23266409fe/lxml-6.1.1-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:aad9aa39483ed8ec44d6d2e59e5b98a0d80676ef0d92f44bfc374836111f62f5", size = 5635874, upload-time = "2026-05-18T19:18:51.914Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/45/37/0544bc21dde2a88f3a17b504e6fc79c0e01d25a33c2f6079724e9e72b9c7/lxml-6.1.1-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:d49514be2f28d895c38cf9d2b72d7b9a07d00314519f456c0b50b53cfcf4c785", size = 5223987, upload-time = "2026-05-18T19:18:59.715Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/4d/f8/f6a5e8185bcb28c2befae3d31f8e3df3b811cb0f47746517a81279fcafe1/lxml-6.1.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:47402e62c52ff5988c1e8c6c63177f5708bccf48e366dea4e3dcf1e645e04947", size = 5250276, upload-time = "2026-05-18T19:19:03.834Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/c7/f2/1a2b9f1b7a49d45495369be7ef9ad05b262930f2eab3e3145706fca8083f/lxml-6.1.1-cp313-cp313-win32.whl", hash = "sha256:3483644525531e1d5762b0c44a8e18b6efba321b6dcf8a8952de10b037618bca", size = 3596903, upload-time = "2026-05-18T19:17:29.863Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/e6/99/f4ffb024f238eec2131aaa09f3278fb6129cf892741bf68e1fc1afb8c100/lxml-6.1.1-cp313-cp313-win_amd64.whl", hash = "sha256:a10bd2fd62e8ce916ececb342f348f190724a098c1faa056fdfb2a22ad5e8660", size = 3995869, upload-time = "2026-05-18T19:18:02.596Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/d1/53/70eb8c5c6037f27448f1e3c54ebede9545a801ae63f0a7254afca4fe8e45/lxml-6.1.1-cp313-cp313-win_arm64.whl", hash = "sha256:424aa57aca0897eb922aef34395bd1289b3b6f04e6bae20ea123c0c7e333cffc", size = 3658490, upload-time = "2026-05-19T19:22:53.846Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/13/e2/2e325795566de01d0d7c3bb57d3c370616b2d07b01214e84eec5d3b10963/lxml-6.1.1-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:19b7ab10b210b0b3ad7985d9ac4eb66ab09a90b20fe6e2f7ba55d01a234345d0", size = 8577146, upload-time = "2026-05-18T19:18:17.765Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/93/cf/5630b5e4be7d2e6bee8efe83865c925221103cf0221303b104ce134b01e2/lxml-6.1.1-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:c08e5c694306507275f2290073350c4f32e383db15213b2c69e7ff39c1193840", size = 4623866, upload-time = "2026-05-18T19:18:30.669Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/d2/51/3904907c063451cf8d4a5c9fe0cad95fa1f4ec57f4e3884fa0731bd7a305/lxml-6.1.1-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:74a9717fd0d82effef5c2854f0d917231d5324b5a3eb7275c43ac9fa32f97a14", size = 4950022, upload-time = "2026-05-18T19:19:31.958Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/94/cd/9c7611a51c37a2830928405817cc5d56a97f64fab83cc3f628748b135749/lxml-6.1.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:efe0374196335f93b53269acd811b944f2e6bdc88e8894f214bd636455484909", size = 5086695, upload-time = "2026-05-18T19:19:34.764Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/da/d6/24e3b5906abb0b674ff2ae195bc3ce59708df2bcd17cf17703b2d7dd643a/lxml-6.1.1-cp314-cp314-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ac931cdc9442c1763b8a8f6cd62c0c938737eafc5be75eff88df55fc73bc0d00", size = 5031642, upload-time = "2026-05-18T19:19:37.771Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/2d/db/6ec54f99019838bff54785c51da07f189eb4676861c5f2730962b0d8d665/lxml-6.1.1-cp314-cp314-manylinux_2_26_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:aee395f5d0927f947758b4ec119fd5fc8ec71f07a1c5c52077b30b04c0fa6955", size = 5647338, upload-time = "2026-05-18T19:19:40.553Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/42/3d/ef4dcfffd22d27a61805d8ed9f7fb888495bc6aa88648fa07c1eaa5586b6/lxml-6.1.1-cp314-cp314-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9395002973c827b3ed67db77e6ec09f092919a587022174554096a269378fb13", size = 5239528, upload-time = "2026-05-18T19:19:43.657Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/62/bb/37fb3f0dff146bdcfa78eec47879273820b2a0bf350ec236ce14bd0b1c26/lxml-6.1.1-cp314-cp314-manylinux_2_28_i686.whl", hash = "sha256:73bc2086f141224ebddb7fc5c6a36ca58b31b94b561e1dfe8e073e3270fad1e7", size = 5350730, upload-time = "2026-05-18T19:19:46.307Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/90/42/43253f168388df4fae1f38c01df36ddb9bee39e2048167b54cdcbae85ea3/lxml-6.1.1-cp314-cp314-manylinux_2_31_armv7l.whl", hash = "sha256:3779def59032b81e44a5f70096ef6bf2082f8d901937dca354474ba09782e245", size = 4697530, upload-time = "2026-05-18T19:19:49.889Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/eb/a8/c5a8504f81bbdfc8e7094c2c850cdb4ed6777fc4d5ddd9e5ab819f3b0d54/lxml-6.1.1-cp314-cp314-manylinux_2_38_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:86c89b9d55ebf820ad7c90bc533410f0d098054f293351f10603c0c46ff598f5", size = 5250670, upload-time = "2026-05-18T19:19:53.199Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/77/b7/c7e76ab18744d75e21f320ebf9ff9d1ceae2b54dd431ea5a64caf26c9672/lxml-6.1.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:19607c6bbff2a44cf3fe8250abccd20942d3462473e0a721d01d379ed017e462", size = 5084485, upload-time = "2026-05-18T19:19:08.422Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/31/31/b35c53f8ef7b7c31cacd23d3638652fff7bcd1deb6eedb709ab43b685908/lxml-6.1.1-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:c6ed5141a5c7507cf3ee76bd363b0d6f801e3321adc35b5d825a23115faa5465", size = 4737635, upload-time = "2026-05-18T19:19:12.321Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/d9/06/31f23c813a7fe8e0cb1b175e915b08c9bf4e86d225b210feadbdbe519667/lxml-6.1.1-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:62aeb7e85b5d60320b9d77eef2e773994e2c0ce10121b277e0a19804e1654a5a", size = 5670681, upload-time = "2026-05-18T19:19:15.001Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/1a/bc/ce619bccc89b1fd9ad8a8e1330ee3f3beff9f2ff95b712d7bbcdd6e22fc3/lxml-6.1.1-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:b1b963fd8f5caa68e99dfae060d54de1fe9cba899b8718b44a00cdca53c3e590", size = 5238229, upload-time = "2026-05-18T19:19:18.131Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/2f/5d/b329acbbedc0b619ebc2be6cf7ee9ed07e80892c88d4dfd612c33805789a/lxml-6.1.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:63876be28efefa04a1df615b46770e82042cce445cfdce55160522f57b231ccb", size = 5264191, upload-time = "2026-05-18T19:19:21.118Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/d6/85/be36fb1425b30db3c3f9df75fe86343ebffb79e6320bd7f588e25bfeac39/lxml-6.1.1-cp314-cp314-win32.whl", hash = "sha256:7f7a92e8583f06b1fd49d01158143b8461cfcd135dcb10ec807270a3051bd603", size = 3657202, upload-time = "2026-05-18T19:17:39.509Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/b8/ce/3cf9a827342269f54d405a6202397de63f07c69cbd6ce7d183a3f0cba1e9/lxml-6.1.1-cp314-cp314-win_amd64.whl", hash = "sha256:b2d444f2e66624d68e9c6b211e28a76e22fff5fcabcfff4deac18b529b7d4137", size = 4064497, upload-time = "2026-05-18T19:18:14.662Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/d9/3e/1a957bde8f0760039e627f94699f82caa782c9d838d86c3d28245ee67212/lxml-6.1.1-cp314-cp314-win_arm64.whl", hash = "sha256:3fd9728a2735fda14f4e8235830c86b539e9661e849665bf926d3f867943b4bf", size = 3741991, upload-time = "2026-05-19T19:22:59.111Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/78/b2/00ed55b3a2efa4658fb795c38d1090ec9b3e8a6c3683d4441fa517f09c3b/lxml-6.1.1-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:787b2496d0dbe8cd180984e8d29e3a6f76e7ea34db781cb3bd55e4ba1ef8b4ee", size = 8827545, upload-time = "2026-05-18T19:18:41.193Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/c0/73/74573db19baa618d5f266f2407898b087ff6927115b00b71e5fc1b700847/lxml-6.1.1-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:2c8daa471358dc2d6fcf02165e80ec68f77871a286df95bc5cc3816153b0fd2c", size = 4735736, upload-time = "2026-05-18T19:18:46.761Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/16/02/6f7061f4f95f51e545d48e87647c54791d204a4e881be4156e7a26ba5338/lxml-6.1.1-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:acd7d70b64c0aae0c7922cca83d288a16f5f6da523637697872253415269baef", size = 4970291, upload-time = "2026-05-18T19:19:56.215Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/b0/02/55fc057d8283427dea7d6edb102e7a840239c77a64a983d92f62a304c0e9/lxml-6.1.1-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:4f0dd2f01f9f8a89f565d000e03abcf0a13d692a346c8d22f628d49af098777a", size = 5102822, upload-time = "2026-05-18T19:19:59.223Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/e4/48/8e1cf78d89d66850121d9255a2a24414c98f775da93b90cf976956c24b14/lxml-6.1.1-cp314-cp314t-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0b7e8a14c8634bf6f7a568634cb395305a6d964aeb5b7ee32248094bed3a7e2c", size = 5027923, upload-time = "2026-05-18T19:20:01.549Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/ed/00/0632a0647612c8af24d26997b3b961397daa9d5b2581444805933629a4cb/lxml-6.1.1-cp314-cp314t-manylinux_2_26_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:86281fbdd6a8162756f8d603f37e3435bfa38043adb79c6dc6a2dfee065e7525", size = 5595843, upload-time = "2026-05-18T19:20:03.93Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/bc/86/ab008a7dc360711b66858d61c80a5979a70a09f2aa2b05d9698df80b803d/lxml-6.1.1-cp314-cp314t-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c5d7152ec39ca7c402d8fb9bad86140a15b9503bd0c54484e3f1bbe3dd37ceca", size = 5224515, upload-time = "2026-05-18T19:20:06.381Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/75/c6/2702ff375e728e34f56d9a45339a9cf7e4427e917f542225242d63a05afa/lxml-6.1.1-cp314-cp314t-manylinux_2_28_i686.whl", hash = "sha256:88d8cb75b9d82858497a5393e3c63cfbf03035225e4b35a49ed7ccb151e4dc0e", size = 5312511, upload-time = "2026-05-18T19:20:09.308Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/b7/57/a5807c98f87a86f10ef9ffab35516df7c0f0c4b6d5d33e9f608ab9c04a31/lxml-6.1.1-cp314-cp314t-manylinux_2_31_armv7l.whl", hash = "sha256:f64ec5397ea6a41fc1b4af0380d79b44a755b5531dcaccd9940fb260dca93038", size = 4639206, upload-time = "2026-05-18T19:20:11.704Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/1f/e1/8a0a2c35734812395f4da4eaf33748a7e5705bfb2a58b128da764339d5ec/lxml-6.1.1-cp314-cp314t-manylinux_2_38_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:d34bbf07dbc7ca5970671b1512e928991fb5e9d95365636c9b2d8b4f53af405e", size = 5232404, upload-time = "2026-05-18T19:20:14.064Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/c2/e2/0e6a4dd5ad84d01d99aa7bae7cfefd4a760a0e0f8176818241de17d9b6c0/lxml-6.1.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:17e0e18d4ad8adbd0399291bc44845b69d9dd68439a3cdebdf35ff902ec05072", size = 5083769, upload-time = "2026-05-18T19:19:23.758Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/a0/7e/161f33d463f6ffc1c7679104b65086dea120080d49dde4d238f015aaee2f/lxml-6.1.1-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:3ab541146f1f6968c462d6c2ac495148e8cdba2f8347700b2141b6ec5a75bf52", size = 4758936, upload-time = "2026-05-18T19:19:27.256Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/f1/fb/2369825e3f6ca99305bf9f7b7085fda91c8b0922a89e54d900974aa3ef85/lxml-6.1.1-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:2a0217714657e023ef4293500f65aa20fce6164c8fd6b08fa5bd4a859fb14b9b", size = 5620296, upload-time = "2026-05-18T19:19:29.993Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/30/90/d61e383146f74c5ab683947ea14dc7b82778838ab9b95ea73a23b60d0191/lxml-6.1.1-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:05a82eb6e1530a64f26225b55cbd178113bd0b5af1c2b625f25e5296742c26d2", size = 5228598, upload-time = "2026-05-18T19:19:33.523Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/76/2d/2dafd8149e94b05bb070690efd5bb2680720681e03ff03fc57d2b70a1105/lxml-6.1.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:9e36f163528fc50cbef305f02a5fd66d404edf7049cdaff211dbc2cba5a7013e", size = 5247845, upload-time = "2026-05-18T19:19:36.649Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/ce/68/b30e913340c380ddac9580c6e6230991fc37240ec4f64704833e4f3e2769/lxml-6.1.1-cp314-cp314t-win32.whl", hash = "sha256:649dda677cf3bd6ac9ae14007ba0c824ded8ce5808b53fc7431d9140399118c1", size = 3897345, upload-time = "2026-05-18T19:17:33.562Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/3c/4e/9eb2af5335545f9fbcd7af57bcf87c6025d31eaa31b14ec184a6c8675328/lxml-6.1.1-cp314-cp314t-win_amd64.whl", hash = "sha256:793033d6c5cdf33a573f910d9bea14ef8f5771820411d118da8e1182edb53d5e", size = 4393350, upload-time = "2026-05-18T19:18:10.076Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/7f/2c/0f1e93c636720e8a3eb59af2bfda99d98b55891e1c53bc30c2e0e865f01b/lxml-6.1.1-cp314-cp314t-win_arm64.whl", hash = "sha256:58bb955caba94e467d2a96da17660d2d704e0675894cba21ab8a775b8621fd1c", size = 3817223, upload-time = "2026-05-19T19:22:56.823Z" },
]
[[package]]
name = "markdown-it-py"
version = "4.2.0"
@@ -818,25 +965,47 @@ dependencies = [
]
[package.optional-dependencies]
all = [
{ name = "beautifulsoup4" },
{ name = "ebooklib" },
{ name = "openai" },
{ name = "pymupdf" },
{ name = "requests" },
]
dev = [
{ name = "httpx" },
{ name = "pytest" },
]
epub = [
{ name = "ebooklib" },
]
html = [
{ name = "beautifulsoup4" },
]
pdf = [
{ name = "pymupdf" },
]
[package.metadata]
requires-dist = [
{ name = "beautifulsoup4", marker = "extra == 'html'", specifier = ">=4.12.0" },
{ name = "chromadb", specifier = ">=0.5.0" },
{ name = "ebooklib", marker = "extra == 'epub'", specifier = ">=0.18" },
{ name = "fastapi", specifier = ">=0.115.0" },
{ name = "httpx", marker = "extra == 'dev'", specifier = ">=0.27.0" },
{ name = "markdown-it-py", specifier = ">=3.0.0" },
{ name = "md-vector-db", extras = ["pdf", "html", "epub"], marker = "extra == 'all'" },
{ name = "openai", marker = "extra == 'all'", specifier = ">=1.0.0" },
{ name = "pymupdf", marker = "extra == 'pdf'", specifier = ">=1.24.0" },
{ name = "pytest", marker = "extra == 'dev'", specifier = ">=8.0" },
{ name = "python-dotenv", specifier = ">=1.2.2" },
{ name = "pyyaml", specifier = ">=6.0" },
{ name = "requests", marker = "extra == 'all'", specifier = ">=2.31.0" },
{ name = "sentence-transformers", specifier = ">=3.0.0" },
{ name = "typer", specifier = ">=0.12.0" },
{ name = "uvicorn", extras = ["standard"], specifier = ">=0.30.0" },
]
provides-extras = ["dev"]
provides-extras = ["dev", "pdf", "html", "epub", "all"]
[[package]]
name = "mdurl"
@@ -1215,6 +1384,25 @@ wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/b7/f6/2bac21f722aa45d876d4a51f26bd0ef30e704068a3cd5021a5a7cd784271/onnxruntime-1.27.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:370d211e1ceeac4cd5f45301655463ac59e27cdc74d9f7aeb2d19ff4b7a76715", size = 18670781, upload-time = "2026-06-15T22:43:17.151Z" },
]
[[package]]
name = "openai"
version = "2.45.0"
source = { registry = "https://pypi.tuna.tsinghua.edu.cn/simple/" }
dependencies = [
{ name = "anyio" },
{ name = "distro" },
{ name = "httpx" },
{ name = "jiter" },
{ name = "pydantic" },
{ name = "sniffio" },
{ name = "tqdm" },
{ name = "typing-extensions" },
]
sdist = { url = "https://pypi.tuna.tsinghua.edu.cn/packages/78/60/d4219875289b11d2c2f7da93c36283da224a2e55865ed865ab64e0ce9217/openai-2.45.0.tar.gz", hash = "sha256:10d34ca9c5643bce775852fddbfc172505cb1d4de1ccd101696c3ecff358765d", size = 1109653, upload-time = "2026-07-09T18:02:44.091Z" }
wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/f1/b0/2291689e3ec4723fbf5bbf3b54afcd7b160f9ddc98ca7aedfd0132af5677/openai-2.45.0-py3-none-any.whl", hash = "sha256:5df105f5f8c9b711fcb9d06d2d3888cebc82506db216484c14a4e53cdf651777", size = 1629470, upload-time = "2026-07-09T18:02:42.21Z" },
]
[[package]]
name = "opentelemetry-api"
version = "1.43.0"
@@ -1641,6 +1829,23 @@ wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl", hash = "sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176", size = 1231151, upload-time = "2026-03-29T13:29:30.038Z" },
]
[[package]]
name = "pymupdf"
version = "1.28.0"
source = { registry = "https://pypi.tuna.tsinghua.edu.cn/simple/" }
sdist = { url = "https://pypi.tuna.tsinghua.edu.cn/packages/8e/e9/6d6c5d6c0a3551bffd47681a6240caf941727f195b45593cf20ab36f018f/pymupdf-1.28.0.tar.gz", hash = "sha256:e53f3567403a92da15caa9e7ae0164327fff48817e9f40175367fb9de524258d", size = 87637751, upload-time = "2026-06-29T09:08:47.547Z" }
wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/c8/b7/88043e38cc7529de070f0c9bd267fa258035cca0b4ad5260536b994594a7/pymupdf-1.28.0-cp310-abi3-macosx_10_15_x86_64.whl", hash = "sha256:892b89ba88e8f98b53133b62877a9dc9b5e7dc6a4aeb837b612db56a8d2e03ac", size = 24597385, upload-time = "2026-06-29T09:03:30.608Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/33/f4/23775bbda0781b61fc398cc75079a2b0e64696d8fcf93271748883e9627e/pymupdf-1.28.0-cp310-abi3-macosx_11_0_arm64.whl", hash = "sha256:4d692dcf44d3566ae96bc6f6346c6ad432274a29ba617bf7a9fe18009e24adb4", size = 23828292, upload-time = "2026-06-29T09:03:46.129Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/1c/f5/bf75fc7a415722f8b33662054f82d88520c0cbfd4c36d0e08aeaec605e49/pymupdf-1.28.0-cp310-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:47a5c29ed4eb0744de9c4e37bb49b1259b18d4d75fcc8a7c130f7c9fa15956f6", size = 25045507, upload-time = "2026-06-29T09:04:03.86Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/58/69/5d12c9f1f2d76f28383d6110a069c79fbfced5a4f97bb1ee6e8354f52bb7/pymupdf-1.28.0-cp310-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:44f0973f5e5edbaec95bc34b64e71d1959d4ee90b1328de1b4f4f5b4fa78673f", size = 25716599, upload-time = "2026-06-29T09:04:19.367Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/4d/b4/ec0e017bc42857cc86bd651441dbc41cc18be48d4698ecd27aac491e0c9a/pymupdf-1.28.0-cp310-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:4d61ec323a706e153a12e262e51febfb43eeaa20977785ace135d18d48bcdc83", size = 25940489, upload-time = "2026-06-29T09:04:36.624Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/06/86/f831fef09013f33b3c9c09fb3923f2ff53e1e437f6ace14b8ae46392f558/pymupdf-1.28.0-cp310-abi3-win32.whl", hash = "sha256:caea2b3b67347fd79e5d15ed7929b0e886aac594ea228073b6d39de0078189da", size = 18489703, upload-time = "2026-06-29T20:50:30.599Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/2e/5d/1a03f53eb0449900469335fcfc742ca28e3ba159b7d650e0921d50b8b308/pymupdf-1.28.0-cp310-abi3-win_amd64.whl", hash = "sha256:e01e90fd86abfeb37ceb921eddb951f988a11d45ff6ce6b7664f2039849068ec", size = 19773102, upload-time = "2026-06-29T09:04:49.773Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/72/f6/1e52ce243ca792254f6223b4017c5667194c146ce9b88baf37bc5eb3d1c9/pymupdf-1.28.0-cp313-abi3-pyemscripten_2025_0_wasm32.whl", hash = "sha256:74c6d00ba2a9aad3a635db73b07c15db462b480741d831a34a75a56535ebc22b", size = 18357011, upload-time = "2026-06-29T20:50:50.353Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/62/b1/46b5b3d8ef3cc71114667cf10c4d8b33f39af97253af32e9a0986775b638/pymupdf-1.28.0-cp314-cp314t-manylinux_2_28_x86_64.whl", hash = "sha256:b3e1399c7a64c6914239116a369efcdaac4cfb9e838bde2656d7accc4a85c72d", size = 25753599, upload-time = "2026-06-29T09:05:09.398Z" },
]
[[package]]
name = "pypika"
version = "0.51.1"
@@ -2084,6 +2289,24 @@ wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/b7/ce/149a00dd41f10bc29e5921b496af8b574d8413afcd5e30dfa0ed46c2cc5e/six-1.17.0-py2.py3-none-any.whl", hash = "sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274", size = 11050, upload-time = "2024-12-04T17:35:26.475Z" },
]
[[package]]
name = "sniffio"
version = "1.3.1"
source = { registry = "https://pypi.tuna.tsinghua.edu.cn/simple/" }
sdist = { url = "https://pypi.tuna.tsinghua.edu.cn/packages/a2/87/a6771e1546d97e7e041b6ae58d80074f81b7d5121207425c964ddf5cfdbd/sniffio-1.3.1.tar.gz", hash = "sha256:f4324edc670a0f49750a81b895f35c3adb843cca46f0530f79fc1babb23789dc", size = 20372, upload-time = "2024-02-25T23:20:04.057Z" }
wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/e9/44/75a9c9421471a6c4805dbf2356f7c181a29c1879239abab1ea2cc8f38b40/sniffio-1.3.1-py3-none-any.whl", hash = "sha256:2f6da418d1f1e0fddd844478f41680e794e6051915791a034ff65e5f100525a2", size = 10235, upload-time = "2024-02-25T23:20:01.196Z" },
]
[[package]]
name = "soupsieve"
version = "2.8.4"
source = { registry = "https://pypi.tuna.tsinghua.edu.cn/simple/" }
sdist = { url = "https://pypi.tuna.tsinghua.edu.cn/packages/47/2c/0a5f6f8ee0d5589e48c7640213ed5175d52cf540a06725b628cc1a45d6ce/soupsieve-2.8.4.tar.gz", hash = "sha256:e121fd02e975c695e4e9e8774a5ee35d74714b59307868dcc5319ad2d9e3328e", size = 121110, upload-time = "2026-05-24T13:55:57.154Z" }
wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/5e/f5/0c41cb68dcae6b7de4fac4188a3a9589e21fb31df21ea3a2e888db95e6c9/soupsieve-2.8.4-py3-none-any.whl", hash = "sha256:e7e6b0769c8f51ed59acab6e994b00621096cfb1c640a7509295987388fbaf65", size = 37304, upload-time = "2026-05-24T13:55:55.406Z" },
]
[[package]]
name = "starlette"
version = "1.3.1"