fix: 修复 44 个代码审查问题 (CRITICAL/HIGH/MEDIUM/LOW)

Batch 1 — CRITICAL (1):
- 提取 is_safe_path() 到 src/core/security.py 公共模块
- CLI 和 ingest_obsidian.py 统一添加路径遍历防护

Batch 2 — HIGH (13) + 架构重构:
- CLI 复用 deps.py AppState, 消除 30 行重复代码
- AppState/get_state 添加线程安全锁
- serve 命令传递 --config 到 uvicorn (H1)
- OpenAIEmbedder 懒创建+复用 HTTP 客户端 (H2)
- DashscopeEmbedder import 移到模块顶部 (H3)
- 路径检查改用 os.path.commonpath (H4)
- embedder.embed() 返回值长度检查 (H5)
- 健康检查不泄露内部错误详情 (H7)
- /api/v1/collections 添加 API Key 认证 (H8)
- API Key 使用 hmac.compare_digest 恒定时间比较 (H9)
- 添加 CORS 中间件 (H10)
- ServerConfig 支持 SSL 配置 (H11)
- HF_ENDPOINT 修改添加详细注释 (H12)

Batch 3 — MEDIUM (20) + Splitter Protocol:
- 定义 Splitter(Protocol) 接口, DocumentIngestor 接受可选 splitter
- DashScope 响应添加结构验证 (M2)
- ingest_obsidian.py 支持 CLI 参数和 OBSIDIAN_DIRS 环境变量 (M6)
- scripts/serve.py 添加废弃警告 (M7)
- content 限制 500KB, collection 正则限制字符集 (M12-M14)
- 默认监听地址 127.0.0.1 (M16)
- 添加安全响应头中间件 (M17)
- verify_api_key 认证失败记录日志 (M19)

Batch 4 — LOW (10):
- CLI emoji 清理为纯文本标记 (L5)
- logging.basicConfig 移到 FastAPI lifespan (L1)
- VectorDB 添加 write_guard() 上下文管理器 (L3)
- IngestRequest file_path/content 互斥校验 (L10)
- ingest_obsidian.py 注释修正 (L6)

测试: 46 → 70 (+24)
- tests/test_security.py: 11 个路径安全测试
- tests/test_deps.py: 11 个依赖注入测试

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
2026-07-06 16:56:38 +08:00
parent 832201186d
commit 405303e82c
14 changed files with 473 additions and 130 deletions
+21 -7
View File
@@ -21,11 +21,13 @@ import os
import logging
from typing import Protocol
import requests # noqa: F401 — DashscopeEmbedder 使用
from src.core.config import EmbedConfig
logger = logging.getLogger(__name__)
logger = logging.getLogger("md-vector-db")
_HF_MIRROR = "https://hf-mirror.com"
_HF_MIRROR = os.environ.get("HF_MIRROR", "https://hf-mirror.com")
# -- Provider 默认配置 --
_PROVIDER_DEFAULTS: dict[str, dict[str, str | int]] = {
@@ -73,6 +75,8 @@ class LocalEmbedder:
)
except Exception:
logger.info("模型未缓存, 通过镜像下载 %s", config.local_model)
# 通过 HF_ENDPOINT 环境变量设置镜像(sentence-transformers 依赖 huggingface_hub
# 临时设置仅用于模型下载,下载完成后还原
old_endpoint = os.environ.get("HF_ENDPOINT")
os.environ["HF_ENDPOINT"] = _HF_MIRROR
try:
@@ -121,13 +125,19 @@ class OpenAIEmbedder(_BaseAPIEmbedder):
def __init__(self, config: EmbedConfig):
super().__init__(config, "openai")
self._client = None # 懒初始化,首次 embed() 时创建
def _ensure_client(self):
"""懒创建 OpenAI 客户端(避免 import 时依赖 openai 包)."""
if self._client is None:
from openai import OpenAI
self._client = OpenAI(base_url=self._api_base, api_key=self._api_key)
def embed(self, texts: list[str]) -> list[list[float]]:
if not texts:
raise ValueError("文本列表不能为空")
from openai import OpenAI
client = OpenAI(base_url=self._api_base, api_key=self._api_key)
response = client.embeddings.create(model=self._model, input=texts)
self._ensure_client()
response = self._client.embeddings.create(model=self._model, input=texts)
return [d.embedding for d in response.data]
@@ -141,7 +151,6 @@ class DashscopeEmbedder(_BaseAPIEmbedder):
def embed(self, texts: list[str]) -> list[list[float]]:
if not texts:
raise ValueError("文本列表不能为空")
import requests
resp = requests.post(
self._api_base,
headers={
@@ -157,7 +166,12 @@ class DashscopeEmbedder(_BaseAPIEmbedder):
resp.raise_for_status()
data = resp.json()
# DashScope 返回: {"output": {"embeddings": [{"text_index": 0, "embedding": [...]}, ...]}}
embeddings_raw = data.get("output", {}).get("embeddings", [])
output = data.get("output")
if output is None:
raise ValueError(f"DashScope 响应缺少 output 字段: {data}")
embeddings_raw = output.get("embeddings")
if not isinstance(embeddings_raw, list):
raise ValueError(f"DashScope embeddings 不是列表: {type(embeddings_raw)}")
# 按 text_index 排序确保顺序
embeddings_raw.sort(key=lambda x: x.get("text_index", 0))
return [e["embedding"] for e in embeddings_raw]