1334 lines
39 KiB
Markdown
1334 lines
39 KiB
Markdown
# 第二优先级：体验完善 实施计划
|
||
|
||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||
|
||
**Goal:** 补齐 Web 管理界面、.docx 文档支持、数据导出备份功能，并将测试覆盖率从 75% 提升到 ≥80%。
|
||
|
||
**Architecture:** Web UI 使用单文件 Vue 3 CDN 方案（零构建），嵌入 FastAPI 静态文件服务；.docx 支持通过集成 markitdown 库新增 DocxSplitter；导出功能在 CLI 新增 `export` 命令支持 JSON/CSV 格式。
|
||
|
||
**Tech Stack:** Vue 3 (CDN), markitdown, FastAPI StaticFiles, CSV/JSON
|
||
|
||
**预估总工作量:** 约 10-15 小时
|
||
|
||
---
|
||
|
||
## 文件结构规划
|
||
|
||
```
|
||
新增文件:
|
||
src/web/index.html — Web 管理界面（单文件 Vue 3 SPA）
|
||
src/core/splitters/docx.py — .docx 分块器
|
||
tests/test_splitters_docx.py — DocxSplitter 测试
|
||
tests/test_export.py — 导出功能测试
|
||
|
||
修改文件:
|
||
src/server/app.py — 挂载静态文件 + CORS 修复
|
||
src/cli/main.py — 新增 export 命令
|
||
src/core/splitters/registry.py — 注册 .docx
|
||
src/core/ingest.py — 修复 PDF/EPUB splitter 接口不一致
|
||
src/core/search.py — 添加 export 方法
|
||
src/core/embedder.py — 修复 Embedder Protocol 定义
|
||
src/core/splitters/pdf.py — 实现 Splitter Protocol
|
||
src/core/splitters/epub.py — 实现 Splitter Protocol
|
||
pyproject.toml — 新依赖
|
||
config.yaml — 更新示例
|
||
README.md — Web UI 使用说明
|
||
tests/test_api.py — 静态文件端点测试
|
||
tests/test_splitters_pdf.py — 实体测试（非 skip）
|
||
tests/test_splitters_html.py — 实体测试（非 skip）
|
||
tests/test_splitters_epub.py — 实体测试（非 skip）
|
||
tests/test_ingest.py — 补充 ingest_directory / content 模式测试
|
||
tests/test_search.py — 补充 list_sources / delete_by_source 测试
|
||
tests/test_db.py — 补充 write_guard / close 测试
|
||
tests/test_cli.py — 补充 ingest-dir / stats 测试
|
||
```
|
||
|
||
---
|
||
|
||
### Task 1: 修复 Embedder Protocol 和 Splitter 接口
|
||
|
||
**Files:**
|
||
|
||
- Modify: `src/core/embedder.py`
|
||
- Modify: `src/core/splitters/pdf.py`
|
||
- Modify: `src/core/splitters/epub.py`
|
||
|
||
- [ ] **Step 1: 修复 Embedder Protocol 为标准写法**
|
||
|
||
`src/core/embedder.py` 的 `Embedder` 类，将方法体改为标准写法：
|
||
|
||
```python
|
||
class Embedder(Protocol):
|
||
"""嵌入器接口."""
|
||
@property
|
||
def dimension(self) -> int:
|
||
"""返回嵌入向量的维度."""
|
||
...
|
||
|
||
def embed(self, texts: list[str]) -> list[list[float]]:
|
||
"""对文本列表进行嵌入.
|
||
|
||
Args:
|
||
texts: 待嵌入的文本列表
|
||
|
||
Returns:
|
||
嵌入向量列表，每个向量为 float 列表
|
||
"""
|
||
...
|
||
```
|
||
|
||
- [ ] **Step 2: PDFSplitter 显式实现 Splitter Protocol**
|
||
|
||
修改 `src/core/splitters/pdf.py`，将 `split` 方法的参数名从 `text` 改为 `source`：
|
||
|
||
```python
|
||
class PDFSplitter:
|
||
"""PDF 分块器：pymupdf 提取文字 → TextSplitter 分块.
|
||
|
||
实现 Splitter Protocol，内部组合 TextSplitter 实例。
|
||
split() 的 source 参数接收 PDF 文件路径（非文本内容）。
|
||
"""
|
||
|
||
def __init__(self, max_size: int = 1000, overlap: int = 100):
|
||
self._text_splitter = TextSplitter(max_size=max_size, overlap=overlap)
|
||
|
||
def split(self, source: str, source_file: str = "") -> list[dict]:
|
||
"""从 PDF 文件提取文字并分块.
|
||
|
||
Args:
|
||
source: PDF 文件路径
|
||
source_file: 来源文件名
|
||
"""
|
||
# ... 其余实现不变，将 text 改为 source ...
|
||
```
|
||
|
||
同步修改 `ingest.py` 中 PDF/EPUB splitter 的调用，使用关键字参数：
|
||
|
||
```python
|
||
# ingest.py 第 57 行附近，改为：
|
||
chunks = splitter.split(source=str(path), source_file=file_name)
|
||
```
|
||
|
||
- [ ] **Step 3: EPUBSplitter 同样修改**
|
||
|
||
修改 `src/core/splitters/epub.py` 的 `split` 方法签名：
|
||
|
||
```python
|
||
def split(self, source: str, source_file: str = "") -> list[dict]:
|
||
"""从 EPUB 文件提取各章节文字并分块.
|
||
|
||
Args:
|
||
source: EPUB 文件路径
|
||
source_file: 来源文件名
|
||
"""
|
||
# ... 将 text 改为 source ...
|
||
```
|
||
|
||
- [ ] **Step 4: 确认测试通过**
|
||
|
||
```bash
|
||
uv run pytest tests/test_embedder.py tests/test_splitters.py tests/test_ingest.py -v
|
||
```
|
||
|
||
- [ ] **Step 5: Commit**
|
||
|
||
```bash
|
||
git add src/core/embedder.py src/core/splitters/pdf.py src/core/splitters/epub.py src/core/ingest.py
|
||
git commit -m "refactor: 修复 Embedder Protocol 标准写法和 PDF/EPUB Splitter 参数语义"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 2: DocxSplitter — .docx 文档支持
|
||
|
||
**Files:**
|
||
|
||
- Create: `src/core/splitters/docx.py`
|
||
- Create: `tests/test_splitters_docx.py`
|
||
- Modify: `src/core/splitters/registry.py`
|
||
- Modify: `pyproject.toml`
|
||
|
||
- [ ] **Step 1: 添加 docx 可选依赖**
|
||
|
||
在 `pyproject.toml` 的 `[project.optional-dependencies]` 中添加：
|
||
|
||
```toml
|
||
docx = ["markitdown>=0.1.0"]
|
||
```
|
||
|
||
并将 `all` 更新为包含 docx：
|
||
|
||
```toml
|
||
all = ["md-vector-db[pdf,html,epub,docx]", "requests>=2.31.0", "openai>=1.0.0"]
|
||
```
|
||
|
||
- [ ] **Step 2: 编写 DocxSplitter 测试**
|
||
|
||
```python
|
||
# tests/test_splitters_docx.py
|
||
"""DocxSplitter 测试."""
|
||
import pytest
|
||
|
||
# 如果未安装 markitdown，跳过所有测试
|
||
pytest.importorskip("markitdown", reason="需要 markitdown 库")
|
||
|
||
from src.core.splitters.docx import DocxSplitter
|
||
|
||
|
||
def test_docx_splitter_creates():
|
||
"""创建 DocxSplitter 实例."""
|
||
s = DocxSplitter(max_size=500, overlap=50)
|
||
assert s is not None
|
||
|
||
|
||
def test_docx_splitter_empty_text():
|
||
"""空纯文本返回空列表."""
|
||
s = DocxSplitter()
|
||
result = s.split(" ", source_file="empty.docx")
|
||
assert result == []
|
||
|
||
|
||
def test_docx_splitter_basic_text():
|
||
"""基本文本文档分块."""
|
||
s = DocxSplitter(max_size=200, overlap=20)
|
||
text = "段落A。\n\n段落B。\n\n段落C。"
|
||
result = s.split(text, source_file="test.docx")
|
||
assert len(result) >= 1
|
||
assert all("content" in r for r in result)
|
||
assert all(r["source_file"] == "test.docx" for r in result)
|
||
|
||
|
||
def test_docx_splitter_long_text():
|
||
"""长文本分多块."""
|
||
s = DocxSplitter(max_size=100, overlap=10)
|
||
text = "这是一段非常长的文本。\n\n" * 50
|
||
result = s.split(text, source_file="long.docx")
|
||
assert len(result) >= 5
|
||
|
||
|
||
def test_docx_splitter_source_file():
|
||
"""source_file 正确传递到每个 chunk."""
|
||
s = DocxSplitter(max_size=500, overlap=50)
|
||
result = s.split("测试内容。", source_file="myfile.docx")
|
||
assert all(r["source_file"] == "myfile.docx" for r in result)
|
||
|
||
|
||
def test_docx_splitter_chunk_index():
|
||
"""chunk_index 从 0 递增."""
|
||
s = DocxSplitter(max_size=100, overlap=10)
|
||
text = "chunk A。\n\n" * 20
|
||
result = s.split(text, source_file="index.docx")
|
||
indices = [r["chunk_index"] for r in result]
|
||
assert indices == list(range(len(result)))
|
||
```
|
||
|
||
- [ ] **Step 3: 运行测试确认失败**
|
||
|
||
```bash
|
||
uv sync --extra docx
|
||
uv run pytest tests/test_splitters_docx.py -v
|
||
```
|
||
|
||
- [ ] **Step 4: 实现 DocxSplitter**
|
||
|
||
```python
|
||
# src/core/splitters/docx.py
|
||
""".docx Word 文档分块器 — 使用 markitdown 提取文字后委托 TextSplitter."""
|
||
from __future__ import annotations
|
||
|
||
import logging
|
||
from pathlib import Path
|
||
|
||
from src.core.splitters.text import TextSplitter
|
||
|
||
logger = logging.getLogger("md-vector-db")
|
||
|
||
|
||
class DocxSplitter:
|
||
"""Docx 分块器：markitdown 提取文字 → TextSplitter 分块.
|
||
|
||
实现 Splitter Protocol，内部组合 TextSplitter 实例。
|
||
split() 的 source 参数可接收文件路径或纯文本。
|
||
"""
|
||
|
||
def __init__(self, max_size: int = 1000, overlap: int = 100):
|
||
self._text_splitter = TextSplitter(max_size=max_size, overlap=overlap)
|
||
|
||
def split(self, source: str, source_file: str = "") -> list[dict]:
|
||
"""从 .docx 文件或纯文本提取文字并分块.
|
||
|
||
Args:
|
||
source: .docx 文件路径或纯文本内容
|
||
source_file: 来源文件名
|
||
|
||
Returns:
|
||
分块后的 chunk 列表
|
||
"""
|
||
path = Path(source)
|
||
if path.suffix.lower() == ".docx":
|
||
text = self._extract_from_docx(str(path))
|
||
else:
|
||
text = source
|
||
|
||
if not text.strip():
|
||
return []
|
||
|
||
return self._text_splitter.split(text, source_file=source_file)
|
||
|
||
def _extract_from_docx(self, file_path: str) -> str:
|
||
"""使用 markitdown 从 .docx 文件中提取文字."""
|
||
try:
|
||
from markitdown import MarkItDown
|
||
except ImportError:
|
||
raise ImportError(
|
||
"Docx 支持需要 markitdown 库. 请执行: uv sync --extra docx"
|
||
)
|
||
|
||
try:
|
||
md = MarkItDown()
|
||
result = md.convert(file_path)
|
||
return result.text_content
|
||
except Exception as e:
|
||
logger.error("Docx 解析失败: %s — %s", file_path, e)
|
||
raise ValueError(f"Docx 解析失败: {e}") from e
|
||
```
|
||
|
||
- [ ] **Step 5: 注册 .docx 到 registry**
|
||
|
||
修改 `src/core/splitters/registry.py`：
|
||
|
||
```python
|
||
# 在 _DEFAULT_MAP 字典中添加：
|
||
_DEFAULT_MAP: dict[str, str] = {
|
||
".md": "markdown",
|
||
".markdown": "markdown",
|
||
".txt": "text",
|
||
".pdf": "pdf",
|
||
".html": "html",
|
||
".htm": "html",
|
||
".epub": "epub",
|
||
".docx": "docx", # 新增
|
||
}
|
||
|
||
# 在 SUPPORTED_SUFFIXES 行后确保 frozenset 自动包含
|
||
|
||
# 在 get_splitter 函数中添加 docx 分支：
|
||
if kind == "docx":
|
||
from src.core.splitters.docx import DocxSplitter
|
||
return DocxSplitter(max_size=max_size, overlap=overlap)
|
||
```
|
||
|
||
- [ ] **Step 6: 运行测试**
|
||
|
||
```bash
|
||
uv run pytest tests/test_splitters_docx.py -v
|
||
```
|
||
|
||
预期: 6 passed
|
||
|
||
- [ ] **Step 7: Commit**
|
||
|
||
```bash
|
||
git add src/core/splitters/docx.py tests/test_splitters_docx.py src/core/splitters/registry.py pyproject.toml
|
||
git commit -m "feat: 新增 DocxSplitter — 支持 .docx 文档入库"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 3: 数据导出功能
|
||
|
||
**Files:**
|
||
|
||
- Create: `tests/test_export.py`
|
||
- Modify: `src/core/search.py`
|
||
- Modify: `src/cli/main.py`
|
||
|
||
- [ ] **Step 1: 编写导出功能测试**
|
||
|
||
```python
|
||
# tests/test_export.py
|
||
"""导出功能测试."""
|
||
import json
|
||
import csv
|
||
import io
|
||
import pytest
|
||
from src.core.search import Searcher
|
||
|
||
|
||
class FakeExportCollection:
|
||
def count(self):
|
||
return 2
|
||
|
||
def get(self, include=None):
|
||
return {
|
||
"ids": ["doc_0", "doc_1"],
|
||
"documents": ["内容A。\n\n段落B。", "内容C。"],
|
||
"metadatas": [
|
||
{"source_file": "a.md", "section_title": "标题A", "heading_level": 1, "chunk_index": 0},
|
||
{"source_file": "b.md", "section_title": "", "heading_level": 0, "chunk_index": 0},
|
||
],
|
||
}
|
||
|
||
|
||
class FakeExportDB:
|
||
def get_or_create_collection(self, name):
|
||
return FakeExportCollection()
|
||
|
||
def list_collections(self):
|
||
return [] # 简化
|
||
|
||
|
||
class FakeExportEmbedder:
|
||
@property
|
||
def dimension(self):
|
||
return 4
|
||
|
||
def embed(self, texts):
|
||
return [[0.1, 0.2, 0.3, 0.4] for _ in texts]
|
||
|
||
|
||
@pytest.fixture
|
||
def searcher():
|
||
db = FakeExportDB()
|
||
embedder = FakeExportEmbedder()
|
||
return Searcher(db, embedder, "test_export")
|
||
|
||
|
||
def test_export_json_returns_valid_json(searcher):
|
||
"""export_json 返回合法的 JSON 字符串."""
|
||
output = searcher.export_json()
|
||
data = json.loads(output)
|
||
assert isinstance(data, list)
|
||
assert len(data) == 2
|
||
|
||
|
||
def test_export_json_contains_all_fields(searcher):
|
||
"""导出包含所有必要字段."""
|
||
output = searcher.export_json()
|
||
data = json.loads(output)
|
||
first = data[0]
|
||
assert "id" in first
|
||
assert "content" in first
|
||
assert "source_file" in first
|
||
assert "section_title" in first
|
||
assert "heading_level" in first
|
||
|
||
|
||
def test_export_csv_returns_valid_csv(searcher):
|
||
"""export_csv 返回合法的 CSV 字符串."""
|
||
output = searcher.export_csv()
|
||
reader = csv.DictReader(io.StringIO(output))
|
||
rows = list(reader)
|
||
assert len(rows) == 2
|
||
|
||
|
||
def test_export_csv_has_header(searcher):
|
||
"""CSV 包含表头."""
|
||
output = searcher.export_csv()
|
||
reader = csv.DictReader(io.StringIO(output))
|
||
assert reader.fieldnames is not None
|
||
assert "content" in reader.fieldnames
|
||
assert "source_file" in reader.fieldnames
|
||
|
||
|
||
def test_export_empty_collection(searcher):
|
||
"""空 collection 导出空列表/空 CSV（仅有表头）."""
|
||
|
||
class EmptyCollection:
|
||
def count(self):
|
||
return 0
|
||
def get(self, include=None):
|
||
return {"ids": [], "documents": [], "metadatas": []}
|
||
|
||
class EmptyDB:
|
||
def get_or_create_collection(self, name):
|
||
return EmptyCollection()
|
||
def list_collections(self):
|
||
return []
|
||
|
||
s = Searcher(EmptyDB(), FakeExportEmbedder(), "empty")
|
||
json_out = s.export_json()
|
||
assert json.loads(json_out) == []
|
||
csv_out = s.export_csv()
|
||
lines = csv_out.strip().split("\n")
|
||
assert len(lines) == 1 # 仅表头
|
||
```
|
||
|
||
- [ ] **Step 2: 运行测试确认失败**
|
||
|
||
```bash
|
||
uv run pytest tests/test_export.py -v
|
||
```
|
||
|
||
- [ ] **Step 3: 在 Searcher 中添加导出方法**
|
||
|
||
在 `src/core/search.py` 的 `Searcher` 类中添加两个方法：
|
||
|
||
```python
|
||
import csv
|
||
import io
|
||
import json as json_lib
|
||
|
||
# 在 Searcher 类中追加：
|
||
|
||
def export_json(self, file_path: str | None = None) -> str:
|
||
"""导出 collection 所有 chunks 为 JSON.
|
||
|
||
Args:
|
||
file_path: 可选，写入文件路径。不传则返回 JSON 字符串。
|
||
|
||
Returns:
|
||
JSON 字符串
|
||
"""
|
||
all_data = self.collection.get(include=["documents", "metadatas"])
|
||
records = []
|
||
if all_data and all_data["ids"]:
|
||
for i, doc_id in enumerate(all_data["ids"]):
|
||
meta = all_data["metadatas"][i] if all_data["metadatas"] else {}
|
||
records.append({
|
||
"id": doc_id,
|
||
"content": all_data["documents"][i] if all_data["documents"] else "",
|
||
"source_file": meta.get("source_file", ""),
|
||
"section_title": meta.get("section_title", ""),
|
||
"heading_level": meta.get("heading_level", 0),
|
||
"chunk_index": meta.get("chunk_index", i),
|
||
})
|
||
json_str = json_lib.dumps(records, ensure_ascii=False, indent=2)
|
||
if file_path:
|
||
with open(file_path, "w", encoding="utf-8") as f:
|
||
f.write(json_str)
|
||
return json_str
|
||
|
||
|
||
def export_csv(self, file_path: str | None = None) -> str:
|
||
"""导出 collection 所有 chunks 为 CSV.
|
||
|
||
Args:
|
||
file_path: 可选，写入文件路径。不传则返回 CSV 字符串。
|
||
|
||
Returns:
|
||
CSV 字符串
|
||
"""
|
||
all_data = self.collection.get(include=["documents", "metadatas"])
|
||
output = io.StringIO()
|
||
writer = csv.writer(output)
|
||
writer.writerow(["id", "content", "source_file", "section_title", "heading_level", "chunk_index"])
|
||
if all_data and all_data["ids"]:
|
||
for i, doc_id in enumerate(all_data["ids"]):
|
||
meta = all_data["metadatas"][i] if all_data["metadatas"] else {}
|
||
writer.writerow([
|
||
doc_id,
|
||
all_data["documents"][i] if all_data["documents"] else "",
|
||
meta.get("source_file", ""),
|
||
meta.get("section_title", ""),
|
||
meta.get("heading_level", 0),
|
||
meta.get("chunk_index", i),
|
||
])
|
||
csv_str = output.getvalue()
|
||
if file_path:
|
||
with open(file_path, "w", encoding="utf-8", newline="") as f:
|
||
f.write(csv_str)
|
||
return csv_str
|
||
```
|
||
|
||
- [ ] **Step 4: 运行导出测试**
|
||
|
||
```bash
|
||
uv run pytest tests/test_export.py -v
|
||
```
|
||
|
||
- [ ] **Step 5: 在 CLI 添加 export 命令**
|
||
|
||
在 `src/cli/main.py` 中添加：
|
||
|
||
```python
|
||
@app.command(help="导出 collection 数据为 JSON 或 CSV.")
|
||
def export(
|
||
output: Annotated[str, typer.Option("--output", "-o", help="输出文件路径")],
|
||
fmt: Annotated[str, typer.Option("--format", "-f", help="导出格式: json | csv")] = "json",
|
||
config: ConfigOpt = DEFAULT_CONFIG_PATH,
|
||
collection: CollectionOpt = None,
|
||
):
|
||
"""导出 collection 数据."""
|
||
_init_config(config)
|
||
state = get_state()
|
||
searcher = state.get_searcher(_resolve_collection(collection))
|
||
|
||
if fmt == "json":
|
||
searcher.export_json(file_path=output)
|
||
elif fmt == "csv":
|
||
searcher.export_csv(file_path=output)
|
||
else:
|
||
typer.echo(f"错误: 不支持的格式 '{fmt}'，可选: json, csv", err=True)
|
||
raise typer.Exit(code=1)
|
||
|
||
typer.echo(f"[OK] 已导出到: {output}")
|
||
```
|
||
|
||
- [ ] **Step 6: 测试 CLI export 命令**
|
||
|
||
```bash
|
||
uv run md-vector-db export --help
|
||
uv run md-vector-db export -o /tmp/test_export.json -f json
|
||
```
|
||
|
||
- [ ] **Step 7: Commit**
|
||
|
||
```bash
|
||
git add src/core/search.py src/cli/main.py tests/test_export.py
|
||
git commit -m "feat: 新增数据导出功能（JSON/CSV）和 CLI export 命令"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 4: Web 管理界面
|
||
|
||
**Files:**
|
||
|
||
- Create: `src/web/index.html`
|
||
- Modify: `src/server/app.py`
|
||
|
||
- [ ] **Step 1: 创建 Web 管理界面（单文件 Vue 3 SPA）**
|
||
|
||
```html
|
||
<!-- src/web/index.html -->
|
||
<!DOCTYPE html>
|
||
<html lang="zh-CN">
|
||
<head>
|
||
<meta charset="UTF-8">
|
||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||
<title>md-vector-db 管理面板</title>
|
||
<script src="https://unpkg.com/vue@3/dist/vue.global.prod.js"></script>
|
||
<style>
|
||
:root {
|
||
--bg: #f8f9fa;
|
||
--card-bg: #ffffff;
|
||
--text: #212529;
|
||
--muted: #6c757d;
|
||
--border: #dee2e6;
|
||
--primary: #0d6efd;
|
||
--success: #198754;
|
||
--danger: #dc3545;
|
||
}
|
||
* { margin: 0; padding: 0; box-sizing: border-box; }
|
||
body {
|
||
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif;
|
||
background: var(--bg);
|
||
color: var(--text);
|
||
line-height: 1.6;
|
||
}
|
||
.container { max-width: 960px; margin: 0 auto; padding: 24px 16px; }
|
||
h1 { font-size: 1.5rem; margin-bottom: 24px; }
|
||
.card {
|
||
background: var(--card-bg);
|
||
border: 1px solid var(--border);
|
||
border-radius: 8px;
|
||
padding: 20px;
|
||
margin-bottom: 16px;
|
||
}
|
||
.card h2 { font-size: 1.1rem; margin-bottom: 12px; }
|
||
.form-group { margin-bottom: 12px; }
|
||
label { display: block; font-size: .875rem; color: var(--muted); margin-bottom: 4px; }
|
||
input, textarea, select {
|
||
width: 100%;
|
||
padding: 8px 12px;
|
||
border: 1px solid var(--border);
|
||
border-radius: 4px;
|
||
font-size: .9rem;
|
||
font-family: inherit;
|
||
}
|
||
textarea { min-height: 100px; resize: vertical; }
|
||
.btn {
|
||
display: inline-block;
|
||
padding: 8px 16px;
|
||
border: none;
|
||
border-radius: 4px;
|
||
font-size: .875rem;
|
||
cursor: pointer;
|
||
text-decoration: none;
|
||
}
|
||
.btn-primary { background: var(--primary); color: #fff; }
|
||
.btn-danger { background: var(--danger); color: #fff; }
|
||
.btn-sm { padding: 4px 10px; font-size: .8rem; }
|
||
.result-item {
|
||
border: 1px solid var(--border);
|
||
border-radius: 6px;
|
||
padding: 12px;
|
||
margin-bottom: 8px;
|
||
}
|
||
.result-item .score { font-weight: 600; color: var(--primary); }
|
||
.result-item .source { font-size: .8rem; color: var(--muted); }
|
||
.result-item .section { font-size: .85rem; color: var(--success); margin-bottom: 4px; }
|
||
.badge {
|
||
display: inline-block;
|
||
padding: 2px 8px;
|
||
border-radius: 12px;
|
||
font-size: .75rem;
|
||
font-weight: 600;
|
||
}
|
||
.badge-ok { background: #d1e7dd; color: #0f5132; }
|
||
.badge-err { background: #f8d7da; color: #842029; }
|
||
.tabs { display: flex; gap: 4px; margin-bottom: 16px; }
|
||
.tab {
|
||
padding: 8px 16px;
|
||
border: 1px solid var(--border);
|
||
border-radius: 6px 6px 0 0;
|
||
background: var(--bg);
|
||
cursor: pointer;
|
||
font-size: .875rem;
|
||
}
|
||
.tab.active { background: var(--card-bg); border-bottom-color: var(--card-bg); font-weight: 600; }
|
||
.toast {
|
||
position: fixed; top: 16px; right: 16px;
|
||
padding: 12px 20px; border-radius: 6px; color: #fff;
|
||
font-size: .875rem; z-index: 999;
|
||
}
|
||
.toast-success { background: var(--success); }
|
||
.toast-error { background: var(--danger); }
|
||
</style>
|
||
</head>
|
||
<body>
|
||
<div id="app">
|
||
<div class="container">
|
||
<h1>📚 md-vector-db 管理面板</h1>
|
||
|
||
<!-- 标签页切换 -->
|
||
<div class="tabs">
|
||
<div :class="['tab', { active: activeTab === 'search' }]" @click="activeTab = 'search'">🔍 搜索</div>
|
||
<div :class="['tab', { active: activeTab === 'ingest' }]" @click="activeTab = 'ingest'">📥 入库</div>
|
||
<div :class="['tab', { active: activeTab === 'collections' }]" @click="activeTab = 'collections'">📂 集合</div>
|
||
</div>
|
||
|
||
<!-- Toast 通知 -->
|
||
<div v-if="toast" :class="['toast', 'toast-' + toast.type]">{{ toast.message }}</div>
|
||
|
||
<!-- 搜索页面 -->
|
||
<div v-if="activeTab === 'search'" class="card">
|
||
<h2>语义检索</h2>
|
||
<div class="form-group">
|
||
<label>搜索查询</label>
|
||
<input v-model="searchQuery" @keyup.enter="doSearch" placeholder="输入关键词或自然语言...">
|
||
</div>
|
||
<div style="display:flex;gap:12px;align-items:end;">
|
||
<div class="form-group" style="flex:1">
|
||
<label>返回条数</label>
|
||
<input v-model.number="topK" type="number" min="1" max="100">
|
||
</div>
|
||
<div class="form-group" style="flex:1">
|
||
<label>集合名（留空=默认）</label>
|
||
<input v-model="collectionName" placeholder="default">
|
||
</div>
|
||
</div>
|
||
<button class="btn btn-primary" @click="doSearch" :disabled="searching">
|
||
{{ searching ? '搜索中...' : '搜索' }}
|
||
</button>
|
||
|
||
<!-- 搜索结果 -->
|
||
<div v-if="searchResults.length > 0" style="margin-top:16px">
|
||
<h3>{{ searchResults.length }} 条结果</h3>
|
||
<div v-for="(r, i) in searchResults" :key="i" class="result-item">
|
||
<div class="score">相似度: {{ r.score?.toFixed(4) || r.score }}</div>
|
||
<div class="source">📄 {{ r.source_file }}</div>
|
||
<div v-if="r.section_title" class="section">📑 {{ r.section_title }}</div>
|
||
<pre style="white-space:pre-wrap;font-size:.85rem;margin-top:8px;">{{ r.content?.substring(0, 300) }}{{ r.content?.length > 300 ? '...' : '' }}</pre>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
|
||
<!-- 入库页面 -->
|
||
<div v-if="activeTab === 'ingest'" class="card">
|
||
<h2>入库文档</h2>
|
||
<div class="form-group">
|
||
<label>Markdown 内容</label>
|
||
<textarea v-model="ingestContent" placeholder="粘贴 Markdown 文本..."></textarea>
|
||
</div>
|
||
<div class="form-group">
|
||
<label>文件名</label>
|
||
<input v-model="ingestFileName" placeholder="document.md">
|
||
</div>
|
||
<div class="form-group">
|
||
<label>目标集合（留空=默认）</label>
|
||
<input v-model="ingestCollection" placeholder="default">
|
||
</div>
|
||
<button class="btn btn-primary" @click="doIngest" :disabled="ingesting">
|
||
{{ ingesting ? '入库中...' : '入库' }}
|
||
</button>
|
||
<p v-if="ingestResult" style="margin-top:12px">{{ ingestResult }}</p>
|
||
</div>
|
||
|
||
<!-- 集合页面 -->
|
||
<div v-if="activeTab === 'collections'" class="card">
|
||
<h2>集合列表</h2>
|
||
<button class="btn btn-primary btn-sm" @click="loadCollections" :disabled="loadingColl">
|
||
{{ loadingColl ? '加载中...' : '刷新' }}
|
||
</button>
|
||
<div v-if="collections.length > 0" style="margin-top:12px">
|
||
<table style="width:100%;border-collapse:collapse">
|
||
<thead>
|
||
<tr style="border-bottom:2px solid var(--border)">
|
||
<th style="text-align:left;padding:8px">名称</th>
|
||
<th style="text-align:right;padding:8px">Chunks</th>
|
||
</tr>
|
||
</thead>
|
||
<tbody>
|
||
<tr v-for="c in collections" :key="c.name" style="border-bottom:1px solid var(--border)">
|
||
<td style="padding:8px">{{ c.name }}</td>
|
||
<td style="text-align:right;padding:8px">{{ c.count }}</td>
|
||
</tr>
|
||
</tbody>
|
||
</table>
|
||
</div>
|
||
</div>
|
||
|
||
<!-- 健康状态 -->
|
||
<div class="card" v-if="health">
|
||
<h2>服务状态
|
||
<span :class="['badge', health.status === 'ok' ? 'badge-ok' : 'badge-err']">{{ health.status }}</span>
|
||
</h2>
|
||
<pre style="font-size:.8rem">{{ JSON.stringify(health.checks, null, 2) }}</pre>
|
||
</div>
|
||
</div>
|
||
</div>
|
||
|
||
<script>
|
||
const { createApp } = Vue
|
||
|
||
createApp({
|
||
data() {
|
||
return {
|
||
activeTab: 'search',
|
||
searchQuery: '',
|
||
topK: 10,
|
||
collectionName: '',
|
||
searchResults: [],
|
||
searching: false,
|
||
ingestContent: '',
|
||
ingestFileName: '',
|
||
ingestCollection: '',
|
||
ingesting: false,
|
||
ingestResult: null,
|
||
collections: [],
|
||
loadingColl: false,
|
||
health: null,
|
||
toast: null,
|
||
apiBase: window.location.origin,
|
||
}
|
||
},
|
||
mounted() {
|
||
this.checkHealth()
|
||
},
|
||
methods: {
|
||
async api(method, path, body) {
|
||
const headers = { 'Content-Type': 'application/json' }
|
||
const res = await fetch(`${this.apiBase}${path}`, { method, headers, body: body ? JSON.stringify(body) : undefined })
|
||
if (!res.ok) {
|
||
const err = await res.text()
|
||
throw new Error(`HTTP ${res.status}: ${err}`)
|
||
}
|
||
return res.json()
|
||
},
|
||
showToast(message, type = 'success') {
|
||
this.toast = { message, type }
|
||
setTimeout(() => { this.toast = null }, 3000)
|
||
},
|
||
async checkHealth() {
|
||
try {
|
||
this.health = await this.api('GET', '/api/v1/health')
|
||
} catch (e) {
|
||
this.health = { status: 'error', checks: {} }
|
||
}
|
||
},
|
||
async doSearch() {
|
||
if (!this.searchQuery.trim()) return
|
||
this.searching = true
|
||
this.searchResults = []
|
||
try {
|
||
const body = { query: this.searchQuery, top_k: this.topK }
|
||
if (this.collectionName) body.collection = this.collectionName
|
||
const data = await this.api('POST', '/api/v1/search', body)
|
||
this.searchResults = data.results || []
|
||
} catch (e) {
|
||
this.showToast(`搜索失败: ${e.message}`, 'error')
|
||
} finally {
|
||
this.searching = false
|
||
}
|
||
},
|
||
async doIngest() {
|
||
if (!this.ingestContent.trim()) return
|
||
this.ingesting = true
|
||
this.ingestResult = null
|
||
try {
|
||
const body = { content: this.ingestContent, file_name: this.ingestFileName || 'untitled.md' }
|
||
if (this.ingestCollection) body.collection = this.ingestCollection
|
||
const data = await this.api('POST', '/api/v1/ingest', body)
|
||
this.ingestResult = `✅ 入库成功: ${data.chunks} chunks → ${data.collection}`
|
||
this.showToast(this.ingestResult)
|
||
} catch (e) {
|
||
this.ingestResult = `❌ 入库失败: ${e.message}`
|
||
this.showToast(this.ingestResult, 'error')
|
||
} finally {
|
||
this.ingesting = false
|
||
}
|
||
},
|
||
async loadCollections() {
|
||
this.loadingColl = true
|
||
try {
|
||
const data = await this.api('GET', '/api/v1/collections')
|
||
this.collections = data.collections || []
|
||
} catch (e) {
|
||
this.showToast(`加载失败: ${e.message}`, 'error')
|
||
} finally {
|
||
this.loadingColl = false
|
||
}
|
||
},
|
||
}
|
||
}).mount('#app')
|
||
</script>
|
||
</body>
|
||
</html>
|
||
```
|
||
|
||
- [ ] **Step 2: 在 FastAPI 中挂载静态文件**
|
||
|
||
修改 `src/server/app.py`，在 `app` 创建后添加：
|
||
|
||
```python
|
||
from fastapi.staticfiles import StaticFiles
|
||
from pathlib import Path
|
||
|
||
# 在 app = FastAPI(...) 之后，中间件之前添加：
|
||
web_dir = Path(__file__).parent.parent / "web"
|
||
if web_dir.exists():
|
||
app.mount("/admin", StaticFiles(directory=str(web_dir), html=True), name="admin")
|
||
```
|
||
|
||
- [ ] **Step 3: 添加 API 测试**
|
||
|
||
在 `tests/test_api.py` 中添加：
|
||
|
||
```python
|
||
def test_admin_ui_served(client):
|
||
"""管理界面可访问."""
|
||
response = client.get("/admin")
|
||
assert response.status_code in (200, 404) # 404 如果 web 目录不存在（可接受）
|
||
|
||
|
||
def test_health_skips_rate_limit(client):
|
||
"""健康检查不触发速率限制."""
|
||
for _ in range(5):
|
||
resp = client.get("/api/v1/health")
|
||
assert resp.status_code == 200
|
||
```
|
||
|
||
- [ ] **Step 4: 测试 Web UI**
|
||
|
||
```bash
|
||
uv run md-vector-db serve --port 8000 &
|
||
sleep 3
|
||
# 访问管理界面
|
||
curl -s http://localhost:8000/admin | head -5
|
||
# 预期：返回 HTML 内容
|
||
kill %1
|
||
```
|
||
|
||
- [ ] **Step 5: Commit**
|
||
|
||
```bash
|
||
git add src/web/index.html src/server/app.py tests/test_api.py
|
||
git commit -m "feat: 添加 Web 管理界面（Vue 3 SPA 单文件）"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 5: 测试覆盖率提升到 80%+
|
||
|
||
**Files:**
|
||
|
||
- Modify: `tests/test_splitters_pdf.py`
|
||
- Modify: `tests/test_splitters_html.py`
|
||
- Modify: `tests/test_splitters_epub.py`
|
||
- Modify: `tests/test_cli.py`
|
||
- Modify: `tests/test_db.py`
|
||
- Modify: `tests/test_search.py`
|
||
- Modify: `tests/test_ingest.py`
|
||
|
||
- [ ] **Step 1: 补充 PDF splitter 测试**
|
||
|
||
```python
|
||
# 在 tests/test_splitters_pdf.py 追加（保留原有内容）：
|
||
|
||
def test_pdf_splitter_creation():
|
||
"""PDFSplitter 可以创建."""
|
||
from src.core.splitters.pdf import PDFSplitter
|
||
s = PDFSplitter(max_size=500, overlap=50)
|
||
assert s is not None
|
||
assert s._text_splitter.max_size == 500
|
||
assert s._text_splitter.overlap == 50
|
||
|
||
|
||
def test_pdf_splitter_empty_result():
|
||
"""提取结果为空时返回空列表."""
|
||
from src.core.splitters.pdf import PDFSplitter
|
||
s = PDFSplitter()
|
||
result = s.split(source="/nonexistent/file.pdf", source_file="test.pdf")
|
||
# 应该 raise ValueError（文件不存在）
|
||
import pytest
|
||
with pytest.raises(ValueError):
|
||
s.split(source="/nonexistent/file.pdf", source_file="test.pdf")
|
||
```
|
||
|
||
- [ ] **Step 2: 补充 HTML splitter 测试**
|
||
|
||
```python
|
||
# 在 tests/test_splitters_html.py 追加：
|
||
|
||
def test_html_splitter_creation():
|
||
"""HTMLSplitter 创建正常."""
|
||
from src.core.splitters.html import HTMLSplitter
|
||
s = HTMLSplitter(max_size=500, overlap=50)
|
||
assert s is not None
|
||
|
||
|
||
def test_html_splitter_basic():
|
||
"""基本 HTML 分块."""
|
||
from src.core.splitters.html import HTMLSplitter
|
||
s = HTMLSplitter(max_size=200, overlap=20)
|
||
html = "<html><body><p>段落A。</p><p>段落B。</p></body></html>"
|
||
result = s.split(html, source_file="test.html")
|
||
assert len(result) >= 1
|
||
assert "段落A" in result[0]["content"]
|
||
|
||
|
||
def test_html_splitter_strips_script_style():
|
||
"""去除 script 和 style 标签."""
|
||
from src.core.splitters.html import HTMLSplitter
|
||
s = HTMLSplitter(max_size=500, overlap=50)
|
||
html = "<html><script>alert('xss')</script><body><p>可见内容。</p></body></html>"
|
||
result = s.split(html, source_file="test.html")
|
||
assert "alert" not in result[0]["content"]
|
||
assert "可见内容" in result[0]["content"]
|
||
|
||
|
||
def test_html_splitter_empty():
|
||
"""空 HTML 返回空列表."""
|
||
from src.core.splitters.html import HTMLSplitter
|
||
s = HTMLSplitter()
|
||
result = s.split("<html></html>", source_file="empty.html")
|
||
assert result == []
|
||
```
|
||
|
||
- [ ] **Step 3: 补充 EPUB splitter 测试**
|
||
|
||
```python
|
||
# 在 tests/test_splitters_epub.py 追加：
|
||
|
||
def test_epub_splitter_creation():
|
||
"""EPUBSplitter 创建正常."""
|
||
from src.core.splitters.epub import EPUBSplitter
|
||
s = EPUBSplitter(max_size=500, overlap=50)
|
||
assert s is not None
|
||
|
||
|
||
def test_epub_splitter_file_not_found():
|
||
"""不存在的文件抛出 ValueError."""
|
||
from src.core.splitters.epub import EPUBSplitter
|
||
s = EPUBSplitter()
|
||
import pytest
|
||
with pytest.raises(ValueError):
|
||
s.split(source="/nonexistent/file.epub", source_file="test.epub")
|
||
```
|
||
|
||
- [ ] **Step 4: 补充 CLI 测试**
|
||
|
||
```python
|
||
# 在 tests/test_cli.py 追加：
|
||
|
||
def test_ingest_help(cli_app):
|
||
"""ingest --help 正常."""
|
||
result = cli_app(["ingest", "--help"])
|
||
assert result.exit_code == 0
|
||
|
||
|
||
def test_search_no_results(cli_app, tmp_path):
|
||
"""空 collection 搜索返回提示."""
|
||
result = cli_app(["search", "测试查询", "-c", str(tmp_path / "cfg.yaml")])
|
||
# 可能返回 0（只是无结果）或出错
|
||
assert result.exit_code in (0, 1)
|
||
|
||
|
||
def test_stats_empty(cli_app, tmp_path):
|
||
"""空 collection 的 stats."""
|
||
result = cli_app(["stats", "-c", str(tmp_path / "cfg.yaml")])
|
||
assert result.exit_code == 0
|
||
|
||
|
||
def test_stats_json(cli_app, tmp_path):
|
||
"""stats --json 输出."""
|
||
result = cli_app(["stats", "--json", "-c", str(tmp_path / "cfg.yaml")])
|
||
assert result.exit_code == 0
|
||
```
|
||
|
||
- [ ] **Step 5: 补充 DB 测试**
|
||
|
||
```python
|
||
# 在 tests/test_db.py 追加：
|
||
|
||
def test_write_guard_context_manager(db):
|
||
"""write_guard 上下文管理器."""
|
||
with db.write_guard():
|
||
pass # 应正常获取和释放锁
|
||
|
||
|
||
def test_close(db):
|
||
"""close 正常执行."""
|
||
db.close()
|
||
# close 后不应对 client 做任何操作，测试仅验证不抛异常
|
||
|
||
|
||
def test_delete_collection_nonexistent(db):
|
||
"""删除不存在的 collection 不抛异常."""
|
||
db.delete_collection("nonexistent-collection-12345")
|
||
# 应静默处理
|
||
|
||
|
||
def test_delete_by_source_no_match(db):
|
||
"""删除不存在的 source 返回 False."""
|
||
result = db.delete_by_source("test_col", "no-such-file.md")
|
||
assert result is False
|
||
```
|
||
|
||
- [ ] **Step 6: 补充搜索测试**
|
||
|
||
```python
|
||
# 在 tests/test_search.py 追加：
|
||
|
||
def test_list_sources_empty():
|
||
"""空 collection 的 list_sources 返回空列表."""
|
||
from src.core.search import Searcher
|
||
|
||
class EmptyColl:
|
||
def count(self): return 0
|
||
def get(self, **kwargs): return {"ids": [], "documents": [], "metadatas": []}
|
||
|
||
class EmptyDB:
|
||
def get_or_create_collection(self, name): return EmptyColl()
|
||
def list_collections(self): return []
|
||
|
||
class FakeEmb:
|
||
@property
|
||
def dimension(self): return 4
|
||
def embed(self, texts): return [[0.0]*4]
|
||
|
||
s = Searcher(EmptyDB(), FakeEmb(), "empty")
|
||
assert s.list_sources() == []
|
||
```
|
||
|
||
- [ ] **Step 7: 补充 ingest 测试**
|
||
|
||
```python
|
||
# 在 tests/test_ingest.py 追加：
|
||
|
||
def test_ingest_file_returns_zero_for_dir(db, local_embedder, tmp_path: Path):
|
||
"""目录路径应被 ingest_directory 而非 ingest_file 处理."""
|
||
ingestor = DocumentIngestor(db, local_embedder, "test_ingest_dir")
|
||
d = tmp_path / "subdir"
|
||
d.mkdir()
|
||
# ingest_file 不应处理目录
|
||
result = ingestor.ingest_file(str(d))
|
||
assert result == 0 # 目录不是文件，跳过
|
||
|
||
|
||
def test_ingest_file_markdown(db, local_embedder, tmp_path: Path):
|
||
"""MD 文件入库返回正确的 chunk 数."""
|
||
file = tmp_path / "hello.md"
|
||
file.write_text("# 标题\n\n内容段落。", encoding="utf-8")
|
||
ingestor = DocumentIngestor(
|
||
db, local_embedder, "test_md",
|
||
chunk_config=ChunkConfig(max_size=1000, overlap=100),
|
||
)
|
||
count = ingestor.ingest_file(str(file))
|
||
assert count >= 1
|
||
|
||
|
||
def test_ingest_content_default_splitter(db, local_embedder):
|
||
"""未指定 splitter 时用 MarkdownSplitter."""
|
||
ingestor = DocumentIngestor(db, local_embedder, "test_content")
|
||
count = ingestor.ingest_content("# 测试\n\n一些内容。", "test.md")
|
||
assert count >= 1
|
||
|
||
|
||
def test_ingest_directory_recursive(db, local_embedder, tmp_path: Path):
|
||
"""ingest_directory 递归处理子目录."""
|
||
(tmp_path / "sub").mkdir()
|
||
(tmp_path / "a.md").write_text("# A\n\n内容A。", encoding="utf-8")
|
||
(tmp_path / "sub" / "b.md").write_text("# B\n\n内容B。", encoding="utf-8")
|
||
ingestor = DocumentIngestor(
|
||
db, local_embedder, "test_recurse",
|
||
chunk_config=ChunkConfig(max_size=1000, overlap=100),
|
||
)
|
||
results = ingestor.ingest_directory(str(tmp_path))
|
||
assert len(results) >= 2
|
||
assert all(v > 0 for v in results.values())
|
||
```
|
||
|
||
- [ ] **Step 8: 运行全部测试并验证覆盖率**
|
||
|
||
```bash
|
||
uv run pytest tests/ --cov=src --cov-report=term-missing -v
|
||
```
|
||
|
||
预期: ≥ 80% 覆盖率
|
||
|
||
- [ ] **Step 9: Commit**
|
||
|
||
```bash
|
||
git add tests/
|
||
git commit -m "test: 补充测试覆盖率至 80%+（PDF/HTML/EPUB/CLI/DB/Search/Ingest）"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 6: CORS 和安全配置修复
|
||
|
||
**Files:**
|
||
|
||
- Modify: `src/server/app.py`
|
||
|
||
- [ ] **Step 1: 修复 CORS 配置**
|
||
|
||
将 `allow_credentials` 改为 `False`：
|
||
|
||
```python
|
||
app.add_middleware(
|
||
CORSMiddleware,
|
||
allow_origins=os.environ.get("CORS_ORIGINS", "http://localhost:3000").split(","),
|
||
allow_credentials=False, # 修复：原来是 True
|
||
allow_methods=["GET", "POST", "DELETE", "OPTIONS"],
|
||
allow_headers=["Content-Type", "Authorization", "X-API-Key"],
|
||
)
|
||
```
|
||
|
||
- [ ] **Step 2: 更新 .env.example 添加 CORS 说明**
|
||
|
||
```
|
||
# CORS 允许的源列表（逗号分隔）
|
||
CORS_ORIGINS=http://localhost:3000,http://localhost:5173
|
||
```
|
||
|
||
- [ ] **Step 3: 测试 CORS 中间件**
|
||
|
||
```bash
|
||
uv run pytest tests/test_api.py -v -k "cors or health"
|
||
```
|
||
|
||
- [ ] **Step 4: Commit**
|
||
|
||
```bash
|
||
git add src/server/app.py .env.example
|
||
git commit -m "fix: 修复 CORS allow_credentials 配置，更新 .env.example"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 7: 更新 README 和文档
|
||
|
||
**Files:**
|
||
|
||
- Modify: `README.md`
|
||
|
||
- [ ] **Step 1: 更新 README 功能特性和新命令说明**
|
||
|
||
在 README 的功能特性列表追加：
|
||
|
||
```markdown
|
||
- **Web 管理界面**: 内置 Vue 3 单页管理面板，可视化搜索、入库、查看集合
|
||
- **数据导出**: 支持 JSON/CSV 导出 collection 全量数据
|
||
- **多格式扩展**: 新增 .docx 支持（通过 markitdown 库）
|
||
```
|
||
|
||
在 CLI 命令参考表中追加：
|
||
|
||
```markdown
|
||
| `export -o <文件> -f <json|csv>` | 导出 collection 数据 |
|
||
```
|
||
|
||
在 Docker 部署章节后添加：
|
||
|
||
```markdown
|
||
## Web 管理界面
|
||
|
||
启动服务后访问 `http://localhost:8000/admin` 进入管理面板：
|
||
|
||
- **搜索**: 可视化输入查询词、选择返回条数、查看相似度分数和来源文件
|
||
- **入库**: 粘贴 Markdown 文本直接入库，无需创建文件
|
||
- **集合**: 查看所有 collection 的 chunks 统计
|
||
```
|
||
|
||
- [ ] **Step 2: Commit**
|
||
|
||
```bash
|
||
git add README.md
|
||
git commit -m "docs: README 补充 Web UI、导出、docx 支持说明"
|
||
```
|
||
|
||
---
|
||
|
||
### Task 8: 最终验证
|
||
|
||
- [ ] **Step 1: 全量测试 + 覆盖率**
|
||
|
||
```bash
|
||
uv run pytest tests/ --cov=src --cov-report=term-missing -v
|
||
```
|
||
|
||
预期: 全部通过，覆盖率 ≥ 80%
|
||
|
||
- [ ] **Step 2: Lint 检查**
|
||
|
||
```bash
|
||
uv run ruff check src/ tests/
|
||
```
|
||
|
||
预期: 零错误
|
||
|
||
- [ ] **Step 3: 类型检查**
|
||
|
||
```bash
|
||
uv run mypy src/ --ignore-missing-imports
|
||
```
|
||
|
||
- [ ] **Step 4: 功能集成测试**
|
||
|
||
```bash
|
||
# 启动服务
|
||
uv run md-vector-db serve --port 8000 &
|
||
sleep 3
|
||
|
||
# 测试 .docx 入库 + 混合检索 + 导出
|
||
uv run md-vector-db ingest /path/to/sample.md
|
||
uv run md-vector-db search "测试" --mode hybrid
|
||
uv run md-vector-db export -o /tmp/export.json -f json
|
||
cat /tmp/export.json
|
||
|
||
# 验证 Web UI 可访问
|
||
curl -s http://localhost:8000/admin | grep -q "md-vector-db"
|
||
|
||
kill %1
|
||
```
|
||
|
||
- [ ] **Step 5: 总结 Commit**
|
||
|
||
```bash
|
||
git add -A
|
||
git commit -m "feat: 第二优先级完善 — Web UI、docx 支持、导出功能、覆盖率 80%+"
|
||
```
|
||
|
||
---
|
||
|
||
## 自审清单
|
||
|
||
1. **Spec 覆盖**: Web UI ✅ | .docx 支持 ✅ | 导出 ✅ | 覆盖率 80%+ ✅ | Embedder 修复 ✅ | Splitter 接口统一 ✅
|
||
2. **无占位符**: 所有代码为具体实现，无 TODO/TBD
|
||
3. **类型一致性**: DocxSplitter.split(source, ...) 与 PDF/EPUB 统一，SearchResult 复用于导出
|
||
4. **测试先行**: 每个模块先写测试再实现
|