feat: 添加 EPUBSplitter — ebooklib 提取章节文字后分块
- 新增 splitters/epub.py: ebooklib 读取 EPUB → BeautifulSoup 去标签 → TextSplitter 分块 - epub 作为可选依赖: uv sync --extra epub - registry 新增 .epub 映射, ingest_file 处理 EPUB 二进制文件 - 测试: 3 个 EPUBSplitter 测试 (含空 EPUB/Protocol 合规) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -10,6 +10,7 @@ _DEFAULT_MAP: dict[str, str] = {
|
||||
".pdf": "pdf",
|
||||
".html": "html",
|
||||
".htm": "html",
|
||||
".epub": "epub",
|
||||
}
|
||||
|
||||
# 所有支持的扩展名集合(供外部遍历文件使用)
|
||||
@@ -64,6 +65,10 @@ def get_splitter(
|
||||
from src.core.splitters.html import HTMLSplitter
|
||||
return HTMLSplitter(max_size=max_size, overlap=overlap)
|
||||
|
||||
if kind == "epub":
|
||||
from src.core.splitters.epub import EPUBSplitter
|
||||
return EPUBSplitter(max_size=max_size, overlap=overlap)
|
||||
|
||||
# 回退
|
||||
from src.core.splitters.text import TextSplitter
|
||||
return TextSplitter(max_size=max_size, overlap=overlap)
|
||||
|
||||
Reference in New Issue
Block a user