fix: 修复 MarkdownSplitter 死循环 + 添加 GPU 自动检测
- ingest.py: _split_single_paragraph 中分隔符距 start 小于 overlap 时 start 会回退为负数,Python rfind 负索引导致无限循环 修复: start = max(start + 1, next_start) 确保始终前进 - embedder.py: LocalEmbedder 自动检测 CUDA,优先使用 GPU - ingest_obsidian.py: MAX_SIZE 调整为 200KB 适配 GPU 嵌入 - 清理临时日志文件
This commit is contained in:
+3
-1
@@ -155,7 +155,9 @@ class MarkdownSplitter:
|
||||
part = text[start:break_point].strip()
|
||||
if part:
|
||||
parts.append(part)
|
||||
start = break_point - self.overlap if self.overlap > 0 else break_point
|
||||
# 确保 start 始终前进(避免分隔符距 start 小于 overlap 时 start 回退导致死循环)
|
||||
next_start = break_point - self.overlap if self.overlap > 0 else break_point
|
||||
start = max(start + 1, next_start)
|
||||
return parts
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user