feat(ocr): replace Google Vision with Mistral OCR as PDF fallback
Switches the scanned-PDF fallback from Google Cloud Vision to Mistral OCR (mistral-ocr-latest) for better Hebrew accuracy and robustness against broken embedded OCR layers (e.g. case 1044-03-26 which returned English garbage through Vision). Routing strategy (document-level, not per-page): - PyMuPDF extracts all pages; pages that pass _text_quality_ok() use PyMuPDF output directly (free, ~50ms). - If ANY page fails quality → Mistral OCR called once for the whole PDF, returning per-page Markdown for all pages (consistent format, no plain-text/Markdown mix within a document). Markdown output preserved: Mistral returns ## headers and |tables|; chunker updated to recognise ATX Markdown headers (##/###) as section boundaries in _split_into_sections(). Config: GOOGLE_CLOUD_VISION_API_KEY → MISTRAL_API_KEY; allowlist updated vision.googleapis.com → api.mistral.ai. MISTRAL_API_KEY added to Coolify container env. Invariants: G1 (single OCR fallback path, not parallel), G2 (no duplicate extractor route), INV-AH (Mistral handles gershayim natively; quote-fix only on PyMuPDF path). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -144,9 +144,12 @@ def _split_into_sections(text: str) -> list[tuple[str, str]]:
|
||||
markers: list[tuple[int, str]] = []
|
||||
|
||||
for pattern, section_type in SECTION_PATTERNS:
|
||||
# ^ + MULTILINE: line start only. Optional leading spaces/tabs and an
|
||||
# ^ + MULTILINE: line start only. Optional leading spaces/tabs, an
|
||||
# optional Markdown ATX header prefix (``## ``/``### ``), and an
|
||||
# optional ordinal prefix ("5.", "5)", "ג.") before the keyword.
|
||||
anchored = rf"^[ \t]*(?:\d+[.)]\s*|[א-ת][.)]\s*)?(?:{pattern})"
|
||||
# The Markdown prefix handles Mistral OCR output where section
|
||||
# titles are rendered as ``## נימוקי הערר`` etc.
|
||||
anchored = rf"^[ \t]*(?:#{1,3}\s+)?(?:\d+[.)]\s*|[א-ת][.)]\s*)?(?:{pattern})"
|
||||
for match in re.finditer(anchored, text, re.MULTILINE):
|
||||
markers.append((match.start(), section_type))
|
||||
|
||||
|
||||
Reference in New Issue
Block a user