feat(ocr): replace Google Vision with Mistral OCR as PDF fallback
All checks were successful
G12 Leak-Guard / leak-guard (pull_request) Successful in 4s
Lint — undefined names / undefined-names (pull_request) Successful in 10s

Switches the scanned-PDF fallback from Google Cloud Vision to
Mistral OCR (mistral-ocr-latest) for better Hebrew accuracy and
robustness against broken embedded OCR layers (e.g. case 1044-03-26
which returned English garbage through Vision).

Routing strategy (document-level, not per-page):
- PyMuPDF extracts all pages; pages that pass _text_quality_ok()
  use PyMuPDF output directly (free, ~50ms).
- If ANY page fails quality → Mistral OCR called once for the whole
  PDF, returning per-page Markdown for all pages (consistent format,
  no plain-text/Markdown mix within a document).

Markdown output preserved: Mistral returns ## headers and |tables|;
chunker updated to recognise ATX Markdown headers (##/###) as section
boundaries in _split_into_sections().

Config: GOOGLE_CLOUD_VISION_API_KEY → MISTRAL_API_KEY; allowlist
updated vision.googleapis.com → api.mistral.ai.
MISTRAL_API_KEY added to Coolify container env.

Invariants: G1 (single OCR fallback path, not parallel), G2 (no
duplicate extractor route), INV-AH (Mistral handles gershayim
natively; quote-fix only on PyMuPDF path).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-06-27 10:11:45 +00:00
parent f2a264a7da
commit 9ae7304d44
4 changed files with 183 additions and 185 deletions

View File

@@ -144,9 +144,12 @@ def _split_into_sections(text: str) -> list[tuple[str, str]]:
markers: list[tuple[int, str]] = []
for pattern, section_type in SECTION_PATTERNS:
# ^ + MULTILINE: line start only. Optional leading spaces/tabs and an
# ^ + MULTILINE: line start only. Optional leading spaces/tabs, an
# optional Markdown ATX header prefix (``## ``/``### ``), and an
# optional ordinal prefix ("5.", "5)", "ג.") before the keyword.
anchored = rf"^[ \t]*(?:\d+[.)]\s*|[א-ת][.)]\s*)?(?:{pattern})"
# The Markdown prefix handles Mistral OCR output where section
# titles are rendered as ``## נימוקי הערר`` etc.
anchored = rf"^[ \t]*(?:#{1,3}\s+)?(?:\d+[.)]\s*|[א-ת][.)]\s*)?(?:{pattern})"
for match in re.finditer(anchored, text, re.MULTILINE):
markers.append((match.start(), section_type))