feat(ocr): replace Google Vision with Mistral OCR as PDF fallback
Switches the scanned-PDF fallback from Google Cloud Vision to Mistral OCR (mistral-ocr-latest) for better Hebrew accuracy and robustness against broken embedded OCR layers (e.g. case 1044-03-26 which returned English garbage through Vision). Routing strategy (document-level, not per-page): - PyMuPDF extracts all pages; pages that pass _text_quality_ok() use PyMuPDF output directly (free, ~50ms). - If ANY page fails quality → Mistral OCR called once for the whole PDF, returning per-page Markdown for all pages (consistent format, no plain-text/Markdown mix within a document). Markdown output preserved: Mistral returns ## headers and |tables|; chunker updated to recognise ATX Markdown headers (##/###) as section boundaries in _split_into_sections(). Config: GOOGLE_CLOUD_VISION_API_KEY → MISTRAL_API_KEY; allowlist updated vision.googleapis.com → api.mistral.ai. MISTRAL_API_KEY added to Coolify container env. Invariants: G1 (single OCR fallback path, not parallel), G2 (no duplicate extractor route), INV-AH (Mistral handles gershayim natively; quote-fix only on PyMuPDF path). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -275,8 +275,8 @@ HALACHA_CANONICAL_SYNTH_MODEL = os.environ.get("HALACHA_CANONICAL_SYNTH_MODEL",
|
||||
HALACHA_CANONICAL_SYNTH_EFFORT = os.environ.get("HALACHA_CANONICAL_SYNTH_EFFORT", "high")
|
||||
HALACHA_CANONICAL_SYNTH_DRIFT_FLOOR = float(os.environ.get("HALACHA_CANONICAL_SYNTH_DRIFT_FLOOR", "0.80"))
|
||||
|
||||
# Google Cloud Vision (OCR for scanned PDFs)
|
||||
GOOGLE_CLOUD_VISION_API_KEY = os.environ.get("GOOGLE_CLOUD_VISION_API_KEY", "")
|
||||
# Mistral OCR (fallback for scanned PDFs — replaces Google Cloud Vision)
|
||||
MISTRAL_API_KEY = os.environ.get("MISTRAL_API_KEY", "")
|
||||
|
||||
# Data directory
|
||||
DATA_DIR = Path(os.environ.get("DATA_DIR", str(Path.home() / "legal-ai" / "data")))
|
||||
@@ -361,7 +361,7 @@ PARENT_DOC_CHILD_OVERLAP_TOKENS = int(
|
||||
# External service allowlist — case materials may ONLY be sent to these domains
|
||||
ALLOWED_EXTERNAL_SERVICES = {
|
||||
"api.voyageai.com", # Voyage AI (embeddings)
|
||||
"vision.googleapis.com", # Google Cloud Vision (OCR)
|
||||
"api.mistral.ai", # Mistral OCR (scanned PDFs)
|
||||
}
|
||||
|
||||
# Audit
|
||||
|
||||
Reference in New Issue
Block a user