Stage C: per-page image embeddings via voyage-multimodal-3 + hybrid
text+image search. Off by default; enable with MULTIMODAL_ENABLED=true.
- Schema V9: document_image_embeddings + precedent_image_embeddings
(vector(1024), page_number, image_thumbnail_path)
- extractor.render_pages_for_multimodal renders PDF pages at
MULTIMODAL_DPI (144) for embedding + JPEG thumbnails at
MULTIMODAL_THUMB_DPI (96) for UI preview, in one pass
- embeddings.embed_images calls voyage-multimodal-3 in 50-page batches
- services/hybrid_search.py orchestrator: rerank applied to text side
first (rerank-2 is text-only); image side cosine; weighted merge
with text_weight 0.65 (env-tunable); image-only pages surface as
match_type='image' so dense scanned content still appears
- processor.process_document and precedent_library.ingest_precedent
gated by flag — non-fatal on multimodal failure
- scripts/multimodal_backfill.py — idempotent per-case CLI to embed
existing documents without re-extracting text
Validated locally on a 5-page response brief: render 0.31s, embed 8.32s,
hybrid merge surfaces image rows correctly. Production rollout starts
with flag=false (no behavior change), then per-case A/B.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PATCH /api/halachot/{id} was returning 500 because the row included
``embedding`` as a numpy.ndarray of np.float32, which FastAPI's
jsonable_encoder cannot serialize (vars() and dict() both fail on it).
The bug had been latent — it triggered for the first time today after
the auto-approve batch left only low-confidence halachot for the chair
to review manually, and her first PATCH hit the unserializable response.
Replace ``RETURNING *`` with an explicit column list (everything except
``embedding``). Callers that need the embedding can re-fetch via
``get_halacha`` — but no current caller does.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Halachot extracted by halacha_extractor with confidence >= 0.80 are now
inserted with review_status='approved' instead of 'pending_review' —
they appear in search_precedent_library immediately. Halachot below the
threshold still require manual chair approval.
Threshold tunable via env (HALACHA_AUTO_APPROVE_THRESHOLD), defaults to
0.80. Rationale: 89% of historical extractions (356/400) score 0.80+,
spot-checks confirmed quality, and the manual review backlog was the
single biggest reason rerank-2 was returning passages-only on
ההבחנה-style queries.
After this change + the one-time backfill UPDATE, search now returns
9/10 halachot for "ההבחנה בין השבחה לפיצויים" instead of 0 — and the
top-3 are exact-match rules, not adjacent passages.
Reviewer field records "auto-approved (confidence ≥ X.XX)" with the
threshold value at insert time, for traceability.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
set_case_law_extraction_status and set_case_law_halacha_status now NULL
the corresponding *_requested_at timestamp when status transitions to
"completed" or "failed". Without this, completed rows kept lingering in
the local-MCP work queue (which scans by `WHERE *_requested_at IS NOT NULL`)
and the UI's isPrecedentActive check, leaving them undeletable until a
manual SQL cleanup.
The pre-existing process_pending_extractions path already called
clear_extraction_request, but other paths (re-extraction, status set
during upload) didn't — so the cleanup belongs at the status setter.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The chair pointed out three UX gaps after uploading a new precedent:
1. The status said "מחלץ הלכות" but nothing was actually running — the
field only meant "halacha_extraction_status != completed", which
includes the post-upload "pending" state where the local MCP worker
hasn't been told to drain anything yet. Misleading.
2. The page didn't refresh on its own. The chair had to F5 to see new
counts after extraction completed.
3. Clicking the trash icon mid-extraction would cascade-delete the row
while the extractor was still using it (FK errors, partial writes).
Fixes:
- ingest_precedent now auto-queues both metadata and halacha extraction
on upload by stamping the request timestamps. The chair (or me) drains
the queue with one `precedent_process_pending` call from chat —
no need to click any button before that.
- StatusPill is now five-state with proper labels:
"נכשל" (extraction_status=failed) — red
"מעבד טקסט" — shimmer (extraction_status=processing)
"בתור" — neutral (chunks queued, not yet running)
"מחלץ הלכות" — shimmer (halacha_extraction_status=processing)
"ממתין לחילוץ" — neutral (queued for local MCP worker)
"לא חולץ" — neutral (pending without queue stamp — shouldn't happen)
"X/Y מאושרות" — gold (done, with halachot count)
The shimmer is a CSS-only sliding-stripe animation defined in globals.
- usePrecedents has a conditional refetchInterval — polls every 5s while
any row is mid-extraction or queued, then stops once everything settles
to completed/failed. New helper isPrecedentActive() centralises the
"is this row mid-something" check so the UI and the destructive-action
guard agree.
- Trash button is disabled (opacity 30%, tooltip explains) while the row
is active. Pencil/edit stays enabled — editing metadata fields during
extraction is safe (last write wins, low-stakes race).
Schema: list_external_case_law now exposes the two *_requested_at
timestamps so the UI can distinguish "queued" from "never asked".
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The chair wanted a one-click "extract metadata" button on the edit sheet.
The constraint stays the same — claude_session needs the local CLI which
the container doesn't have, so the button can't run the extractor itself.
Compromise: button stamps a queue marker; the local MCP server drains the
queue on demand.
DB (V8): two nullable timestamps on case_law,
metadata_extraction_requested_at and halacha_extraction_requested_at,
with partial indexes for cheap "find pending" scans.
API:
POST /api/precedent-library/{id}/request-metadata → stamp the row
POST /api/precedent-library/{id}/request-halachot → same for halacha
GET /api/precedent-library/queue/pending?kind=... → read-only view
UI: Sparkles button in the edit sheet header. Click → toast tells the
chair what to run from Claude Code. The button never triggers the
extractor directly from the container.
MCP tool: precedent_process_pending(kind, limit) — runs from Claude Code
with the local CLI, picks up everything stamped, calls the extractor for
each, clears the timestamp on success. Failures keep the timestamp so the
next invocation retries them.
Architectural rule (claude_session local-only) is preserved end-to-end
and called out in the new endpoint comment + tool docstring.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a third corpus of legal authority distinct from style_corpus
(Daphna's prior decisions for voice) and case_precedents (chair-attached
quotes per case). The new corpus holds chair-uploaded court rulings and
other appeals committee decisions, with binding rules (הלכות) extracted
automatically and queued for chair approval.
Pipeline (web/app.py + services/precedent_library.py):
file → extract → chunk → Voyage embed → halacha_extractor → store +
publish progress over the existing Redis SSE channel.
Schema V7 (services/db.py): extends case_law with source_kind +
extraction status fields under a CHECK constraint pinning practice_area
to the three appeals committee domains (rishuy_uvniya, betterment_levy,
compensation_197). New precedent_chunks (vector(1024)) and halachot
tables (vector(1024) over rule_statement, IVFFlat indexes, gin on
practice_areas/subject_tags). Halachot start as pending_review; only
approved/published rows are visible to search_precedent_library.
Agents: legal-writer, legal-researcher, legal-analyst, legal-ceo,
legal-qa get search_precedent_library. legal-writer prompt explains
the three-corpus distinction and CREAC use; legal-qa now verifies that
every cited halacha resolves to an approved row in the corpus.
UI: /precedents page with four tabs — library / semantic search /
pending review (J/K nav, A/R/E shortcuts, badge count) / stats.
Reuses the existing upload-sheet progress + SSE pattern.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The case_delete tool in tools/cases.py and the DELETE /api/cases endpoint
in web/app.py both invoke await db.delete_case(case_id), but no such
function existed in services/db.py — every call returned 500 with an
AttributeError. Discovered while wiping case 8174-24 for a clean rerun.
Implementation is straightforward because the FK graph already does the
work: 7 dependent tables CASCADE on cases.id (documents, document_chunks,
claims, appraiser_facts, decisions, qa_results, case_precedents) and 2
SET NULL (audit_log, chair_feedback). A single DELETE FROM cases is
enough — no manual ordering needed.
Documented in the docstring that this only touches the legal-ai DB —
Paperclip projects/issues and Gitea repos for the case are separate
systems and must be cleaned up by the caller.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a comprehensive archive flow for closed cases — separate /archive
screen in the UI, archive/restore actions on the case detail page, and
automatic two-way sync with Paperclip.
Backend (web/app.py + mcp-server/services/db.py):
- New SCHEMA_V6 migration: cases.archived_at TIMESTAMPTZ + partial index
- list_cases gains include_archived/archived_only flags; default excludes
archived rows so the main /api/cases list hides closed cases
- archive_case / restore_case helpers in db.py
- POST /api/cases/{n}/archive sets archived_at and calls
pc_archive_project (sets Paperclip projects.archived_at via direct DB)
- POST /api/cases/{n}/restore clears archived_at and calls
pc_restore_project (clears Paperclip archived_at)
- archive_project / restore_project in paperclip_client.py — name-based
match consistent with create_project's lookup
Frontend (web-ui):
- cases.ts: scope param ("active"|"archived"|"all") on useCases;
useArchiveCase / useRestoreCase mutations
- /archive page (new): table of archived cases with restore button +
search, sort, empty state matching the editorial aesthetic of /
- case-archive-action.tsx: button on case detail header. Active case →
confirm dialog → archive. Archived case → restore (no confirm).
Toast announces both legal-ai and Paperclip outcomes (synced, not
found in pc, error)
- case-header shows "בארכיון" badge when archived_at is set
- Nav: ארכיון link added to AppShell after בית
Tested end-to-end against the live DB:
- 1130-25 archive → list_cases(include_archived=False) excludes it,
list_cases(archived_only=True) includes it, restore reverses
- pc archive/restore on 1194-25 verified via direct DB lookup
- TypeScript compiles clean
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Until now changing a document's doc_type required a manual SQL update.
Adds an inline editor on the document badge so the chair can retag
without leaving the case page, and threads an appraiser_side tag
(committee / appellant / deciding) through the appraisal pipeline so
betterment-levy cases — which usually have 2-3 appraisers — render
conflicts with the deciding appraiser's view marked as governing.
Backend
- New appraiser_facts.appraiser_side column (V5.1) populated from
documents.metadata.appraiser_side at extraction time.
- extract_appraiser_facts now returns status='sides_missing' with the
list of untagged appraisals instead of running with empty side
labels — chair must tag every appraisal first via the UI.
- Conflict detection orders entries committee → appellant → deciding so
the deciding appraiser appears last; block-tet's prompt instructs the
writer to phrase the deciding appraiser's view as the governing
factual finding ("ואולם, השמאי המכריע קבע...").
- New PATCH /api/cases/{n}/documents/{doc_id} (Pydantic model with
whitelist validation) and matching document_update MCP tool. Both
merge appraiser_side into metadata JSONB instead of touching the
schema.
UI
- New shared doc-types module exports the canonical 11 doc_type
options plus the 3 appraiser-side options; both upload-sheet and
the document badge now read from it instead of duplicating Hebrew
labels.
- New DocumentTypeEditor renders a Popover off the doc-type Badge
with two Selects. The save button stays disabled while doc_type is
appraisal but no side has been picked, mirroring the backend
enforcement so the user finds out before triggering extraction.
- usePatchDocument React-Query mutation invalidates the case detail
on success so the badge updates without a manual refresh.
Lets the chair generate a partial decision DOCX before the discussion-and-
ruling block is decided. Same template, skill and DOCX styling as the final
decision (David, RTL, bookmarks) — only the block selection and order differ:
רקע (ו) → תכניות+היתרים (ט) → טענות (ז) → הליכים (ח). The opening (ה),
ruling (י), summary (יא), and signatures (יב) are omitted.
- New appraiser_facts table + CRUD + conflict detection in db.py (V5 schema).
Conflict = same plan/permit identifier reported differently by 2+ appraisers.
- New appraiser_facts_extractor service: per-appraisal Claude extraction of
plans + permits with raw quotes and page numbers.
- block-tet prompt extended with a permits sub-section sourced from the
extracted facts, plus an explicit instruction to flag inter-appraiser
conflicts in neutral wording without resolving them (deferred to block-yod).
- block-chet prompt extended with a post-hearing materials context sourced
from documents.metadata.is_post_hearing.
- docx_exporter.export_decision now accepts mode='interim' which reorders
the blocks per the chair's mental model and writes
טיוטת-ביניים-v{N}.docx (versioned independently of regular drafts).
- 3 new MCP tools: extract_appraiser_facts, write_interim_draft,
export_interim_draft. write_interim_draft auto-runs extraction if the
appraiser_facts table is empty for the case.
Fixes critical bug in 1033-25: user-uploaded עריכה-*.docx files were
orphaned on disk while exports kept rebuilding from stale DB blocks.
New architecture:
- User-uploaded DOCX becomes the source of truth (cases.active_draft_path)
- System edits via XML surgery with real Word <w:ins>/<w:del> revisions
- User can Accept/Reject each change from within Word
Components:
- docx_reviser.py: XML surgery for Track Changes (15 tests)
- docx_retrofit.py: retroactive bookmark injection with Hebrew marker
detection + heading heuristic (9 tests)
- docx_exporter.py: emits bookmarks around each of the 12 blocks
- 3 new MCP tools: apply_user_edit, list_bookmarks, revise_draft
- 4 new/updated endpoints: upload (auto-registers active draft),
/exports/revise, /exports/bookmarks, /exports/{filename}/retrofit,
/active-draft
- DB migration: cases.active_draft_path column
- UI: correct banner using real v-numbers, "מקור האמת" badge,
detailed upload toast with bookmarks_added/missing_blocks
- agents: legal-exporter (3 export modes), legal-ceo (stage G for
revision handling), legal-writer (revision mode)
Multi-tenancy:
- Works for both CMP (1xxx cases) and CMPA (8xxx/9xxx cases)
- New revise-draft skill added to both companies
- deploy-track-changes.sh syncs skills CMP ↔ CMPA
- retrofit_case.py: one-off retrofit of existing files
Tests: 34 passing (15 reviser + 9 retrofit + 4 exporter bookmarks + 6 e2e)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Add strip_nevo_preamble() to extractor.py — auto-removes Nevo database
headers (bibliography, legislation, mini-ratio) during training upload
- Add appeal_subtype column to style_patterns table — patterns are now
stored per subtype instead of globally mixed
- Update clear_style_patterns() to support subtype-scoped deletion
- Pass appeal_subtype through analyze_corpus → store → upsert pipeline
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Support ingestion of betterment levy (היטל השבחה) decisions into a
separate training corpus (CMPA). Key changes:
- Add .doc file extraction via LibreOffice conversion in extractor
- Add practice_area/appeal_subtype columns to style_corpus table
- Route training files to cmp/ or cmpa/ subdirs based on appeal subtype
- Fix derive_subtype to handle ARAR-YY-NNNN format (was matching year digit)
- Expose practice_area/appeal_subtype params in MCP upload_training tool
- Add appeal_subtype filter to analyze_style for per-type style analysis
- Update betterment levy methodology in lessons.py: checklist (from generic
to corpus-based), opening/closing strategies, and discussion rules
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When a case is created, a Paperclip project is now automatically created in
the correct company based on the appeal_subtype tag. Tag-to-company mappings
are managed via a new Settings page that pulls companies from Paperclip DB.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The merge of ui-rewrite removed these parameters from db.search_similar()
and db.create_case() but left the callers passing them, causing TypeError
on any corpus search. Restores the parameters and adds schema migration.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Four functions were called by tools/precedents.py but never implemented
in services/db.py: create_case_precedent, list_case_precedents,
delete_case_precedent, search_precedent_library. This caused 500 errors
on the /api/cases/{n}/precedents endpoint.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Documents tab was limited to ~9 visible items due to fixed max-height
without overflow-hidden. Now uses 70vh with proper overflow. Added
click-to-preview (shows extracted text in dialog) and delete button
with confirmation dialog + backend DELETE endpoint.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New columns for methodology-aware decision pipeline:
claims table:
- claim_handling (address/bundle/skip) — per-claim handling mode
- bundle_group — group name for bundled claims
- handling_reason — explanation for skip/bundle
cases table:
- standard_of_review — review standard (independent discretion / etc.)
- subject_categories — JSONB array of topics in the appeal
case_law table:
- precedent_level — hierarchy (supreme/administrative/national/district)
- is_binding — binding holding vs. obiter dictum
- creac_role — how it serves reasoning (rule/explanation/analogy)
decisions table:
- issue_order — JSONB array of ordered issues with type
- claim_handling — JSONB overrides from chair_directions
Migration tested and applied successfully on production DB.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Addresses Dafna's observation that licensing decisions lack comprehensive
planning discussion. Systematic corpus analysis of all 24 training decisions
revealed the system learned writing style but not substantive content.
Changes:
- Corpus analysis of all 24 decisions (docs/corpus-analysis.md)
- 5 content checklists by appeal subtype injected into block-yod prompt
- chair_feedback DB table + API endpoints + MCP tools
- Feedback management page in Next.js UI (/feedback)
- Navigation updated with "הערות יו״ר" link
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- DELETE /api/training/corpus/{id} + delete button on training page,
with confirmation dialog and recompute hint
- /api/system/tasks + floating process panel (bottom-left) showing
active background tasks with live 3s polling
- /api/system/recent-activity derives a feed from cases, style_corpus,
and last style_patterns run; sidebar on home page renders with
relative timestamps
- /api/system/diagnostics + /#/diagnostics page showing DB health,
row counts per table, active tasks, stuck documents (>10 min),
failed extractions
- Cosmetic: signature phrase headline now prefers clean phrases over
bracket-heavy templates for display
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add delete_document_chunks for reprocessing, save extracted text to disk
- Expand case directory structure (original/extracted/proofread/backup)
- Update classifier patterns (תגובה, הודעת עמדה)
- Fix proofreader agent paths for new directory layout
- Update HEARTBEAT to notify on every task completion
- Improve bidi_table with LRE/PDF directional embedding
- Add Paperclip project verification and auto-close setup issue
- Add auto-sync-cases.sh for Gitea synchronization
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add expected_outcome field to cases (rejection/partial/full/betterment_levy)
- New lessons.py module with golden ratios, templates, and drafting guidance per outcome type
- Style analyzer now uses Opus with full decision text (no truncation), with multi-pass fallback for large corpora
- Drafting tool provides outcome-specific templates, section guidance, and ratio comments
- Improved JSON extraction with bracket-matching fallback
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Ezer Mishpati - AI legal decision drafting system with:
- MCP server (FastMCP) with document processing pipeline
- Web upload interface (FastAPI) for file upload and classification
- pgvector-based semantic search
- Hebrew legal document chunking and embedding
