Commit Graph

34 Commits

Author SHA1 Message Date
0c4886afe6 Wire legal-writer to chair directions from analysis-and-research.md
Closes the loop so דפנה's positions (written inline in the UI and
saved to analysis-and-research.md) automatically become binding
direction for the legal-writer agent — no manual copy-paste,
no bypass.

Backend:
- research_md.extract_chair_directions(path) returns a compact dict
  with status (missing/empty/partial/complete), filled_count,
  empty_count, and a reduced list of threshold_claims + issues each
  with {id, number, title, direction}. Designed to be directly usable
  as direction_doc by the writer.
- New MCP tool: drafting.get_chair_directions(case_number) wraps the
  helper, resolves the case research file path via config.find_case_dir,
  returns formatted JSON.
- Registered in server.py as mcp__legal-ai__get_chair_directions.

legal-writer agent update:
- Adds get_chair_directions to the tools list.
- New mandatory "שלב 1ב" before any block writing: call
  get_chair_directions, branch on status.
  - missing → halt, report "legal-analyst לא רץ עדיין"
  - empty → halt, instruct Dafna to fill positions via the UI URL
  - partial → halt unless user confirms; write only filled sections
  - complete → proceed
- New "שלב 1ג" constructs an internal direction_doc from the
  received chair rulings before writing block י.
- Block י section expanded with 5 binding rules:
  1. Open each discussion with Dafna's ruling as the thesis
  2. Frame the reasoning in her style (use get_style_guide phrases)
  3. Match her tone (decisive vs nuanced)
  4. Must NOT contradict her position — if she disagreed with your
     own inclination, her position rules
  5. Use legal_questions from the analysis file as the analytical
     structure (principle question first, concrete application second)
- New bullet section for block יא: summarize each chair ruling
  briefly, state final outcome, close with the signed date formula.

Verified all four status paths (missing/empty/partial/complete) via
local test. Now Dafna's workflow is fully end-to-end: she reads the
analyst report in the UI, fills "עמדת ועדת הערר" in each card, hits
blur to auto-save, then triggers legal-writer — which picks up her
positions as direction without any file shuffle.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 13:04:30 +00:00
753fe0d57d Research analysis cards with inline chair-position editor
New feature on case view: the analysis-and-research.md produced by the
legal-analyst agent is now rendered as structured cards in the UI,
with inline editing of "עמדת ועדת הערר" that writes directly back to
the markdown file (atomic rename).

Backend (research_md.py):
- parse(Path) → dict with header, prose sections, threshold_claims[],
  issues[], conclusions, other_sections
- Tolerant field extractor handles both block ("**LABEL:**\ncontent")
  and inline ("**LABEL:** content") variants
- Detects [ימולא ע"י יו"ר הוועדה] placeholder → empty chair_position
- update_chair_position(path, section_id, text) locates the exact
  subsection by ordinal, replaces or appends the chair field, writes
  atomically via temp file + os.replace
- Section IDs: threshold_N / issue_N (1-based)

Endpoints:
- GET /api/cases/{n}/research/analysis — returns parsed JSON or 404
- PATCH /api/cases/{n}/research/analysis/chair-position — {section_id, position}

Frontend (#page-case):
- New card "ניתוח משפטי ומחקר" below local-files card
- Prose sections as justified text panels (background + gold border)
- Threshold claims and issues as collapsible <details> items with
  gold right-border on open, numbered pills
- Each item shows all extracted fields with label above content
- Chair position editor: gold-wash background, 📝 icon label, textarea
  with placeholder prompt
- onblur → PATCH with save indicator:  שומר → ✓ נשמר HH:MM → fade
- Status pill next to each item title: "ממתין לעמדה" / "✓ עמדה נקבעה"
- First threshold claim opens by default, rest closed
- Card hidden entirely when no analysis file exists (404)

Tested against real file: case 1033-25 with 3 threshold claims and
6 issues, all chair positions correctly empty, update writes only the
targeted section, atomic rewrite preserves all other content.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 12:47:36 +00:00
3e0221ccec Management UI: corpus delete, process panel, activity feed, diagnostics
- DELETE /api/training/corpus/{id} + delete button on training page,
  with confirmation dialog and recompute hint
- /api/system/tasks + floating process panel (bottom-left) showing
  active background tasks with live 3s polling
- /api/system/recent-activity derives a feed from cases, style_corpus,
  and last style_patterns run; sidebar on home page renders with
  relative timestamps
- /api/system/diagnostics + /#/diagnostics page showing DB health,
  row counts per table, active tasks, stuck documents (>10 min),
  failed extractions
- Cosmetic: signature phrase headline now prefers clean phrases over
  bracket-heavy templates for display

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 12:04:13 +00:00
32f18de049 Add training corpus UI with Nevo proofreading pipeline
- New proofreader service strips Nevo editorial additions (front matter,
  postamble, page headers, watermarks, inline codes) from DOCX/PDF/MD
- PDF pages use Google Vision OCR for clean Hebrew RTL extraction
- New training page at #/training with drag-and-drop upload, automatic
  metadata extraction (decision number, date, categories), reviewable
  preview, and style pattern report grouped by type
- API endpoints: /api/training/{analyze,upload,corpus,patterns,
  analyze-style,analyze-style/status}
- Fix claude_session.query to pipe prompt via stdin, avoiding ARG_MAX
  overflow when analyzing 900K+ char corpus
- CLI scripts for batch proofreading and corpus upload

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 11:04:58 +00:00
3f759d3610 Improve document processing pipeline and agent workflows
- Add delete_document_chunks for reprocessing, save extracted text to disk
- Expand case directory structure (original/extracted/proofread/backup)
- Update classifier patterns (תגובה, הודעת עמדה)
- Fix proofreader agent paths for new directory layout
- Update HEARTBEAT to notify on every task completion
- Improve bidi_table with LRE/PDF directional embedding
- Add Paperclip project verification and auto-close setup issue
- Add auto-sync-cases.sh for Gitea synchronization

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 16:45:49 +00:00
22e819363e Flatten cases directory structure and unify paths
- Remove cases/new|in-progress|completed subdivision (status managed in DB)
- Rename documents/original → documents/originals (consistent plural)
- Move exports from global data/exports/ into cases/{num}/exports/
- Add documents/research/ for case law and analysis files
- Update all agents, scripts, config, web API endpoints, and DB paths

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 14:33:27 +00:00
6aaca14e31 Replace Claude Vision OCR with Google Cloud Vision
Benchmark results on Hebrew legal docs (case 1130-25):
- Google Vision: 1s/page, $0.001/page, high accuracy
- Claude Opus Vision: 90s/page, $0.05/page, poor accuracy
- PyMuPDF broken OCR layers now detected via quality check

Changes:
- extractor.py: Google Vision OCR with Hebrew language hint (300 DPI)
- extractor.py: text quality detection (word length, words-per-line, Hebrew ratio)
- extractor.py: Hebrew abbreviation quote fixer (15 known patterns)
- config.py: add GOOGLE_CLOUD_VISION_API_KEY, remove ANTHROPIC_API_KEY
- pyproject.toml: add google-cloud-vision, remove anthropic

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 20:17:58 +00:00
bc72a83a71 Switch embedding model from voyage-3-large to voyage-law-2
Benchmark on case 1130-25 (4 Hebrew legal docs, 8 queries) showed:
- voyage-law-2: avg top-1 score 0.5839 (+27% over voyage-3-large)
- voyage-4-large: avg top-1 score 0.4119 (worse than current)
- voyage-3-large: avg top-1 score 0.4589 (baseline)

voyage-law-2 costs ~4.6x more per run but delivers significantly
better retrieval quality for Hebrew legal text. Model is now
configurable via VOYAGE_MODEL env var.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 19:05:58 +00:00
5a8d5cac0a Add exports panel: versioned drafts, download, upload revisions, mark final
Export DOCX now saves to data/exports/{case_number}/ with auto-versioning
(טיוטה-v1, v2...). The case view UI shows all drafts with download buttons,
allows uploading revised versions (עריכה-v1...), and marking a version as
final (copies to training corpus for style learning).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 12:10:02 +00:00
4df2040a40 Fix: save_block_content now writes draft file + writer must update status
Two issues that caused QA agent to fail:
1. save_block_content saved to DB only — now also rebuilds drafts/decision.md
2. legal-writer.md now has explicit mandatory step: case_update(status="drafted")

Without these, workflow_status reports has_draft=false and QA can't run.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 15:25:53 +00:00
96ea54dc6e Add claim_type field: distinguish claims vs responses vs replies
Legal documents have 3 types of assertions:
- claim: from appeal documents (כתב ערר)
- response: from original responses (כתב תשובה)
- reply: from supplementary responses (תגובה, השלמת טיעון)

DB: added claim_type column to claims table
Extractor: _infer_claim_type() auto-detects from doc_type + title
Updated existing 113 records: 29 claims, 28 responses, 56 replies

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 15:35:16 +00:00
328436f56d Remove stale classifier import from processor.py (was deleted)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 14:45:19 +00:00
911c797eb2 Reorganize: skills/ directory + move memory to docs/
skill-legal-decision/ → skills/decision/
skill-legal-assistant/ → skills/assistant/
skill-legal-docx/ → skills/docx/
memory/*.md → docs/

Also removed: TASKS.md (use TaskMaster), classifier.py (replaced by local_classifier.py)
Updated all references in CLAUDE.md, scripts, PRDs, docs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 14:27:07 +00:00
bacb330a2a Replace all Anthropic API calls with Claude Code session (claude -p)
New module claude_session.py provides query() and query_json() that
run prompts via `claude -p` CLI — uses the claude.ai session, zero API cost.

Converted 6 services:
- claims_extractor.py: extract_claims_with_ai
- brainstorm.py: brainstorm_directions
- block_writer.py: write_block (was streaming+thinking, now simple)
- qa_validator.py: claims_coverage check
- style_analyzer.py: 3 API calls (single pass, multi pass, synthesis)
- learning_loop.py: extract_lessons

Only extractor.py still uses Anthropic API (for PDF OCR with Vision).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 14:14:08 +00:00
52ee3419d3 Add local rule-based classifier with Claude Code headless fallback
Replaces API-based classifier with:
1. Filename pattern matching (covers 95%+ of legal docs)
2. Content keyword matching for ambiguous filenames
3. Claude Code headless (claude -p) fallback for edge cases

No Anthropic API calls needed for classification.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 13:14:13 +00:00
9e7492e761 Make classification and reference extraction non-fatal in document pipeline
Text extraction, chunking and embedding proceed even if Claude API
classification or reference extraction fails (e.g. API quota exceeded).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 13:00:34 +00:00
5fc52ce530 Switch to cases/{new,in-progress,completed}/ directory structure
Replace single CASES_DIR with find_case_dir() that searches across
all status directories. New cases created in cases/new/{number}/.

Config: CASES_BASE, CASES_NEW, CASES_IN_PROGRESS, CASES_COMPLETED
Docker: added -v /home/chaim/legal-ai/cases:/cases volume mount

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 10:45:47 +00:00
081c7fb17a Replace Haiku with Sonnet in classifier for better accuracy
classify_document and identify_parties both used Haiku, which produced
parsing failures and 0% confidence on Beit HaKerem documents.
Sonnet handles Hebrew legal documents more reliably.

No more Haiku usage in the entire codebase.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 07:47:12 +00:00
586f1db402 QA claims check: Haiku→Sonnet + filter appellant claims only
Two fixes for claims_coverage false negatives (55% → expected ~85%+):

1. Model upgrade: Haiku → Sonnet for semantic matching. Haiku missed
   obvious matches (e.g., paragraph about "כריתת עצים" not matching
   claim about tree cutting). Sonnet understands context better.

2. Filter: only check appellant/respondent claims, not committee or
   permit_applicant claims. Committee claims are defensive positions
   ("the application complies with the plan") — they don't need to
   be "addressed" in the discussion section.

3. Send full discussion text (was truncated to 12K chars).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 07:37:23 +00:00
9d0a73a1dc Add context-only mode: Claude Code writes blocks, no API needed
New architecture: MCP provides context, Claude Code writes.

New functions:
- get_block_context(case_id, block_id) → returns full context package
  (prompt, source docs, claims, direction, precedents, style guide)
  WITHOUT calling Anthropic API
- save_block_content(case_id, block_id, content) → saves block to DB

New MCP tools: get_block_context, save_block_content

The old write_block (API-based) still works as fallback.
The new flow uses Claude Code's own model (Opus 4.6, 1M context)
which has no separate API billing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 16:18:25 +00:00
7033d2d3ee Embed full style guide in block prompts for Dafna's voice
_build_style_context rewritten from 10-line summary to comprehensive
style guide including:
- Tone rules per appeal type (warm for licensing, cold for levy)
- 15 mandatory expressions ("כידוע", "ברי כי", "אין בידנו לקבל")
- Discussion structure rules (continuous prose, conclusion first)
- Per-party phrasing templates (appellants, committee, permit applicants)
- DB patterns grouped by type (phrases, transitions, openings, closings)

This addresses the main quality gap: style rated 2/5 because the output
was "dry and overly formal" vs Dafna's "direct and clear" voice.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 16:12:09 +00:00
e725f9ecd7 Fix claims parsing: truncated JSON recovery + chunking + compact output
config.py parse_llm_json: Added truncated JSON recovery. When Claude's
output is cut mid-JSON (common with long claim lists), the parser now:
- Finds the last complete JSON item (closing "}")
- Closes the array/object brackets
- Returns partial but valid results instead of None
Tested: recovers 2/3 items from truncated array, all cases pass.

claims_extractor.py:
- Prompt asks for compact output (150 words max per claim, group similar)
- Explicitly requests "no markdown, no explanations, JSON only"
- Long documents split into chunks at paragraph boundaries
- Each chunk processed separately, results merged
- max_tokens already at 8192

This fixes the recurring "0 claims" bug for committee responses and
permit applicant responses where the JSON was getting truncated.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 16:04:34 +00:00
7d1dc73112 Fix max_tokens to 16K for Opus (API limit is 32K, need room for thinking)
block-yod max_tokens reduced from 32K to 16K — the API returned
"max_tokens: 32768 > 32000" error. With thinking enabled, the actual
limit for output is lower. 16K is sufficient for discussion blocks.

Also: extractor.py now supports .md files (was missing, blocked
Beit HaKerem upload).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 16:00:49 +00:00
e24e24dac5 Maximize context and output per Anthropic best practices
Per official Anthropic documentation (April 2026):

Output tokens increased to match model capabilities:
- block-yod (discussion): 8K → 32K (Opus supports 128K)
- block-zayin (claims): 4K → 16K
- block-vav (background): 4K → 16K
- claims_extractor: 4K → 8K (fixes truncated JSON)
- qa_validator: 4K → 8K

Source documents sent in full (not truncated):
- Was: 3000 chars per doc, 15K total
- Now: full document text, no truncation
- Reduces hallucinations: "extract word-for-word quotes first"

Prompt structure follows long-context tips:
- Source documents placed FIRST (top of prompt)
- Instructions and query placed LAST
- "Queries at the end improve quality by up to 30%"

Extended thinking uses adaptive mode for Opus 4.6.
Streaming enabled for all requests > 21K tokens.

Unified JSON parsing via parse_llm_json() helper in config.py.
Applied to: classifier, claims_extractor, brainstorm, qa_validator,
learning_loop (5 files).

Also: extractor.py now supports .md files.

Sources:
- https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking
- https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/long-context-tips
- https://docs.anthropic.com/en/docs/minimizing-hallucinations
- https://docs.anthropic.com/en/docs/about-claude/models/overview

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 14:17:43 +00:00
bed9d5c7e9 Improve block-zayin: synthesize claims by topic + fix markdown JSON parsing
block_writer: Rewrote block-zayin prompt to require synthesis by topic
instead of listing each claim separately. Now produces 3 organized
sections (appellants 8, committee 6, permit applicants 3+) instead
of 40 scattered paragraphs. Target: 800-1500 words.

claims_extractor: Fix markdown code block stripping (same bug as
qa_validator had). Enables parsing claims from Claude responses
wrapped in ```json blocks.

Tested on Hecht: block-zayin from 40 paragraphs/1049 words to
17 organized paragraphs/1039 words. Structure now matches Dafna's
original (3 parties, grouped by topic).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 12:54:42 +00:00
e438740ab4 Add renumber_all_blocks + fix sequential_numbering check for bold format
block_writer: new renumber_all_blocks() function that renumbers all
paragraphs across all blocks sequentially (1, 2, 3...). Handles both
plain "N." and bold "**N.**" formats. Added missing 'import re'.

qa_validator: sequential_numbering check now matches bold-formatted
numbers (**N.**) in addition to plain (N.).

Tested on Hecht: renumbered 115 paragraphs across 7 blocks, QA 6/6.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 12:30:31 +00:00
7781987c3a Fix precedents search + auto-update case parties
block_writer: _build_precedents_context now searches both
paragraph_embeddings (other decisions by Dafna) and case_law_embeddings
(precedent case law). Previously only searched document_chunks which
had no cross-case data. Now returns ~2400 chars from 3 other decisions.

processor: Step 1.6 auto-updates case appellants/respondents from
classifier results when they're empty. Prevents blank party fields.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 11:59:33 +00:00
52beb6ebdc Replace keyword claims check with Claude-based semantic check
claims_coverage now uses Claude Haiku to check if each claim is
semantically addressed in the discussion, not just keyword-matched.

- Sends all claims + discussion to Claude in one API call
- Returns addressed/partial/missing for each claim
- Handles markdown code block wrapping in response
- max_tokens 4096 (was 2048) for 48+ claims

Result on Hecht: 45/48 addressed (94%), 1 partial, 3 missing.
The 3 missing are genuinely unaddressed (personal/procedural claims).
Previously keyword check showed 47/48 but missed semantic gaps.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 11:38:31 +00:00
018b5936a1 Fix claims handling: filter block-zayin duplicates, improve QA matching
block_writer: _build_claims_context now filters out block-zayin claims
(from final decision) and uses only claims from original pleadings.
Reduces noise from 78 to 48 real claims for Hecht case.

qa_validator: claims_coverage check rewritten:
- Filter block-zayin claims (same reason)
- Keyword-based matching instead of 3-word phrase matching
- 25% keyword overlap threshold (was: any 3-word match)
- Allow up to 20% uncovered claims before failing
- Check both block-yod and block-zayin for coverage

Result: Hecht case QA goes from 4/6 to 6/6, 47/48 claims covered (98%).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 11:32:29 +00:00
570f745823 Improve block-yod prompt: require minimum length, numbered claims, precedent citations
- Add minimum word count guidance (2000-4000 words)
- Number each claim in claims_context for explicit tracking
- Require 3-5 case law citations minimum
- Fix max_tokens > budget_tokens for extended thinking
- Use streaming for opus+thinking requests (>10min timeout)

Tested on Hecht case: block-yod improved from 1039 to 1927 words.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 11:28:12 +00:00
38a61712bc Fix plan regex: require numeric identifier after תב"ע
Previously matched any word after תב"ע (e.g., "תב"ע ואין", "תב"ע קיפחה").
Now requires a plan number (digits/hyphens) — reduces false positives from 24 to 4
on the Hecht case test.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 10:50:56 +00:00
d9e5ef0f46 Add full decision writing pipeline: classify, extract, brainstorm, write, QA, export
New services (11 files):
- classifier.py: auto doc-type classification + party identification (Claude Haiku)
- claims_extractor.py: claim extraction from pleadings (Claude Sonnet + regex)
- references_extractor.py: plan/case-law/legislation detection (regex)
- brainstorm.py: direction generation with 2-3 options (Claude Sonnet)
- block_writer.py: 12-block decision writer (template + Claude Sonnet/Opus)
- docx_exporter.py: DOCX export with David font, RTL, headings
- qa_validator.py: 6 QA checks with export blocking on critical failure
- learning_loop.py: draft vs final comparison + lesson extraction
- metrics.py: KPIs dashboard per case and global
- audit.py: action audit log
- cli.py: standalone CLI with 11 commands

Updated pipeline: extract → classify → chunk → embed → store → extract_references
New MCP tools: 29 total (was 16)
New DB tables: audit_log, decisions CRUD, claims CRUD
Config: Infisical support, external service allowlist

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 10:21:47 +00:00
39089dcef5 Add outcome-aware drafting, lessons system, and improved style analysis
- Add expected_outcome field to cases (rejection/partial/full/betterment_levy)
- New lessons.py module with golden ratios, templates, and drafting guidance per outcome type
- Style analyzer now uses Opus with full decision text (no truncation), with multi-pass fallback for large corpora
- Drafting tool provides outcome-specific templates, section guidance, and ratio comments
- Improved JSON extraction with bracket-matching fallback

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 18:58:42 +00:00
6f515dc2cb Initial commit: MCP server + web upload interface
Ezer Mishpati - AI legal decision drafting system with:
- MCP server (FastMCP) with document processing pipeline
- Web upload interface (FastAPI) for file upload and classification
- pgvector-based semantic search
- Hebrew legal document chunking and embedding
2026-03-23 12:33:07 +00:00