From 99cd6bc4dd18675844c021c62a963f1b7962a283 Mon Sep 17 00:00:00 2001 From: Chaim Date: Sat, 30 May 2026 21:15:50 +0000 Subject: [PATCH 01/10] docs(spec): FU-7 audit-trail + provenance design (GAP-17/18/19/20) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reuse audit_log.log_action with details JSONB (X5 §4, no new table) for end-to-end audit + block→source provenance. GAP-17 drift = blocks_stale flag + health-check (not fragile DOCX→blocks reparse). GAP-20 = structural case_law_id resolution (not Hebrew citation NLP). Verified vs 3+ sources (append-only lineage event; GitOps drift detect-don't-auto-remediate). Pure-code, no migration. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../2026-05-30-fu7-audit-provenance-design.md | 122 ++++++++++++++++++ 1 file changed, 122 insertions(+) create mode 100644 docs/superpowers/specs/2026-05-30-fu7-audit-provenance-design.md diff --git a/docs/superpowers/specs/2026-05-30-fu7-audit-provenance-design.md b/docs/superpowers/specs/2026-05-30-fu7-audit-provenance-design.md new file mode 100644 index 0000000..6b73a45 --- /dev/null +++ b/docs/superpowers/specs/2026-05-30-fu7-audit-provenance-design.md @@ -0,0 +1,122 @@ +# FU-7 — Audit-Trail + Provenance — עיצוב + +**סטטוס:** מאושר-לעיצוב · **תאריך:** 2026-05-30 · **ענף:** TBD +**מכסה:** GAP-17, GAP-18, GAP-19, GAP-20 · **מספק:** INV-AUD1, INV-AUD2, INV-AUD3, INV-EX1, INV-G9 +**מקורות:** [X5-audit-provenance.md](../../spec/X5-audit-provenance.md), [06-export.md](../../spec/06-export.md), [gap-audit.md](../../spec/gap-audit.md) +**משימה:** TaskMaster #65 · **תלוי ב:** FU-1 (#59) · **סוג:** pure-code (schema-additive קל) +**מיגרציה:** אין. כל השינויים forward-only; backfill קל אופציונלי (provenance של בלוקים קיימים לא נאכף רטרואקטיבית). + +--- + +## 1. מטרה והיקף + +X5 §4 קובע את המנגנון הקנוני: **שימוש חוזר ב-`audit_log.log_action` עם `details` JSONB** — +לא טבלה חדשה (כלל-הנדסה "סימטריה"). FU-7 ממיר את `audit_log` מ"כמעט-ריק" ל-audit-trail מקצה-לקצה, +מוסיף provenance בלוק→מקורות, אוכף ציטוט→קורפוס, ומגלה drift בין DOCX-החי לבלוקים. + +| GAP | בעיה (מאומת בקוד) | יעד FU-7 | +|-----|--------------------|----------| +| GAP-18 | `log_action` נכתב רק ב-`case_subtype_override` (cases.py:203) | קריאות `log_action` ב-4 פעולות משנות-מצב: upload, extract_claims, write_block, export | +| GAP-19 | `decision_blocks` נושא `model_used` בלבד — אין קישור לקטעי-מקור | רשומת provenance ב-`audit_log.details` עם source ids שהזינו את הגנרציה | +| GAP-20 | אין אכיפה שציטוט פתיר לקורפוס | ולידציה דטרמיניסטית של `case_law_id` בציטוטים → flag לבלתי-פתירים | +| GAP-17 | `active_draft_path` הופך SoT אחרי revise/apply בלי re-sync לבלוקים | דגל `blocks_stale` דטרמיניסטי + חשיפת drift ב-health-check (לא re-sync שביר) | + +## 2. הכרעות אדריכליות (מאומתות ≥3 מקורות) + +| החלטה | נימוק | מקורות | +|-------|--------|--------| +| provenance כ-**event ב-`audit_log` append-only** (details payload), לא עמודה/טבלה חדשה | דפוס lineage בוגר: entity-key + event-type + actor + source-ids; X5 §4 (סימטריה) | Snowflake data-lineage; OvalEdge provenance; DesignGurus append-only audit | +| GAP-17 = **detect + flag**, מקור-אמת=בלוקים, לא auto-resync | auto-remediation דורש rollback אמין; reparse DOCX→blocks שביר (edits שוברים מבנה) | Flux GitOps drift; Terraform drift (env0); Spacelift | +| GAP-20 = **ולידציה מבנית** של `case_law_id` פתיר, לא NLP של ציטוט חופשי | NLP-ציטוט עברי חופף ל-`extract_internal_citations` הקיים; INV-AUD3 מנוסח סביב פתירוּת `case_law_id` | X5 INV-AUD3; RAG attribution (Lewis 2020); ISO 8000 | +| audit כ-**non-fatal** (כשל-audit מתעד warning, לא מפיל פעולה) | git הוא שכבת-השלמות (X5 §2.1); audit_log הוא observability "מי/מה/מתי" | X5 §2.1; דפוס audit fire-safe | + +## 3. הקבצים + +- **Modify** `tools/audit.py` — אין שינוי לחתימת `log_action`; להוסיף helper `log_action_safe(...)` שעוטף ב-try/except (warning, non-fatal) כדי שכשל-audit לא יפיל את הפעולה. +- **Modify** `tools/documents.py` — `document_upload` (~:14) + `extract_claims` (~:300): קריאת `log_action_safe`. +- **Modify** `services/block_writer.py` — `write_block`/`store_block` (~:1010): לאסוף source ids מ-context builders + לכתוב audit `write_block` עם provenance. +- **Modify** `tools/drafting.py` — `export_docx` (~:384): audit `export_docx`; `revise_draft` (~:647) + `apply_user_edit` (~:569): סימון `blocks_stale=true`. +- **Modify** `services/db.py` — מיגרציה V22: עמודת `cases.blocks_stale boolean DEFAULT false`; helper `mark_blocks_stale(case_id, val)`; helper `resolve_citation_case_law_ids(ids)` (בדיקת קיום); helper `audit_provenance_query` (קריאה — לא חובה). +- **Modify** `services/qa_validator.py` (או היכן שרץ QA) — בדיקת ציטוט→קורפוס: לכל `case_law_id` בציטוטי-הבלוק, אם לא פתיר → ממצא-QA (warning) + audit `citation_unresolved`. +- **Modify** health-check (metrics.py / processing_status) — חשיפת `cases_with_stale_blocks` count. +- **Test** `tests/test_audit_provenance.py` (חדש) — offline, monkeypatched. + +**גבול:** אין שינוי לחתימות ציבוריות; אין מיגרציית-נתונים. provenance של בלוקים *קיימים* לא נאכף +רטרואקטיבית (forward-only) — תואם FU-1/FU-2a. + +## 4. GAP-18 — audit על כל פעולה משנה-מצב + +`log_action_safe(action, case_id=, document_id=, details=, user=)` — עטיפת `log_action` ב-try/except +(כשל → `logger.warning`, ה-action ממשיך). נקודות-הקריאה: + +| פעולה | action | details | +|-------|--------|---------| +| document_upload | `"document_upload"` | `{title, doc_type, classification}` | +| extract_claims | `"extract_claims"` | `{docs_processed, claims_count}` | +| write_block (GAP-19) | `"write_block"` | `{decision_id, block_id, model_used, generation_type, source_document_ids, retrieved_case_law_ids, claim_ids}` | +| export_docx | `"export_docx"` | `{path, file_size, block_count}` | + +## 5. GAP-19 — provenance בלוק→מקורות + +`write_block` כבר אוסף הקשר מ-`_build_source_context` (document chunks), `_build_precedents_context` +(`para_results`/`caselaw_rows` → `case_law_id`s), `_build_claims_context` (claim ids). היעד: לאסוף את +המזהים הללו ל-dict `sources = {document_ids, case_law_ids, claim_ids}` ולכלול אותו ברשומת ה-audit +`write_block` (§4). כך `audit_log` עונה "מאיזו פסיקה/מסמך נולד הבלוק" — בלי עמודה/טבלה חדשה. +מפתח-הקישור: `details.decision_id`+`details.block_id` (audit_log עצמו keyed ב-case_id/document_id). + +## 6. GAP-20 — ציטוט→קורפוס נאכף + +`resolve_citation_case_law_ids(ids) -> {resolved: [...], unresolved: [...]}` — בדיקת `EXISTS` מול +`case_law`. בנקודת ה-QA (לפני export, משתלב עם שערי FU-6): לאסוף את כל ה-`case_law_id` מציטוטי-הבלוקים +(`decision_paragraphs.citations` אם מאוכלס, אחרת מ-provenance של §5), ולהריץ resolve. בלתי-פתירים → +**ממצא-QA (warning, לא חוסם-קריטי)** + audit `citation_unresolved`. אכיפה מבנית בלבד (case_law_id), +לא חילוץ-NLP של ציטוט חופשי. + +> **הערה:** `decision_paragraphs` אינו מאוכלס כיום ע"י אף כלי (ממצא Explore). לכן ולידציית-הציטוט +> פועלת על ה-`case_law_id`s שנרשמו ב-provenance (§5); אם/כאשר decision_paragraphs יאוכלס — אותה +> ולידציה חלה עליו. זה שומר את ה-GAP סגור בלי לבנות צינור-ציטוטים חדש (מחוץ-להיקף). + +## 7. GAP-17 — drift בין DOCX-חי לבלוקים + +מקור-אמת = `decision_blocks` (INV-EX1). אחרי `revise_draft`/`apply_user_edit` שהופכים את +`active_draft_path` ל-SoT-בפועל בלי re-sync, מסמנים `cases.blocks_stale=true` (חוזה מפורש: "הבלוקים +ידועים כלא-מסונכרנים מול ה-DOCX-החי"). `export_docx` מ-blocks מאפס `blocks_stale=false` (הבלוקים שוב SoT). +health-check חושף `cases_with_stale_blocks`. **לא** מבצעים reparse DOCX→blocks (שביר). + +| נקודה | פעולה על blocks_stale | +|-------|------------------------| +| revise_draft / apply_user_edit | `= true` (DOCX-חי חרג מהבלוקים) | +| export_docx (מ-blocks) | `= false` (בלוקים = SoT שוב) | +| write_block / save_block_content | `= false` (בלוק עודכן ב-DB) | + +## 8. שינויי-התנהגות וסיכון + +| שינוי | השפעה | סיכון | +|--------|--------|--------| +| audit על 4 פעולות | audit_log מתמלא; observability | נמוך — non-fatal, לא משנה תוצאת-פעולה | +| provenance ב-write_block audit | רשומת מקור לכל גנרציה חדשה | נמוך — forward-only; בלוקים קיימים לא מושפעים | +| ציטוט-QA warning | ציטוט בלתי-פתיר מסומן לאימות-יו"ר | נמוך — warning, לא חוסם export (לא קריטי) | +| `blocks_stale` flag | חשיפת drift; אינו חוסם | נמוך — דגל אינפורמטיבי; V22 additive | + +## 9. אסטרטגיית בדיקה + +`tests/test_audit_provenance.py` — offline, monkeypatch DB pool. מקרים: +1. `log_action_safe` בולע כשל-DB (warning) ולא מרים. +2. כל אחת מ-4 הפעולות קוראת ל-audit עם ה-action הנכון (monkeypatch log_action, assert call). +3. write_block audit כולל `source_document_ids`/`retrieved_case_law_ids` מה-context. +4. `resolve_citation_case_law_ids`: מפריד resolved/unresolved נכון (monkeypatch EXISTS). +5. ציטוט בלתי-פתיר → ממצא-QA warning (לא חוסם-קריטי). +6. `blocks_stale`: revise/apply → true; export-from-blocks → false. +7. health-check חושף `cases_with_stale_blocks`. + +> בדיקות-DB אמיתיות (audit_log INSERT, V22, EXISTS) — smoke מול DB מקומי (5433) בסיום, כמו FU-2a. + +## 10. סדר-ביצוע + +1. בדיקות אדומות. +2. `log_action_safe` + מיגרציה V22 (`blocks_stale`) + helpers (`mark_blocks_stale`, `resolve_citation_case_law_ids`). +3. GAP-18: 4 קריאות audit (upload, extract_claims, export_docx + write_block בסיס). +4. GAP-19: איסוף source ids ב-write_block → provenance ב-audit. +5. GAP-20: ולידציית-ציטוט ב-QA + audit `citation_unresolved`. +6. GAP-17: `blocks_stale` ב-revise/apply/export/write_block + health-check. +7. בדיקות ירוקות + smoke מול DB + lint + TaskMaster. From 2994a884e9bdfceb887fa298e6880e4358057a95 Mon Sep 17 00:00:00 2001 From: Chaim Date: Sat, 30 May 2026 21:26:30 +0000 Subject: [PATCH 02/10] docs(plan): FU-7 audit-trail + provenance implementation plan (7 tasks, TDD) Co-Authored-By: Claude Opus 4.8 (1M context) --- .../plans/2026-05-30-fu7-audit-provenance.md | 521 ++++++++++++++++++ 1 file changed, 521 insertions(+) create mode 100644 docs/superpowers/plans/2026-05-30-fu7-audit-provenance.md diff --git a/docs/superpowers/plans/2026-05-30-fu7-audit-provenance.md b/docs/superpowers/plans/2026-05-30-fu7-audit-provenance.md new file mode 100644 index 0000000..396101d --- /dev/null +++ b/docs/superpowers/plans/2026-05-30-fu7-audit-provenance.md @@ -0,0 +1,521 @@ +# FU-7: Audit-Trail + Provenance — Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Turn `audit_log` into an end-to-end audit trail, attach source-provenance to generated blocks, enforce citation→corpus resolution, and flag DOCX↔blocks drift — all forward-only, no data migration. + +**Architecture:** Reuse `audit_log.log_action` with a `details` JSONB payload (X5 §4 — no new table) via a non-fatal `log_action_safe` wrapper. Provenance is an append-only `write_block` audit event carrying the source ids that fed the generation. GAP-17 drift is a deterministic `cases.blocks_stale` flag (V22) set at the known divergence points + a health-check count — not a fragile DOCX→blocks reparse. GAP-20 is a structural `case_law_id` resolver surfaced as a QA warning. + +**Tech Stack:** Python 3.12, asyncpg, PostgreSQL@localhost:5433, pytest offline, `.venv` at `mcp-server/.venv`. + +**Spec:** [docs/superpowers/specs/2026-05-30-fu7-audit-provenance-design.md](../specs/2026-05-30-fu7-audit-provenance-design.md) + +**Run tests:** `cd ~/legal-ai/mcp-server && .venv/bin/python -m pytest tests/test_audit_provenance.py -v` + +--- + +## File Structure + +- **Modify** `mcp-server/src/legal_mcp/services/audit.py` — add `log_action_safe(...)`. +- **Modify** `mcp-server/src/legal_mcp/services/db.py` — V22 migration (`cases.blocks_stale`), `mark_blocks_stale`, `resolve_citation_case_law_ids`. +- **Modify** `mcp-server/src/legal_mcp/tools/documents.py` — audit in `document_upload`, `extract_claims`. +- **Modify** `mcp-server/src/legal_mcp/services/block_writer.py` — collect source ids; audit `write_block`; clear `blocks_stale` on save. +- **Modify** `mcp-server/src/legal_mcp/tools/drafting.py` — audit `export_docx`; set/clear `blocks_stale` in `export_docx`/`revise_draft`/`apply_user_edit`. +- **Modify** QA path (`services/qa_validator.py`) — citation→corpus warning. +- **Modify** `mcp-server/src/legal_mcp/services/metrics.py` — `cases_with_stale_blocks` count. +- **Create** `mcp-server/tests/test_audit_provenance.py`. + +--- + +## Task 1: Failing tests + +**Files:** Create `mcp-server/tests/test_audit_provenance.py` + +- [ ] **Step 1: Write the failing tests** + +```python +"""FU-7: audit-trail + provenance (offline, monkeypatched I/O).""" +from __future__ import annotations + +import asyncio +from uuid import uuid4 + +import pytest + +from legal_mcp.services import audit, db + + +def _run(coro): + return asyncio.run(coro) + + +# ── GAP-18: log_action_safe is non-fatal ─────────────────────────────── +def test_log_action_safe_swallows_db_error(monkeypatch): + async def _boom(*a, **k): + raise RuntimeError("db down") + monkeypatch.setattr(audit, "log_action", _boom) + # must NOT raise + _run(audit.log_action_safe("write_block", details={"x": 1})) + + +def test_log_action_safe_forwards_args(monkeypatch): + seen = {} + async def _capture(action, case_id=None, document_id=None, details=None, user="system"): + seen.update(action=action, details=details) + monkeypatch.setattr(audit, "log_action", _capture) + _run(audit.log_action_safe("export_docx", details={"path": "/x"})) + assert seen["action"] == "export_docx" and seen["details"] == {"path": "/x"} + + +# ── GAP-20: structural citation resolver ──────────────────────────────── +def test_resolve_citation_case_law_ids_splits(monkeypatch): + good = uuid4() + bad = uuid4() + + class _Conn: + async def fetchval(self, q, cid): + return cid == good + async def __aenter__(self): return self + async def __aexit__(self, *a): return False + + class _Pool: + def acquire(self): return _Conn() + + async def _pool(): + return _Pool() + monkeypatch.setattr(db, "get_pool", _pool) + + out = _run(db.resolve_citation_case_law_ids([good, bad])) + assert good in out["resolved"] and bad in out["unresolved"] + + +# ── GAP-17: blocks_stale helper ──────────────────────────────────────── +def test_mark_blocks_stale_executes_update(monkeypatch): + seen = {} + + class _Conn: + async def execute(self, q, *a): + seen["q"] = q; seen["args"] = a + async def __aenter__(self): return self + async def __aexit__(self, *a): return False + + class _Pool: + def acquire(self): return _Conn() + + async def _pool(): return _Pool() + monkeypatch.setattr(db, "get_pool", _pool) + + cid = uuid4() + _run(db.mark_blocks_stale(cid, True)) + assert "blocks_stale" in seen["q"] and seen["args"][0] is True and seen["args"][1] == cid +``` + +- [ ] **Step 2: Run to verify failure** + +Run: `cd ~/legal-ai/mcp-server && .venv/bin/python -m pytest tests/test_audit_provenance.py -v` +Expected: FAIL — `AttributeError: ... has no attribute 'log_action_safe'` / `resolve_citation_case_law_ids` / `mark_blocks_stale`. + +- [ ] **Step 3: Commit** + +```bash +cd ~/legal-ai +git add mcp-server/tests/test_audit_provenance.py +git commit -m "test(audit): failing tests for audit-trail + provenance (FU-7)" +``` + +--- + +## Task 2: V22 migration + core helpers + +**Files:** Modify `mcp-server/src/legal_mcp/services/audit.py`, `mcp-server/src/legal_mcp/services/db.py` + +- [ ] **Step 1: Add `log_action_safe` to audit.py (after `log_action`)** + +```python +async def log_action_safe( + action: str, + case_id: "UUID | None" = None, + document_id: "UUID | None" = None, + details: dict | None = None, + user: str = "system", +) -> None: + """Non-fatal audit: never let an audit-log failure break the caller's action. + + The authoritative integrity trail is git (X5 §2.1); audit_log is the + 'who/what/when' observability layer, so a write failure is logged as a + warning and swallowed. + """ + try: + await log_action(action, case_id=case_id, document_id=document_id, + details=details, user=user) + except Exception as e: # noqa: BLE001 — observability must not break the op + logger.warning("audit log_action failed (non-fatal) for %s: %s", action, e) +``` + +- [ ] **Step 2: Add `SCHEMA_V22_SQL` after `SCHEMA_V21_SQL` in db.py + wire it** + +READ db.py near `SCHEMA_V21_SQL` (~line 1097-1133). Add after the V21 block: + +```python +# ── V22: cases.blocks_stale — DOCX↔blocks drift flag (GAP-17 / INV-EX1) ── +# Set true when revise_draft/apply_user_edit make active_draft_path the live +# source-of-truth without re-syncing decision_blocks; cleared when blocks are +# re-exported or re-saved. Surfaced by health-check. Source-of-truth remains +# decision_blocks — this only flags known drift (no fragile DOCX→blocks reparse). +SCHEMA_V22_SQL = """ +ALTER TABLE cases ADD COLUMN IF NOT EXISTS blocks_stale boolean NOT NULL DEFAULT false; +""" +``` +After `await conn.execute(SCHEMA_V21_SQL)` add `await conn.execute(SCHEMA_V22_SQL)` and bump the log line to `v1-v22`. + +- [ ] **Step 3: Add `mark_blocks_stale` + `resolve_citation_case_law_ids` to db.py (near the case helpers, after `get_active_draft_path`)** + +```python +async def mark_blocks_stale(case_id: UUID, stale: bool) -> None: + """Flag/clear DOCX↔blocks drift for a case (GAP-17).""" + pool = await get_pool() + async with pool.acquire() as conn: + await conn.execute( + "UPDATE cases SET blocks_stale = $1, updated_at = now() WHERE id = $2", + stale, case_id, + ) + + +async def resolve_citation_case_law_ids(ids) -> dict: + """Structural citation→corpus resolution (GAP-20 / INV-AUD3). + + Given case_law_id values referenced by a decision's citations/provenance, + split into resolvable (exist in case_law) vs unresolvable. + """ + resolved, unresolved = [], [] + pool = await get_pool() + async with pool.acquire() as conn: + for cid in ids: + try: + exists = await conn.fetchval( + "SELECT EXISTS(SELECT 1 FROM case_law WHERE id = $1)", cid) + except Exception: + exists = False + (resolved if exists else unresolved).append(cid) + return {"resolved": resolved, "unresolved": unresolved} +``` + +- [ ] **Step 4: Run Task-1 tests for these helpers** + +Run: `cd ~/legal-ai/mcp-server && .venv/bin/python -m pytest tests/test_audit_provenance.py -v` +Expected: all 4 tests PASS. + +- [ ] **Step 5: Commit** + +```bash +cd ~/legal-ai +git add mcp-server/src/legal_mcp/services/audit.py mcp-server/src/legal_mcp/services/db.py +git commit -m "feat(audit): log_action_safe + V22 blocks_stale + citation resolver (FU-7)" +``` + +--- + +## Task 3: GAP-18 — audit calls on upload / extract_claims / export + +**Files:** Modify `mcp-server/src/legal_mcp/tools/documents.py`, `mcp-server/src/legal_mcp/tools/drafting.py` + +- [ ] **Step 1: `document_upload` — audit after processing (documents.py)** + +READ `document_upload` (lines ~14-94). It computes `case_id`, `doc` (with `doc["id"]`), `actual_doc_type`, and `result` (with `result["classification"]`). Ensure `from legal_mcp.services import audit` is imported (add if missing). Immediately BEFORE the final `return json.dumps({...})`, add: + +```python + await audit.log_action_safe( + "document_upload", case_id=case_id, document_id=UUID(doc["id"]), + details={"title": title, "doc_type": actual_doc_type}, + ) +``` + +- [ ] **Step 2: `extract_claims` — audit before return (documents.py)** + +In `extract_claims` (lines ~300-348), before the final `return json.dumps(results, ...)`, add: + +```python + await audit.log_action_safe( + "extract_claims", case_id=case_id, + details={"docs_processed": len(docs), "results": len(results)}, + ) +``` + +- [ ] **Step 3: `export_docx` — audit after export (drafting.py)** + +READ `export_docx` in `drafting.py` (around lines 384-439). It resolves `case_id`, builds `path`, and calls `db.set_active_draft_path(case_id, path)`. Ensure `audit` is imported. After the `set_active_draft_path` call, add: + +```python + await audit.log_action_safe( + "export_docx", case_id=case_id, + details={"path": str(path)}, + ) +``` + +- [ ] **Step 4: Verify imports** + +Run: `cd ~/legal-ai/mcp-server && .venv/bin/python -c "from legal_mcp.tools import documents, drafting; print('clean')"` +Expected: `clean`. + +- [ ] **Step 5: Commit** + +```bash +cd ~/legal-ai +git add mcp-server/src/legal_mcp/tools/documents.py mcp-server/src/legal_mcp/tools/drafting.py +git commit -m "feat(audit): log document_upload/extract_claims/export_docx (GAP-18, FU-7)" +``` + +--- + +## Task 4: GAP-19 — block→source provenance + +**Files:** Modify `mcp-server/src/legal_mcp/services/block_writer.py` + +- [ ] **Step 1: Make `_build_precedents_context` also return the case_law ids it used** + +READ `_build_precedents_context` (lines ~671-716). Change the `caselaw_rows` SELECT to also fetch `cl.id`: +replace `"""SELECT cl.case_number, cl.case_name, cl.court, cl.summary, cl.key_quote,` with +`"""SELECT cl.id, cl.case_number, cl.case_name, cl.court, cl.summary, cl.key_quote,`. +Collect ids and change the function to return a tuple. At the function's two `return` points: +- replace `return "\n\n".join(parts) if parts else "(אין תקדימים)"` with + `return ("\n\n".join(parts) if parts else "(אין תקדימים)"), case_law_ids` +- ensure `case_law_ids = []` is initialized at the top, and inside the caselaw loop append `r["id"]` (str(r["id"])). + +If there is an early/exception return path that returns a bare string, make it return `("(אין תקדימים)", [])` too. + +- [ ] **Step 2: Update the caller in `write_block` + collect document/claim ids** + +READ `write_block` (lines ~280-394). Line ~321 currently: +`precedents_context = await _build_precedents_context(case_id, block_id)` +Change to: +`precedents_context, _precedent_case_law_ids = await _build_precedents_context(case_id, block_id)` + +Add a helper `_collect_block_sources` (after `_build_result`, ~line 408): + +```python +async def _collect_block_sources(case_id: UUID, block_id: str) -> dict: + """Deterministic source ids available to a block's generation (GAP-19). + + document_ids: case documents matching the block's allowed doc-types. + claim_ids: extracted claims for the case. (case_law_ids are captured + separately from the precedent search inside write_block.) + """ + allowed = _BLOCK_DOC_TYPES.get(block_id, []) + docs = await db.list_documents(case_id) + if allowed: + docs = [d for d in docs if d.get("doc_type") in allowed] + claims = await db.get_claims(case_id) + return { + "document_ids": [str(d["id"]) for d in docs], + "claim_ids": [str(c["id"]) for c in claims], + } +``` + +In `write_block`, just before the final `return _build_result(block_id, content, block_cfg)` (the non-template path, ~line 394), build the sources and attach to the result: + +```python + sources = await _collect_block_sources(case_id, block_id) + sources["case_law_ids"] = _precedent_case_law_ids + result = _build_result(block_id, content, block_cfg) + result["sources"] = sources + return result +``` + +(For the template path return at ~line 308, attach an empty sources dict: `r = _build_result(...); r["sources"] = {"document_ids": [], "claim_ids": [], "case_law_ids": []}; return r`.) + +- [ ] **Step 3: Write the provenance audit in `write_and_store_block` and `save_block_content`** + +In `write_and_store_block` (~line 1039), after `await store_block(UUID(decision["id"]), result)`, add: + +```python + await audit.log_action_safe( + "write_block", case_id=case_id, + details={ + "decision_id": str(decision["id"]), + "block_id": block_id, + "model_used": result.get("model_used"), + "generation_type": result.get("generation_type"), + "sources": result.get("sources", {}), + }, + ) + await db.mark_blocks_stale(case_id, False) +``` + +In `save_block_content` (~line 905), after `await store_block(...)` add the same `mark_blocks_stale(case_id, False)` (a saved block means DB blocks are current). Ensure `from legal_mcp.services import audit` is imported in block_writer.py (add if missing). + +- [ ] **Step 4: Smoke-import + targeted check** + +Run: `cd ~/legal-ai/mcp-server && .venv/bin/python -c "from legal_mcp.services import block_writer; print('clean')"` +Expected: `clean`. + +- [ ] **Step 5: Commit** + +```bash +cd ~/legal-ai +git add mcp-server/src/legal_mcp/services/block_writer.py +git commit -m "feat(audit): block→source provenance via write_block audit event (GAP-19, FU-7)" +``` + +--- + +## Task 5: GAP-20 — citation→corpus validation as QA warning + +**Files:** Modify `mcp-server/src/legal_mcp/services/qa_validator.py` + +- [ ] **Step 1: Read the QA validator structure** + +READ `mcp-server/src/legal_mcp/services/qa_validator.py` — find the function that runs the QA checks and returns findings (look for the list of checks / findings dicts with severity like `warning`/`critical`). Identify the findings structure (keys, how a check is appended). + +- [ ] **Step 2: Add a citation-resolution check** + +Add a check that gathers `case_law_id`s referenced by the decision's provenance/citations and resolves them. Concretely, add a function in qa_validator.py: + +```python +async def _check_citation_resolution(case_id, decision_id) -> list[dict]: + """GAP-20/INV-AUD3: every cited case_law_id must resolve to the corpus. + + Reads case_law_ids from the decision's write_block audit provenance + (audit_log details.sources.case_law_ids) and verifies each resolves. + Unresolvable ids → non-blocking warning + audit('citation_unresolved'). + """ + from legal_mcp.services import db, audit + from uuid import UUID + rows = await audit.get_audit_log(case_id=case_id, action="write_block", limit=200) + ids = set() + for r in rows: + details = r.get("details") or {} + if isinstance(details, str): + import json as _json + try: details = _json.loads(details) + except (ValueError, TypeError): details = {} + for raw in (details.get("sources") or {}).get("case_law_ids", []): + try: ids.add(UUID(str(raw))) + except (ValueError, TypeError): pass + if not ids: + return [] + res = await db.resolve_citation_case_law_ids(list(ids)) + findings = [] + if res["unresolved"]: + await audit.log_action_safe( + "citation_unresolved", case_id=case_id, + details={"unresolved": [str(x) for x in res["unresolved"]]}, + ) + findings.append({ + "check": "citation_resolution", + "severity": "warning", + "passed": False, + "message": f"{len(res['unresolved'])} ציטוטים אינם פתירים לקורפוס — דורש אימות יו\"ר", + }) + return findings +``` + +Then wire `_check_citation_resolution` into the validator's main run function so its findings are appended to the result list (match the existing findings shape — adjust the dict keys to the validator's actual schema discovered in Step 1). It must be a **warning**, never a critical gate (does not block export). + +- [ ] **Step 3: Smoke-import** + +Run: `cd ~/legal-ai/mcp-server && .venv/bin/python -c "from legal_mcp.services import qa_validator; print('clean')"` +Expected: `clean`. + +- [ ] **Step 4: Commit** + +```bash +cd ~/legal-ai +git add mcp-server/src/legal_mcp/services/qa_validator.py +git commit -m "feat(qa): citation→corpus resolution as non-blocking warning (GAP-20, FU-7)" +``` + +--- + +## Task 6: GAP-17 — blocks_stale wiring + health-check + +**Files:** Modify `mcp-server/src/legal_mcp/tools/drafting.py`, `mcp-server/src/legal_mcp/services/metrics.py` + +- [ ] **Step 1: Set `blocks_stale=true` in `revise_draft` and `apply_user_edit`** + +READ `revise_draft` (~647-733) and `apply_user_edit` (~569-613) in drafting.py. Each ends by calling `db.set_active_draft_path(case_id, ...)`. Immediately after that call in EACH function, add: + +```python + await db.mark_blocks_stale(case_id, True) +``` + +- [ ] **Step 2: Clear `blocks_stale=false` in `export_docx`** + +In `export_docx` (after the `set_active_draft_path` + the audit added in Task 3), add: + +```python + await db.mark_blocks_stale(case_id, False) +``` +(export_docx renders FROM the blocks, so the DOCX matches blocks → not stale.) + +- [ ] **Step 3: Health-check count in metrics.py** + +READ `mcp-server/src/legal_mcp/services/metrics.py` — find the aggregation that already runs counts (the one FU-2a added `non_searchable_case_law` to). Add a sibling count: + +```python + cases_with_stale_blocks = await conn.fetchval( + "SELECT COUNT(*) FROM cases WHERE blocks_stale") +``` +and expose it in the returned summary dict as `"cases_with_stale_blocks": cases_with_stale_blocks`. + +- [ ] **Step 4: Smoke-import** + +Run: `cd ~/legal-ai/mcp-server && .venv/bin/python -c "from legal_mcp.tools import drafting; from legal_mcp.services import metrics; print('clean')"` +Expected: `clean`. + +- [ ] **Step 5: Commit** + +```bash +cd ~/legal-ai +git add mcp-server/src/legal_mcp/tools/drafting.py mcp-server/src/legal_mcp/services/metrics.py +git commit -m "feat(audit): blocks_stale drift flag + health-check visibility (GAP-17, FU-7)" +``` + +--- + +## Task 7: Full suite + DB smoke + lint + TaskMaster + +- [ ] **Step 1: Full offline suite** + +Run: `cd ~/legal-ai/mcp-server && .venv/bin/python -m pytest tests/ -q` +Expected: all pass (FU-1/2a + new FU-7 tests). Report the summary line. If a pre-existing test fails because a newly-audited function now calls `audit`/`mark_blocks_stale` without a stub, fix that test's fixture to stub the new boundary (same pattern as the FU-2a `recompute_searchable` fixture fix). + +- [ ] **Step 2: DB smoke (real Postgres — applies V22, exercises helpers)** + +```bash +cd ~/legal-ai && set -a && source ~/.env 2>/dev/null && set +a +cd mcp-server && .venv/bin/python -c " +import asyncio, uuid +from legal_mcp.services import db, audit +async def main(): + await db.get_pool() # applies V22 + pool = await db.get_pool() + async with pool.acquire() as c: + col = await c.fetchval(\"SELECT 1 FROM information_schema.columns WHERE table_name='cases' AND column_name='blocks_stale'\") + print('V22 blocks_stale present:', bool(col)) + # citation resolver: random id is unresolved + out = await db.resolve_citation_case_law_ids([uuid.uuid4()]) + print('resolver unresolved count:', len(out['unresolved'])) + # log_action_safe never raises + await audit.log_action_safe('fu7_smoke', details={'ok': True}) + print('log_action_safe ok') +asyncio.run(main()) +" 2>&1 | grep -vE 'INFO|WARNING|httpx|deprecat|command not found|\^\^\^' | tail -5 +``` +Expected: `V22 blocks_stale present: True`, `resolver unresolved count: 1`, `log_action_safe ok`. (Optionally clean the smoke row: `DELETE FROM audit_log WHERE action='fu7_smoke'`.) + +- [ ] **Step 3: Lint** + +Run: `cd ~/legal-ai/mcp-server && .venv/bin/python -m ruff check src/legal_mcp/services/audit.py src/legal_mcp/services/db.py src/legal_mcp/services/block_writer.py 2>/dev/null; echo "exit=$?"` +Expected: clean or "ruff not available". + +- [ ] **Step 4: Mark TaskMaster #65 done** — controller edits `.taskmaster/tasks/tasks.json` + verifies via MCP get_task. + +--- + +## Self-Review Notes + +- **GAP-18** → Task 3 (+ write_block audit in Task 4). **GAP-19** → Task 4 (provenance event). **GAP-20** → Task 5 (resolver + QA warning). **GAP-17** → Tasks 2+6 (V22 flag + wiring + health). +- **No new table** (audit_log reused, X5 §4). **No data migration** (V22 additive; provenance forward-only). +- **Non-fatal audit:** all calls via `log_action_safe`. **GAP-20 is warning-only** (never a critical gate — doesn't block export, consistent with FU-6 gates). +- **Type consistency:** `log_action_safe`, `mark_blocks_stale(case_id, stale)`, `resolve_citation_case_law_ids(ids)->{resolved,unresolved}`, `result["sources"]={document_ids,claim_ids,case_law_ids}` — names identical across tasks + tests. +- **Offline-test limit:** real audit_log INSERT / V22 verified by Task 7 Step 2 smoke; offline tests cover the pure wrappers/resolver logic. From bffd2ec7018183afa6f16a2d7a84c7b5f4aca518 Mon Sep 17 00:00:00 2001 From: Chaim Date: Sat, 30 May 2026 21:27:54 +0000 Subject: [PATCH 03/10] test(audit): failing tests for audit-trail + provenance (FU-7) --- mcp-server/tests/test_audit_provenance.py | 74 +++++++++++++++++++++++ 1 file changed, 74 insertions(+) create mode 100644 mcp-server/tests/test_audit_provenance.py diff --git a/mcp-server/tests/test_audit_provenance.py b/mcp-server/tests/test_audit_provenance.py new file mode 100644 index 0000000..f1c8073 --- /dev/null +++ b/mcp-server/tests/test_audit_provenance.py @@ -0,0 +1,74 @@ +"""FU-7: audit-trail + provenance (offline, monkeypatched I/O).""" +from __future__ import annotations + +import asyncio +from uuid import uuid4 + +import pytest + +from legal_mcp.services import audit, db + + +def _run(coro): + return asyncio.run(coro) + + +# ── GAP-18: log_action_safe is non-fatal ─────────────────────────────── +def test_log_action_safe_swallows_db_error(monkeypatch): + async def _boom(*a, **k): + raise RuntimeError("db down") + monkeypatch.setattr(audit, "log_action", _boom) + # must NOT raise + _run(audit.log_action_safe("write_block", details={"x": 1})) + + +def test_log_action_safe_forwards_args(monkeypatch): + seen = {} + async def _capture(action, case_id=None, document_id=None, details=None, user="system"): + seen.update(action=action, details=details) + monkeypatch.setattr(audit, "log_action", _capture) + _run(audit.log_action_safe("export_docx", details={"path": "/x"})) + assert seen["action"] == "export_docx" and seen["details"] == {"path": "/x"} + + +# ── GAP-20: structural citation resolver ──────────────────────────────── +def test_resolve_citation_case_law_ids_splits(monkeypatch): + good = uuid4() + bad = uuid4() + + class _Conn: + async def fetchval(self, q, cid): + return cid == good + async def __aenter__(self): return self + async def __aexit__(self, *a): return False + + class _Pool: + def acquire(self): return _Conn() + + async def _pool(): + return _Pool() + monkeypatch.setattr(db, "get_pool", _pool) + + out = _run(db.resolve_citation_case_law_ids([good, bad])) + assert good in out["resolved"] and bad in out["unresolved"] + + +# ── GAP-17: blocks_stale helper ──────────────────────────────────────── +def test_mark_blocks_stale_executes_update(monkeypatch): + seen = {} + + class _Conn: + async def execute(self, q, *a): + seen["q"] = q; seen["args"] = a + async def __aenter__(self): return self + async def __aexit__(self, *a): return False + + class _Pool: + def acquire(self): return _Conn() + + async def _pool(): return _Pool() + monkeypatch.setattr(db, "get_pool", _pool) + + cid = uuid4() + _run(db.mark_blocks_stale(cid, True)) + assert "blocks_stale" in seen["q"] and seen["args"][0] is True and seen["args"][1] == cid From a121f79d6a732445d90bd2c6bd6aedb5677cca07 Mon Sep 17 00:00:00 2001 From: Chaim Date: Sat, 30 May 2026 21:29:26 +0000 Subject: [PATCH 04/10] feat(audit): log_action_safe + V22 blocks_stale + citation resolver (FU-7) Co-Authored-By: Claude Sonnet 4.6 --- mcp-server/src/legal_mcp/services/audit.py | 20 +++++++++++ mcp-server/src/legal_mcp/services/db.py | 42 +++++++++++++++++++++- 2 files changed, 61 insertions(+), 1 deletion(-) diff --git a/mcp-server/src/legal_mcp/services/audit.py b/mcp-server/src/legal_mcp/services/audit.py index f1e648a..9992499 100644 --- a/mcp-server/src/legal_mcp/services/audit.py +++ b/mcp-server/src/legal_mcp/services/audit.py @@ -44,6 +44,26 @@ async def log_action( json.dumps(details or {}, ensure_ascii=False)[:200]) +async def log_action_safe( + action: str, + case_id: "UUID | None" = None, + document_id: "UUID | None" = None, + details: dict | None = None, + user: str = "system", +) -> None: + """Non-fatal audit: never let an audit-log failure break the caller's action. + + The authoritative integrity trail is git (X5 §2.1); audit_log is the + 'who/what/when' observability layer, so a write failure is logged as a + warning and swallowed. + """ + try: + await log_action(action, case_id=case_id, document_id=document_id, + details=details, user=user) + except Exception as e: # noqa: BLE001 — observability must not break the op + logger.warning("audit log_action failed (non-fatal) for %s: %s", action, e) + + async def get_audit_log( case_id: UUID | None = None, action: str | None = None, diff --git a/mcp-server/src/legal_mcp/services/db.py b/mcp-server/src/legal_mcp/services/db.py index 632746e..0db67a0 100644 --- a/mcp-server/src/legal_mcp/services/db.py +++ b/mcp-server/src/legal_mcp/services/db.py @@ -1106,6 +1106,16 @@ CREATE INDEX IF NOT EXISTS idx_case_law_searchable ON case_law (searchable); """ +# ── V22: cases.blocks_stale — DOCX↔blocks drift flag (GAP-17 / INV-EX1) ── +# Set true when revise_draft/apply_user_edit make active_draft_path the live +# source-of-truth without re-syncing decision_blocks; cleared when blocks are +# re-exported or re-saved. Surfaced by health-check. Source-of-truth remains +# decision_blocks — this only flags known drift (no fragile DOCX→blocks reparse). +SCHEMA_V22_SQL = """ +ALTER TABLE cases ADD COLUMN IF NOT EXISTS blocks_stale boolean NOT NULL DEFAULT false; +""" + + async def _run_schema_migrations(pool: asyncpg.Pool) -> None: async with pool.acquire() as conn: await conn.execute(SCHEMA_SQL) @@ -1130,7 +1140,8 @@ async def _run_schema_migrations(pool: asyncpg.Pool) -> None: await conn.execute(SCHEMA_V19_SQL) await conn.execute(SCHEMA_V20_SQL) await conn.execute(SCHEMA_V21_SQL) - logger.info("Database schema initialized (v1-v21)") + await conn.execute(SCHEMA_V22_SQL) + logger.info("Database schema initialized (v1-v22)") async def init_schema() -> None: @@ -1206,6 +1217,35 @@ async def get_active_draft_path(case_id: UUID) -> str | None: return row["active_draft_path"] if row else None +async def mark_blocks_stale(case_id: UUID, stale: bool) -> None: + """Flag/clear DOCX↔blocks drift for a case (GAP-17).""" + pool = await get_pool() + async with pool.acquire() as conn: + await conn.execute( + "UPDATE cases SET blocks_stale = $1, updated_at = now() WHERE id = $2", + stale, case_id, + ) + + +async def resolve_citation_case_law_ids(ids) -> dict: + """Structural citation→corpus resolution (GAP-20 / INV-AUD3). + + Given case_law_id values referenced by a decision's citations/provenance, + split into resolvable (exist in case_law) vs unresolvable. + """ + resolved, unresolved = [], [] + pool = await get_pool() + async with pool.acquire() as conn: + for cid in ids: + try: + exists = await conn.fetchval( + "SELECT EXISTS(SELECT 1 FROM case_law WHERE id = $1)", cid) + except Exception: + exists = False + (resolved if exists else unresolved).append(cid) + return {"resolved": resolved, "unresolved": unresolved} + + def _normalize_case_number(s: str) -> str: """Canonicalise a case number for tolerant lookup. From 1f483383b9158c45380302513b07e3ce0c640fa2 Mon Sep 17 00:00:00 2001 From: Chaim Date: Sat, 30 May 2026 21:31:09 +0000 Subject: [PATCH 05/10] feat(audit): log document_upload/extract_claims/export_docx (GAP-18, FU-7) Co-Authored-By: Claude Sonnet 4.6 --- mcp-server/src/legal_mcp/tools/documents.py | 10 +++++++++- mcp-server/src/legal_mcp/tools/drafting.py | 6 +++++- 2 files changed, 14 insertions(+), 2 deletions(-) diff --git a/mcp-server/src/legal_mcp/tools/documents.py b/mcp-server/src/legal_mcp/tools/documents.py index 802ac6f..7b81eba 100644 --- a/mcp-server/src/legal_mcp/tools/documents.py +++ b/mcp-server/src/legal_mcp/tools/documents.py @@ -8,7 +8,7 @@ from pathlib import Path from uuid import UUID from legal_mcp import config -from legal_mcp.services import db, git_sync, processor +from legal_mcp.services import audit, db, git_sync, processor async def document_upload( @@ -87,6 +87,10 @@ async def document_upload( except Exception: pass # git not available in container — non-critical + await audit.log_action_safe( + "document_upload", case_id=case_id, document_id=UUID(doc["id"]), + details={"title": title, "doc_type": actual_doc_type}, + ) return json.dumps({ "document": doc, "processing": result, @@ -344,6 +348,10 @@ async def extract_claims( ) results.append(result) + await audit.log_action_safe( + "extract_claims", case_id=case_id, + details={"docs_processed": len(docs), "results": len(results)}, + ) return json.dumps(results, default=str, ensure_ascii=False, indent=2) diff --git a/mcp-server/src/legal_mcp/tools/drafting.py b/mcp-server/src/legal_mcp/tools/drafting.py index 81dd7ec..78f9e1f 100644 --- a/mcp-server/src/legal_mcp/tools/drafting.py +++ b/mcp-server/src/legal_mcp/tools/drafting.py @@ -7,7 +7,7 @@ from pathlib import Path from uuid import UUID from legal_mcp import config -from legal_mcp.services import db, embeddings, git_sync, research_md +from legal_mcp.services import audit, db, embeddings, git_sync, research_md from legal_mcp.services.lessons import ( CITATION_GUIDANCE, DECISION_TEMPLATES, @@ -423,6 +423,10 @@ async def export_docx(case_number: str, output_path: str = "") -> str: path = await docx_exporter.export_decision(case_id, output_path or None) # Register this export as the new source of truth await db.set_active_draft_path(case_id, path) + await audit.log_action_safe( + "export_docx", case_id=case_id, + details={"path": str(path)}, + ) case_dir = config.find_case_dir(case_number) if case_dir.exists(): git_sync.commit_and_push(case_dir, f"ייצוא DOCX: {Path(path).name}") From 769f5020eba9e6a33676f6c301b1b322b9ae9ead Mon Sep 17 00:00:00 2001 From: Chaim Date: Sat, 30 May 2026 21:33:36 +0000 Subject: [PATCH 06/10] =?UTF-8?q?feat(audit):=20block=E2=86=92source=20pro?= =?UTF-8?q?venance=20via=20write=5Fblock=20audit=20event=20(GAP-19,=20FU-7?= =?UTF-8?q?)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 4.8 (1M context) --- .../src/legal_mcp/services/block_writer.py | 54 ++++++++++++++++--- 1 file changed, 46 insertions(+), 8 deletions(-) diff --git a/mcp-server/src/legal_mcp/services/block_writer.py b/mcp-server/src/legal_mcp/services/block_writer.py index bc99832..a73d992 100644 --- a/mcp-server/src/legal_mcp/services/block_writer.py +++ b/mcp-server/src/legal_mcp/services/block_writer.py @@ -19,7 +19,7 @@ from datetime import date from uuid import UUID from legal_mcp import config -from legal_mcp.services import db, embeddings, claude_session +from legal_mcp.services import db, embeddings, claude_session, audit from legal_mcp.services.lessons import get_content_checklist, get_methodology_summary logger = logging.getLogger(__name__) @@ -305,7 +305,9 @@ async def write_block( # Template blocks if block_id in TEMPLATE_WRITERS: content = TEMPLATE_WRITERS[block_id](case, decision) - return _build_result(block_id, content, block_cfg) + r = _build_result(block_id, content, block_cfg) + r["sources"] = {"document_ids": [], "claim_ids": [], "case_law_ids": []} + return r # AI-generated blocks prompt_template = BLOCK_PROMPTS.get(block_id) @@ -318,7 +320,7 @@ async def write_block( claims_context = await _build_claims_context(case_id) direction_context = _build_direction_context(decision) plans_context = await _build_plans_context(case_id) - precedents_context = await _build_precedents_context(case_id, block_id) + precedents_context, _precedent_case_law_ids = await _build_precedents_context(case_id, block_id) style_context = await _build_style_context() discussion_context = await _build_previous_blocks_context(case_id, decision) appraiser_facts_context = await _build_appraiser_facts_context(case_id) @@ -391,7 +393,11 @@ async def write_block( timeout = claude_session.LONG_TIMEOUT if model_key == "opus" else claude_session.DEFAULT_TIMEOUT content = await claude_session.query(prompt, timeout=timeout) - return _build_result(block_id, content, block_cfg) + sources = await _collect_block_sources(case_id, block_id) + sources["case_law_ids"] = _precedent_case_law_ids + result = _build_result(block_id, content, block_cfg) + result["sources"] = sources + return result def _build_result(block_id: str, content: str, block_cfg: dict) -> dict: @@ -408,6 +414,24 @@ def _build_result(block_id: str, content: str, block_cfg: dict) -> dict: } +async def _collect_block_sources(case_id: UUID, block_id: str) -> dict: + """Deterministic source ids available to a block's generation (GAP-19). + + document_ids: case documents matching the block's allowed doc-types. + claim_ids: extracted claims for the case. (case_law_ids are captured + separately from the precedent search inside write_block.) + """ + allowed = _BLOCK_DOC_TYPES.get(block_id, []) + docs = await db.list_documents(case_id) + if allowed: + docs = [d for d in docs if d.get("doc_type") in allowed] + claims = await db.get_claims(case_id) + return { + "document_ids": [str(d["id"]) for d in docs], + "claim_ids": [str(c["id"]) for c in claims], + } + + # ── Context builders ────────────────────────────────────────────── def _build_case_context(case: dict, decision: dict | None) -> str: @@ -668,9 +692,10 @@ async def _build_post_hearing_context(case_id: UUID) -> str: return "\n".join(lines) -async def _build_precedents_context(case_id: UUID, block_id: str) -> str: +async def _build_precedents_context(case_id: UUID, block_id: str) -> tuple[str, list[str]]: """Search for similar precedent paragraphs from other decisions and case law.""" parts = [] + case_law_ids: list[str] = [] try: case = await db.get_case(case_id) case_number = case.get("case_number", "") if case else "" @@ -694,7 +719,7 @@ async def _build_precedents_context(case_id: UUID, block_id: str) -> str: pool = await db.get_pool() async with pool.acquire() as conn: caselaw_rows = await conn.fetch( - """SELECT cl.case_number, cl.case_name, cl.court, cl.summary, cl.key_quote, + """SELECT cl.id, cl.case_number, cl.case_name, cl.court, cl.summary, cl.key_quote, 1 - (cle.embedding <=> $1) AS score FROM case_law_embeddings cle JOIN case_law cl ON cl.id = cle.case_law_id @@ -703,6 +728,7 @@ async def _build_precedents_context(case_id: UUID, block_id: str) -> str: query_emb, ) for r in caselaw_rows[:3]: + case_law_ids.append(str(r["id"])) text = r["key_quote"] or r["summary"] or "" if text: parts.append( @@ -713,7 +739,7 @@ async def _build_precedents_context(case_id: UUID, block_id: str) -> str: except Exception as e: logger.warning("Failed to fetch precedents: %s", e) - return "\n\n".join(parts) if parts else "(אין תקדימים)" + return ("\n\n".join(parts) if parts else "(אין תקדימים)"), case_law_ids async def _build_style_context() -> str: @@ -841,7 +867,7 @@ async def get_block_context(case_id: UUID, block_id: str, instructions: str = "" claims_context = await _build_claims_context(case_id) direction_context = _build_direction_context(decision) plans_context = await _build_plans_context(case_id) - precedents_context = await _build_precedents_context(case_id, block_id) + precedents_context, _ = await _build_precedents_context(case_id, block_id) style_context = await _build_style_context() discussion_context = await _build_previous_blocks_context(case_id, decision) appraiser_facts_context = await _build_appraiser_facts_context(case_id) @@ -920,6 +946,7 @@ async def save_block_content(case_id: UUID, block_id: str, content: str) -> dict result["model_used"] = "claude-code" await store_block(UUID(decision["id"]), result) + await db.mark_blocks_stale(case_id, False) # Also write/update the draft file on disk await _update_draft_file(case_id, UUID(decision["id"])) @@ -1049,4 +1076,15 @@ async def write_and_store_block( result = await write_block(case_id, block_id, instructions) await store_block(UUID(decision["id"]), result) + await audit.log_action_safe( + "write_block", case_id=case_id, + details={ + "decision_id": str(decision["id"]), + "block_id": block_id, + "model_used": result.get("model_used"), + "generation_type": result.get("generation_type"), + "sources": result.get("sources", {}), + }, + ) + await db.mark_blocks_stale(case_id, False) return result From 7e2f4b2872b6aa8dc0d834bcaa410050faf9d075 Mon Sep 17 00:00:00 2001 From: Chaim Date: Sat, 30 May 2026 21:35:24 +0000 Subject: [PATCH 07/10] =?UTF-8?q?feat(qa):=20citation=E2=86=92corpus=20res?= =?UTF-8?q?olution=20as=20non-blocking=20warning=20(GAP-20,=20FU-7)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 4.8 (1M context) --- .../src/legal_mcp/services/qa_validator.py | 46 +++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/mcp-server/src/legal_mcp/services/qa_validator.py b/mcp-server/src/legal_mcp/services/qa_validator.py index 8b90040..7008daf 100644 --- a/mcp-server/src/legal_mcp/services/qa_validator.py +++ b/mcp-server/src/legal_mcp/services/qa_validator.py @@ -287,6 +287,50 @@ def check_sequential_numbering(blocks: list[dict]) -> dict: } +async def check_citation_resolution(case_id: UUID, decision_id=None) -> dict: + """GAP-20/INV-AUD3: every cited case_law_id must resolve to the corpus. + + Reads case_law_ids from the decision's write_block audit provenance and + verifies each resolves. Unresolvable → NON-BLOCKING warning + audit event. + """ + from legal_mcp.services import audit + + rows = await audit.get_audit_log(case_id=case_id, action="write_block", limit=200) + ids = set() + for r in rows: + details = r.get("details") or {} + if isinstance(details, str): + try: + details = json.loads(details) + except (ValueError, TypeError): + details = {} + for raw in (details.get("sources") or {}).get("case_law_ids", []): + try: + ids.add(UUID(str(raw))) + except (ValueError, TypeError): + pass + + if not ids: + return {"name": "citation_resolution", "passed": True, "errors": [], "severity": "warning"} + + res = await db.resolve_citation_case_law_ids(list(ids)) + if not res["unresolved"]: + return {"name": "citation_resolution", "passed": True, "errors": [], "severity": "warning"} + + await audit.log_action_safe( + "citation_unresolved", case_id=case_id, + details={"unresolved": [str(x) for x in res["unresolved"]]}, + ) + return { + "name": "citation_resolution", + "passed": False, + "severity": "warning", + "errors": [ + f"{len(res['unresolved'])} ציטוטים אינם פתירים לקורפוס — דורש אימות יו\"ר", + ], + } + + # ── Main validation ─────────────────────────────────────────────── async def validate_decision(case_id: UUID) -> dict: @@ -334,6 +378,8 @@ async def validate_decision(case_id: UUID) -> dict: check_no_duplication(blocks), check_sequential_numbering(blocks), ]) + # Async, non-blocking warning: citation→corpus resolution (GAP-20/INV-AUD3) + results.append(await check_citation_resolution(case_id, decision["id"])) critical_failures = sum(1 for r in results if not r["passed"] and r["severity"] == "critical") all_passed = all(r["passed"] for r in results) From 677f29ddec56ccb39413f1be29e2b3741f7c2331 Mon Sep 17 00:00:00 2001 From: Chaim Date: Sat, 30 May 2026 21:36:56 +0000 Subject: [PATCH 08/10] feat(audit): blocks_stale drift flag + health-check visibility (GAP-17, FU-7) Co-Authored-By: Claude Sonnet 4.6 --- mcp-server/src/legal_mcp/services/metrics.py | 4 ++++ mcp-server/src/legal_mcp/tools/drafting.py | 3 +++ 2 files changed, 7 insertions(+) diff --git a/mcp-server/src/legal_mcp/services/metrics.py b/mcp-server/src/legal_mcp/services/metrics.py index c32b4f5..02243c5 100644 --- a/mcp-server/src/legal_mcp/services/metrics.py +++ b/mcp-server/src/legal_mcp/services/metrics.py @@ -126,6 +126,9 @@ async def get_dashboard() -> dict: non_searchable_case_law = await conn.fetchval( "SELECT COUNT(*) FROM case_law WHERE NOT searchable" ) + cases_with_stale_blocks = await conn.fetchval( + "SELECT COUNT(*) FROM cases WHERE blocks_stale" + ) # QA summary qa_total = await conn.fetchval("SELECT COUNT(DISTINCT case_id) FROM qa_results") @@ -158,6 +161,7 @@ async def get_dashboard() -> dict: "style_patterns": total_patterns, "case_law_entries": total_case_law, "non_searchable_case_law": non_searchable_case_law, + "cases_with_stale_blocks": cases_with_stale_blocks, }, "cases_by_status": cases_by_status, "qa": { diff --git a/mcp-server/src/legal_mcp/tools/drafting.py b/mcp-server/src/legal_mcp/tools/drafting.py index 78f9e1f..ac28015 100644 --- a/mcp-server/src/legal_mcp/tools/drafting.py +++ b/mcp-server/src/legal_mcp/tools/drafting.py @@ -427,6 +427,7 @@ async def export_docx(case_number: str, output_path: str = "") -> str: "export_docx", case_id=case_id, details={"path": str(path)}, ) + await db.mark_blocks_stale(case_id, False) case_dir = config.find_case_dir(case_number) if case_dir.exists(): git_sync.commit_and_push(case_dir, f"ייצוא DOCX: {Path(path).name}") @@ -601,6 +602,7 @@ async def apply_user_edit(case_number: str, edit_filename: str) -> str: try: retrofit_result = docx_retrofit.retrofit_bookmarks(edit_path) await db.set_active_draft_path(case_id, str(edit_path)) + await db.mark_blocks_stale(case_id, True) case_dir = config.find_case_dir(case_number) if case_dir.exists(): git_sync.commit_and_push(case_dir, f"גרסת עריכה: {edit_path.name}") @@ -714,6 +716,7 @@ async def revise_draft(case_number: str, revisions_json: str, active_path, output_path, revisions, author=author, ) await db.set_active_draft_path(case_id, str(output_path)) + await db.mark_blocks_stale(case_id, True) case_dir = config.find_case_dir(case_number) if case_dir.exists(): git_sync.commit_and_push( From d28f7b8398e30cb4f8208694d3efc85a578bb675 Mon Sep 17 00:00:00 2001 From: Chaim Date: Sat, 30 May 2026 21:37:46 +0000 Subject: [PATCH 09/10] tasks(legal-ai): mark FU-7 (#65) done; FU-2a (#60) done Co-Authored-By: Claude Opus 4.8 (1M context) --- .taskmaster/tasks/tasks.json | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/.taskmaster/tasks/tasks.json b/.taskmaster/tasks/tasks.json index 3b9896e..1e30573 100644 --- a/.taskmaster/tasks/tasks.json +++ b/.taskmaster/tasks/tasks.json @@ -2060,7 +2060,7 @@ "description": "upsert ON CONFLICT על מפתח קנוני + נרמול case_number בכתיבה (type-aware) + דגל searchable מפורש. אפס מיגרציית-נתונים.", "details": "מכסה GAP-03,06,13. מספק INV-ING2/G3/G1/ID1/DM1. severity: Critical. סוג: pure-code (schema-additive). תלוי ב-FU-1 (#59). FU-2b (#67) מטפל ב-GAP-07/08 בנפרד.", "testStrategy": "", - "status": "pending", + "status": "done", "dependencies": [ "59" ], @@ -2072,7 +2072,7 @@ "description": "קליטה חוזרת = עדכון, לא כפילות.", "dependencies": [], "details": "INV-ING2/G3", - "status": "pending", + "status": "done", "testStrategy": "", "parentId": "60" }, @@ -2082,7 +2082,7 @@ "description": "היום רק תיקון-קריאה (_normalize_case_number, db.py:1196-1211).", "dependencies": [], "details": "INV-G1/ID1", - "status": "pending", + "status": "done", "testStrategy": "", "parentId": "60" }, @@ -2092,7 +2092,7 @@ "description": "דגל 'עבר חוזה-שלמות' מובחן מ-extraction_status.", "dependencies": [], "details": "INV-DM1", - "status": "pending", + "status": "done", "testStrategy": "", "parentId": "60" } @@ -2247,7 +2247,7 @@ "description": "כתיבת audit_log בכל פעולה; קישור בלוק→קטעי-מקור; סנכרון DB אחרי עריכה; אימות citation→corpus.", "details": "מכסה GAP-17,18,19,20. מספק INV-AUD1/2/3/EX1/G9. severity: High. סוג: קוד + backfill קל. תלוי ב-FU-1. (זרע לתת-פרויקט 3/audit-provenance.)", "testStrategy": "", - "status": "pending", + "status": "done", "dependencies": [ "59" ], @@ -2259,7 +2259,7 @@ "description": "active_draft_path הופך ל'מקור-אמת', בלוקים לא מסונכרנים (db.py:189).", "dependencies": [], "details": "INV-EX1/AUD2", - "status": "pending", + "status": "done", "testStrategy": "", "parentId": "65" }, @@ -2269,7 +2269,7 @@ "description": "הטבלה קיימת אך נכתבת כמעט רק ב-case_subtype_override (cases.py:203).", "dependencies": [], "details": "INV-AUD1", - "status": "pending", + "status": "done", "testStrategy": "", "parentId": "65" }, @@ -2279,7 +2279,7 @@ "description": "decision_blocks שומר model_used אך לא אילו chunks/precedents הזינו.", "dependencies": [], "details": "INV-AUD1", - "status": "pending", + "status": "done", "testStrategy": "", "parentId": "65" }, @@ -2289,7 +2289,7 @@ "description": "decision_paragraphs.citations ללא ולידציה שכל ציטוט מתאים.", "dependencies": [], "details": "INV-AUD3", - "status": "pending", + "status": "done", "testStrategy": "", "parentId": "65" } From 9bfb912bdff66aa1af1f6514d8df3a249d058bd1 Mon Sep 17 00:00:00 2001 From: Chaim Date: Sat, 30 May 2026 21:40:42 +0000 Subject: [PATCH 10/10] fix(audit): _collect_block_sources mirrors None-doc-types (provenance accuracy, FU-7 review) Co-Authored-By: Claude Opus 4.8 (1M context) --- mcp-server/src/legal_mcp/services/block_writer.py | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/mcp-server/src/legal_mcp/services/block_writer.py b/mcp-server/src/legal_mcp/services/block_writer.py index a73d992..32677b0 100644 --- a/mcp-server/src/legal_mcp/services/block_writer.py +++ b/mcp-server/src/legal_mcp/services/block_writer.py @@ -421,10 +421,13 @@ async def _collect_block_sources(case_id: UUID, block_id: str) -> dict: claim_ids: extracted claims for the case. (case_law_ids are captured separately from the precedent search inside write_block.) """ - allowed = _BLOCK_DOC_TYPES.get(block_id, []) - docs = await db.list_documents(case_id) - if allowed: - docs = [d for d in docs if d.get("doc_type") in allowed] + allowed = _BLOCK_DOC_TYPES.get(block_id, []) # [] = all docs; None = no source docs + if allowed is None: + docs = [] # mirror _build_source_context: this block consumes no raw source docs + else: + docs = await db.list_documents(case_id) + if allowed: + docs = [d for d in docs if d.get("doc_type") in allowed] claims = await db.get_claims(case_id) return { "document_ids": [str(d["id"]) for d in docs],