feat(X11): citation-corroboration Phase 1 — the signal (no approval change) #27
@@ -0,0 +1,504 @@
|
||||
# X11 Citation Corroboration — Phase 1 (Signal) Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Build the citation-corroboration **signal** — classify each incoming citation's *treatment*, link it to the *specific halacha* it supports, and expose per-halacha corroboration — **without yet changing the approval gate** (that is Phase 2).
|
||||
|
||||
**Architecture:** A new schema version (V24) adds a `treatment` column to `precedent_internal_citations` and a `halacha_citation_corroboration` link table. A new service `corroboration.py` does three deterministic-or-LLM steps: (1) classify treatment of a citing passage via Opus 4.8 (INV-COR2), (2) match a citing passage to one halacha via pgvector cosine over `halachot.embedding` (INV-COR3), (3) aggregate distinct positive citations per halacha (INV-COR4). A read-only MCP tool reports the result with full provenance (INV-COR6). Auto-approval (INV-COR4/G10) and enrichment are **Phase 2** (separate plan).
|
||||
|
||||
**Tech Stack:** Python 3.12, asyncpg + pgvector, FastMCP (`@mcp.tool()`), `claude_session.query_json(model=, effort=)`, `embeddings.embed_texts`, pytest (offline deterministic tests mirroring `tests/test_fu2b_reconcile.py`).
|
||||
|
||||
**Spec:** [docs/spec/X11-citation-corroboration.md](../../spec/X11-citation-corroboration.md) (INV-COR1–COR6). Prerequisite (clean identity graph) is **done** (Shafer merge + corpus scan).
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- Create: `mcp-server/src/legal_mcp/services/corroboration.py` — the corroboration service (classify · match · aggregate · persist). One responsibility: turn the citation graph into per-halacha corroboration rows.
|
||||
- Modify: `mcp-server/src/legal_mcp/services/db.py` — add `SCHEMA_V24_SQL` + helpers `store_corroboration`, `list_corroboration_for_halacha`.
|
||||
- Modify: `mcp-server/src/legal_mcp/server.py` — register read-only MCP tool `halacha_corroboration`.
|
||||
- Create: `mcp-server/tests/test_corroboration.py` — offline deterministic tests (treatment parse, aggregation, independence rule).
|
||||
|
||||
**Treatment vocabulary (INV-COR2):** positive = `{followed, explained}`; negative = `{distinguished, criticized, questioned, overruled}`; neutral = `{mentioned}`. Defined once in `corroboration.py`.
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Schema V24 — treatment column + corroboration link table
|
||||
|
||||
**Files:**
|
||||
- Modify: `mcp-server/src/legal_mcp/services/db.py` (add `SCHEMA_V24_SQL` constant near the other `SCHEMA_V23_SQL`; register it in `_run_schema_migrations`, db.py:54-ish)
|
||||
|
||||
- [ ] **Step 1: Add the schema constant**
|
||||
|
||||
Add after the `SCHEMA_V23_SQL = """..."""` block:
|
||||
|
||||
```python
|
||||
SCHEMA_V24_SQL = """
|
||||
-- X11: citation corroboration (treatment + halacha-level link)
|
||||
ALTER TABLE precedent_internal_citations
|
||||
ADD COLUMN IF NOT EXISTS treatment TEXT DEFAULT '';
|
||||
|
||||
CREATE TABLE IF NOT EXISTS halacha_citation_corroboration (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
halacha_id UUID NOT NULL REFERENCES halachot(id) ON DELETE CASCADE,
|
||||
citing_case_law_id UUID REFERENCES case_law(id) ON DELETE CASCADE,
|
||||
citing_decision_id UUID REFERENCES decisions(id) ON DELETE SET NULL,
|
||||
source_citation_id UUID NOT NULL, -- the precedent_internal_citations / case_law_citations row
|
||||
treatment TEXT NOT NULL, -- followed/explained/distinguished/criticized/questioned/overruled/mentioned
|
||||
match_score NUMERIC(4,3) DEFAULT 0, -- cosine(context, rule_statement)
|
||||
match_context TEXT DEFAULT '',
|
||||
created_at TIMESTAMPTZ DEFAULT now(),
|
||||
UNIQUE (halacha_id, source_citation_id)
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_hcc_halacha ON halacha_citation_corroboration(halacha_id);
|
||||
"""
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Register it in `_run_schema_migrations`**
|
||||
|
||||
In `_run_schema_migrations` (db.py), after `await conn.execute(SCHEMA_V23_SQL)` add:
|
||||
|
||||
```python
|
||||
await conn.execute(SCHEMA_V24_SQL)
|
||||
```
|
||||
|
||||
And update the log line to `"Database schema initialized (v1-v24)"`.
|
||||
|
||||
- [ ] **Step 3: Apply + verify against the dev DB**
|
||||
|
||||
Run (from `mcp-server/`):
|
||||
```bash
|
||||
DOTENV_PATH=/home/chaim/.env DATA_DIR=/home/chaim/legal-ai/data .venv/bin/python -c "
|
||||
import asyncio; from legal_mcp.services import db
|
||||
async def m():
|
||||
pool=await db.get_pool()
|
||||
cols=await pool.fetch(\"SELECT column_name FROM information_schema.columns WHERE table_name='precedent_internal_citations' AND column_name='treatment'\")
|
||||
t=await pool.fetchval(\"SELECT to_regclass('halacha_citation_corroboration')\")
|
||||
print('treatment col:', bool(cols), '| table:', t)
|
||||
asyncio.run(m())"
|
||||
```
|
||||
Expected: `treatment col: True | table: halacha_citation_corroboration`
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
```bash
|
||||
git add mcp-server/src/legal_mcp/services/db.py
|
||||
git commit -m "feat(db): V24 — citation treatment column + halacha corroboration link table (X11)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Treatment classifier (deterministic parse, unit-tested)
|
||||
|
||||
The LLM call classifies a citing passage; the **parse/validate** of its output is the deterministic, unit-tested unit (mirrors `halacha_extractor._coerce_halacha`).
|
||||
|
||||
**Files:**
|
||||
- Create: `mcp-server/src/legal_mcp/services/corroboration.py`
|
||||
- Test: `mcp-server/tests/test_corroboration.py`
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
```python
|
||||
# tests/test_corroboration.py
|
||||
from __future__ import annotations
|
||||
import pytest
|
||||
from legal_mcp.services import corroboration as cor
|
||||
|
||||
@pytest.mark.parametrize("raw,expected", [
|
||||
({"treatment": "followed"}, "followed"),
|
||||
({"treatment": "OVERRULED"}, "overruled"), # case-insensitive
|
||||
({"treatment": "bananas"}, "mentioned"), # unknown -> neutral default
|
||||
({}, "mentioned"), # missing -> neutral default
|
||||
])
|
||||
def test_coerce_treatment(raw, expected):
|
||||
assert cor._coerce_treatment(raw) == expected
|
||||
|
||||
def test_treatment_polarity():
|
||||
assert cor.is_positive("followed") and cor.is_positive("explained")
|
||||
assert cor.is_negative("distinguished") and cor.is_negative("overruled")
|
||||
assert not cor.is_positive("mentioned") and not cor.is_negative("mentioned")
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `cd mcp-server && .venv/bin/python -m pytest tests/test_corroboration.py -q`
|
||||
Expected: FAIL — `ModuleNotFoundError: corroboration` / `_coerce_treatment` undefined.
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation**
|
||||
|
||||
```python
|
||||
# src/legal_mcp/services/corroboration.py
|
||||
"""X11 citation corroboration — classify treatment, match to halacha, aggregate.
|
||||
|
||||
Phase 1: builds the SIGNAL only (no approval changes). See docs/spec/X11.
|
||||
All LLM calls go through the local claude_session bridge (Opus 4.8 @ xhigh),
|
||||
same architectural rule as the other extractors (local MCP only).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
import logging
|
||||
from legal_mcp import config
|
||||
from legal_mcp.config import parse_llm_json
|
||||
from legal_mcp.services import claude_session
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
TREATMENT_POSITIVE = {"followed", "explained"}
|
||||
TREATMENT_NEGATIVE = {"distinguished", "criticized", "questioned", "overruled"}
|
||||
TREATMENT_NEUTRAL = {"mentioned"}
|
||||
_VALID_TREATMENT = TREATMENT_POSITIVE | TREATMENT_NEGATIVE | TREATMENT_NEUTRAL
|
||||
|
||||
def is_positive(t: str) -> bool: return t in TREATMENT_POSITIVE
|
||||
def is_negative(t: str) -> bool: return t in TREATMENT_NEGATIVE
|
||||
|
||||
def _coerce_treatment(raw: dict) -> str:
|
||||
t = str((raw or {}).get("treatment", "")).strip().lower()
|
||||
return t if t in _VALID_TREATMENT else "mentioned"
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `cd mcp-server && .venv/bin/python -m pytest tests/test_corroboration.py -q`
|
||||
Expected: PASS (3 params + polarity).
|
||||
|
||||
- [ ] **Step 5: Add the LLM classifier call (integration, not unit-tested)**
|
||||
|
||||
Append to `corroboration.py`:
|
||||
|
||||
```python
|
||||
_TREATMENT_PROMPT = """אתה משפטן בכיר. נתון ציטוט של פסק/החלטה קודמים בתוך החלטה מאוחרת.
|
||||
סווג כיצד ההחלטה המאוחרת **מטפלת** בתקדים המצוטט, לפי אחת מהקטגוריות:
|
||||
- followed — אימצה והחילה את ההלכה.
|
||||
- explained — הסבירה/הזכירה בלי לחלוק.
|
||||
- distinguished — אבחנה (קבעה שלא חל בנסיבות).
|
||||
- criticized — מתחה ביקורת בלי לבטל.
|
||||
- questioned — הטילה ספק.
|
||||
- overruled — דחתה/ביטלה את ההלכה.
|
||||
- mentioned — אזכור-אגב בלי טיפול.
|
||||
החזר JSON בלבד: {"treatment": "<קטגוריה>"}.
|
||||
"""
|
||||
|
||||
async def classify_treatment(cited_citation: str, context: str) -> str:
|
||||
"""Return one treatment label for how `context` treats `cited_citation`."""
|
||||
user = f"תקדים מצוטט: {cited_citation}\n\n--- ההקשר המצטט ---\n{context}\n--- סוף ---"
|
||||
try:
|
||||
result = await claude_session.query_json(
|
||||
user, system=_TREATMENT_PROMPT,
|
||||
model=config.HALACHA_EXTRACT_MODEL or None,
|
||||
effort=config.HALACHA_EXTRACT_EFFORT or None,
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning("classify_treatment failed: %s", e)
|
||||
return "mentioned"
|
||||
return _coerce_treatment(result if isinstance(result, dict) else {})
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
```bash
|
||||
git add mcp-server/src/legal_mcp/services/corroboration.py mcp-server/tests/test_corroboration.py
|
||||
git commit -m "feat(corroboration): treatment classifier + polarity (INV-COR2, X11)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Halacha matcher — pgvector cosine, threshold (INV-COR3)
|
||||
|
||||
The matcher links a citing passage to the single best halacha of the cited precedent. The **threshold decision** is the deterministic unit; the pgvector lookup is integration.
|
||||
|
||||
**Files:**
|
||||
- Modify: `mcp-server/src/legal_mcp/services/corroboration.py`
|
||||
- Modify: `mcp-server/src/legal_mcp/services/db.py` (add `nearest_halacha_for_vector`)
|
||||
- Test: `mcp-server/tests/test_corroboration.py`
|
||||
|
||||
- [ ] **Step 1: Write the failing test (threshold gate is the unit)**
|
||||
|
||||
Append to `tests/test_corroboration.py`:
|
||||
|
||||
```python
|
||||
def test_match_accepts_above_threshold():
|
||||
# (halacha_id, similarity) above floor -> accepted
|
||||
assert cor.accept_match(("h1", 0.62), floor=0.50) == "h1"
|
||||
|
||||
def test_match_rejects_below_threshold():
|
||||
# below floor -> None (INV-COR3: don't attach to a different legal point)
|
||||
assert cor.accept_match(("h1", 0.41), floor=0.50) is None
|
||||
|
||||
def test_match_rejects_empty():
|
||||
assert cor.accept_match(None, floor=0.50) is None
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `cd mcp-server && .venv/bin/python -m pytest tests/test_corroboration.py::test_match_accepts_above_threshold -q`
|
||||
Expected: FAIL — `accept_match` undefined.
|
||||
|
||||
- [ ] **Step 3: Implement the threshold gate + env floor**
|
||||
|
||||
Add to `config.py` (near `HALACHA_EXTRACT_*`):
|
||||
```python
|
||||
HALACHA_CORROBORATION_MATCH_FLOOR = float(os.environ.get("HALACHA_CORROBORATION_MATCH_FLOOR", "0.50"))
|
||||
HALACHA_CORROBORATION_MIN_CITES = int(os.environ.get("HALACHA_CORROBORATION_MIN_CITES", "2"))
|
||||
```
|
||||
Add to `corroboration.py`:
|
||||
```python
|
||||
def accept_match(best: tuple[str, float] | None, floor: float = config.HALACHA_CORROBORATION_MATCH_FLOOR) -> str | None:
|
||||
"""Return the halacha_id iff similarity clears the floor (INV-COR3)."""
|
||||
if not best:
|
||||
return None
|
||||
halacha_id, sim = best
|
||||
return halacha_id if sim >= floor else None
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `cd mcp-server && .venv/bin/python -m pytest tests/test_corroboration.py -q`
|
||||
Expected: PASS (all, incl. Task 2).
|
||||
|
||||
- [ ] **Step 5: Add the pgvector lookup (integration)**
|
||||
|
||||
Add to `db.py`:
|
||||
```python
|
||||
async def nearest_halacha_for_vector(case_law_id: UUID, vec: list[float]) -> tuple[str, float] | None:
|
||||
"""Best-matching halacha of `case_law_id` for a context embedding (cosine)."""
|
||||
pool = await get_pool()
|
||||
row = await pool.fetchrow(
|
||||
"SELECT id::text AS id, 1 - (embedding <=> $2) AS sim "
|
||||
"FROM halachot WHERE case_law_id = $1 AND embedding IS NOT NULL "
|
||||
"ORDER BY embedding <=> $2 LIMIT 1",
|
||||
case_law_id, vec,
|
||||
)
|
||||
return (row["id"], float(row["sim"])) if row else None
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
```bash
|
||||
git add mcp-server/src/legal_mcp/services/corroboration.py mcp-server/src/legal_mcp/services/db.py mcp-server/src/legal_mcp/config.py mcp-server/tests/test_corroboration.py
|
||||
git commit -m "feat(corroboration): halacha matcher + cosine threshold (INV-COR3, X11)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Aggregator — distinct positive citations, negative-flag (INV-COR4)
|
||||
|
||||
Pure function: given per-citation results for one halacha, count **distinct** positive citing sources and detect any negative treatment.
|
||||
|
||||
**Files:**
|
||||
- Modify: `mcp-server/src/legal_mcp/services/corroboration.py`
|
||||
- Test: `mcp-server/tests/test_corroboration.py`
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
Append to `tests/test_corroboration.py`:
|
||||
|
||||
```python
|
||||
def _link(src, treatment):
|
||||
return {"source_id": src, "treatment": treatment}
|
||||
|
||||
def test_aggregate_counts_distinct_positive():
|
||||
links = [_link("d1","followed"), _link("d1","explained"), _link("d2","followed")]
|
||||
agg = cor.aggregate(links, min_cites=2)
|
||||
assert agg["positive_sources"] == 2 # d1 counted once (INV-COR4 independence)
|
||||
assert agg["has_negative"] is False
|
||||
assert agg["corroborated"] is True
|
||||
|
||||
def test_aggregate_negative_blocks():
|
||||
links = [_link("d1","followed"), _link("d2","followed"), _link("d3","distinguished")]
|
||||
agg = cor.aggregate(links, min_cites=2)
|
||||
assert agg["has_negative"] is True
|
||||
assert agg["corroborated"] is False # any negative -> not corroborated (INV-COR2)
|
||||
|
||||
def test_aggregate_below_threshold():
|
||||
agg = cor.aggregate([_link("d1","followed")], min_cites=2)
|
||||
assert agg["corroborated"] is False # single source insufficient (INV-COR4)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `cd mcp-server && .venv/bin/python -m pytest tests/test_corroboration.py::test_aggregate_counts_distinct_positive -q`
|
||||
Expected: FAIL — `aggregate` undefined.
|
||||
|
||||
- [ ] **Step 3: Implement**
|
||||
|
||||
Add to `corroboration.py`:
|
||||
```python
|
||||
def aggregate(links: list[dict], min_cites: int = config.HALACHA_CORROBORATION_MIN_CITES) -> dict:
|
||||
"""Aggregate per-halacha corroboration (INV-COR4/COR2).
|
||||
|
||||
links: [{"source_id": str, "treatment": str}, ...] already matched to ONE halacha.
|
||||
positive_sources = count of DISTINCT source_id whose treatment is positive.
|
||||
has_negative = any negative treatment present.
|
||||
corroborated = positive_sources >= min_cites AND not has_negative.
|
||||
"""
|
||||
positive = {l["source_id"] for l in links if is_positive(l["treatment"])}
|
||||
has_negative = any(is_negative(l["treatment"]) for l in links)
|
||||
return {
|
||||
"positive_sources": len(positive),
|
||||
"has_negative": has_negative,
|
||||
"corroborated": len(positive) >= min_cites and not has_negative,
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `cd mcp-server && .venv/bin/python -m pytest tests/test_corroboration.py -q`
|
||||
Expected: PASS (all).
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
```bash
|
||||
git add mcp-server/src/legal_mcp/services/corroboration.py mcp-server/tests/test_corroboration.py
|
||||
git commit -m "feat(corroboration): aggregator — distinct positive + negative-flag (INV-COR4, X11)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Orchestration + persistence (build the signal for one precedent)
|
||||
|
||||
Wires Tasks 2–4 over a precedent's incoming citations and stores `halacha_citation_corroboration` rows (INV-COR6 provenance). Integration (no new offline unit).
|
||||
|
||||
**Files:**
|
||||
- Modify: `mcp-server/src/legal_mcp/services/corroboration.py`
|
||||
- Modify: `mcp-server/src/legal_mcp/services/db.py` (add `store_corroboration`, `incoming_citations_for_precedent`)
|
||||
|
||||
- [ ] **Step 1: Add DB helpers**
|
||||
|
||||
```python
|
||||
# db.py
|
||||
async def incoming_citations_for_precedent(case_law_id: UUID) -> list[dict]:
|
||||
"""All incoming citations (both graphs) with their context + source id."""
|
||||
pool = await get_pool()
|
||||
rows = await pool.fetch(
|
||||
"SELECT id::text AS source_id, source_case_law_id::text AS citing_case_law_id, "
|
||||
" NULL::text AS citing_decision_id, match_context AS context "
|
||||
"FROM precedent_internal_citations WHERE cited_case_law_id = $1 "
|
||||
"UNION ALL "
|
||||
"SELECT id::text, NULL, decision_id::text, context_text "
|
||||
"FROM case_law_citations WHERE case_law_id = $1",
|
||||
case_law_id,
|
||||
)
|
||||
return [dict(r) for r in rows]
|
||||
|
||||
async def store_corroboration(halacha_id: str, source_id: str, citing_case_law_id, citing_decision_id, treatment: str, score: float, context: str) -> None:
|
||||
pool = await get_pool()
|
||||
await pool.execute(
|
||||
"INSERT INTO halacha_citation_corroboration "
|
||||
"(halacha_id, citing_case_law_id, citing_decision_id, source_citation_id, treatment, match_score, match_context) "
|
||||
"VALUES ($1,$2,$3,$4,$5,$6,$7) "
|
||||
"ON CONFLICT (halacha_id, source_citation_id) DO UPDATE SET "
|
||||
"treatment=EXCLUDED.treatment, match_score=EXCLUDED.match_score",
|
||||
halacha_id, citing_case_law_id, citing_decision_id, source_id, treatment, score, context,
|
||||
)
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add the orchestrator**
|
||||
|
||||
```python
|
||||
# corroboration.py
|
||||
from uuid import UUID
|
||||
from legal_mcp.services import db, embeddings
|
||||
|
||||
async def build_for_precedent(case_law_id: str | UUID) -> dict:
|
||||
"""For one cited precedent: classify+match+store each incoming citation. Idempotent."""
|
||||
if isinstance(case_law_id, str):
|
||||
case_law_id = UUID(case_law_id)
|
||||
cits = await db.incoming_citations_for_precedent(case_law_id)
|
||||
linked = 0
|
||||
for c in cits:
|
||||
ctx = (c.get("context") or "").strip()
|
||||
if not ctx:
|
||||
continue
|
||||
vecs = await embeddings.embed_texts([ctx], input_type="query")
|
||||
best = await db.nearest_halacha_for_vector(case_law_id, vecs[0])
|
||||
halacha_id = accept_match(best)
|
||||
if not halacha_id:
|
||||
continue
|
||||
treatment = await classify_treatment(c.get("citing_case_law_id") or c.get("citing_decision_id") or "", ctx)
|
||||
await db.store_corroboration(
|
||||
halacha_id, c["source_id"],
|
||||
c.get("citing_case_law_id"), c.get("citing_decision_id"),
|
||||
treatment, best[1], ctx,
|
||||
)
|
||||
linked += 1
|
||||
return {"citations": len(cits), "linked": linked}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Integration smoke-test on Shafer (the fixed precedent, 7 citations)**
|
||||
|
||||
Run (from `mcp-server/`):
|
||||
```bash
|
||||
DOTENV_PATH=/home/chaim/.env DATA_DIR=/home/chaim/legal-ai/data .venv/bin/python -c "
|
||||
import asyncio; from legal_mcp.services import corroboration as cor
|
||||
print(asyncio.run(cor.build_for_precedent('9024da7b-f408-4b6f-808f-c514a83728e4')))"
|
||||
```
|
||||
Expected: `{'citations': 7, 'linked': <0..7>}` and no exception. Inspect `halacha_citation_corroboration` rows for treatment/score sanity.
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
```bash
|
||||
git add mcp-server/src/legal_mcp/services/corroboration.py mcp-server/src/legal_mcp/services/db.py
|
||||
git commit -m "feat(corroboration): orchestrator + persistence over both citation graphs (X11)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 6: Read-only MCP tool `halacha_corroboration`
|
||||
|
||||
Exposes per-halacha corroboration + provenance to the chair/agents (INV-COR6). **No approval change** — reporting only.
|
||||
|
||||
**Files:**
|
||||
- Modify: `mcp-server/src/legal_mcp/services/db.py` (add `list_corroboration_for_halacha`)
|
||||
- Modify: `mcp-server/src/legal_mcp/server.py` (register tool)
|
||||
|
||||
- [ ] **Step 1: Add the DB read**
|
||||
|
||||
```python
|
||||
# db.py
|
||||
async def list_corroboration_for_halacha(halacha_id: UUID) -> list[dict]:
|
||||
pool = await get_pool()
|
||||
rows = await pool.fetch(
|
||||
"SELECT treatment, match_score, match_context, citing_case_law_id::text, "
|
||||
" citing_decision_id::text, created_at "
|
||||
"FROM halacha_citation_corroboration WHERE halacha_id = $1 "
|
||||
"ORDER BY match_score DESC", halacha_id,
|
||||
)
|
||||
return [dict(r) for r in rows]
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Register the MCP tool** (copy an existing `@mcp.tool()` idiom in `server.py`)
|
||||
|
||||
```python
|
||||
@mcp.tool()
|
||||
async def halacha_corroboration(halacha_id: str) -> dict:
|
||||
"""החזר את ה-corroboration של הלכה: הציטוטים שמתקפים אותה, הטיפול, וסיכום (X11, read-only)."""
|
||||
from uuid import UUID
|
||||
from legal_mcp.services import corroboration as cor, db
|
||||
links = await db.list_corroboration_for_halacha(UUID(halacha_id))
|
||||
agg = cor.aggregate(
|
||||
[{"source_id": (l["citing_case_law_id"] or l["citing_decision_id"] or ""), "treatment": l["treatment"]} for l in links]
|
||||
)
|
||||
return {"halacha_id": halacha_id, "summary": agg, "citations": links}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Verify the tool loads** (restart MCP server is required to pick up new tools)
|
||||
|
||||
Run: `cd mcp-server && .venv/bin/python -c "from legal_mcp import server; print('halacha_corroboration' in [t.name for t in server.mcp._tool_manager.list_tools()])"`
|
||||
Expected: `True` (adjust the introspection call to the FastMCP version if needed; the point is the import succeeds and the tool registers).
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
```bash
|
||||
git add mcp-server/src/legal_mcp/services/db.py mcp-server/src/legal_mcp/server.py
|
||||
git commit -m "feat(mcp): halacha_corroboration read-only tool (INV-COR6, X11)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Out of scope (Phase 2 — separate plan)
|
||||
|
||||
- **Auto-approval wiring (INV-COR4/COR5 + G10 behavior):** make `corroborated == True` set `halachot.review_status='approved'` with `reviewer='corroborated (≥N judicial citations)'`, leaving the uncorroborated/negative tail in the chair gate. **Sensitive** — gated on Dafna validating the signal from Phase 1 first.
|
||||
- **Enrichment (INV-COR3 secondary):** sharpen `rule_statement` from citing framing.
|
||||
- **Backfill:** run `build_for_precedent` across all precedents with halachot + incoming citations.
|
||||
- **Treatment backfill of `case_law_citations.citation_type`** (currently default `'support'`).
|
||||
|
||||
---
|
||||
|
||||
## Self-Review
|
||||
|
||||
**Spec coverage:** INV-COR2 (Task 2 classifier + polarity), INV-COR3 (Task 3 match floor + pgvector), INV-COR4 (Task 4 distinct positive + min_cites), INV-COR6 (Task 5 store provenance + Task 6 report). INV-COR1 (principle, no code) and INV-COR5 (chair-gate preserved) are honored by **not** changing approval in Phase 1 — explicitly deferred to Phase 2. ✔
|
||||
**Placeholders:** none — every code step has concrete content; the LLM-dependent steps carry the exact prompt + I/O contract; tuning values (`MATCH_FLOOR`, `MIN_CITES`) are real env-defaults, not TBDs.
|
||||
**Type consistency:** `accept_match` returns `halacha_id|None`; `aggregate` consumes `[{"source_id","treatment"}]`; `_coerce_treatment`/`is_positive`/`is_negative` share the `_VALID_TREATMENT` sets. `build_for_precedent` uses `accept_match` + `classify_treatment` + `store_corroboration` with matching signatures. ✔
|
||||
@@ -54,6 +54,8 @@ REDIS_URL = os.environ.get("REDIS_URL", "redis://127.0.0.1:6380/0")
|
||||
# pinned.
|
||||
HALACHA_EXTRACT_MODEL = os.environ.get("HALACHA_EXTRACT_MODEL", "claude-opus-4-8")
|
||||
HALACHA_EXTRACT_EFFORT = os.environ.get("HALACHA_EXTRACT_EFFORT", "xhigh")
|
||||
HALACHA_CORROBORATION_MATCH_FLOOR = float(os.environ.get("HALACHA_CORROBORATION_MATCH_FLOOR", "0.50"))
|
||||
HALACHA_CORROBORATION_MIN_CITES = int(os.environ.get("HALACHA_CORROBORATION_MIN_CITES", "2"))
|
||||
|
||||
# Voyage AI
|
||||
VOYAGE_API_KEY = os.environ.get("VOYAGE_API_KEY", "")
|
||||
|
||||
@@ -922,6 +922,18 @@ async def list_chair_feedback(
|
||||
return await workflow.list_chair_feedback(case_number, category, unresolved_only)
|
||||
|
||||
|
||||
@mcp.tool()
|
||||
async def halacha_corroboration(halacha_id: str) -> dict:
|
||||
"""החזר את ה-corroboration של הלכה: הציטוטים שמתקפים אותה, הטיפול, וסיכום (X11, read-only)."""
|
||||
from uuid import UUID
|
||||
from legal_mcp.services import corroboration as cor, db
|
||||
links = await db.list_corroboration_for_halacha(UUID(halacha_id))
|
||||
agg = cor.aggregate(
|
||||
[{"source_id": (l["citing_case_law_id"] or l["citing_decision_id"] or ""), "treatment": l["treatment"]} for l in links]
|
||||
)
|
||||
return {"halacha_id": halacha_id, "summary": agg, "citations": links}
|
||||
|
||||
|
||||
def main():
|
||||
mcp.run(transport="stdio")
|
||||
|
||||
|
||||
104
mcp-server/src/legal_mcp/services/corroboration.py
Normal file
104
mcp-server/src/legal_mcp/services/corroboration.py
Normal file
@@ -0,0 +1,104 @@
|
||||
"""X11 citation corroboration — classify treatment, match to halacha, aggregate.
|
||||
|
||||
Phase 1: builds the SIGNAL only (no approval changes). See docs/spec/X11.
|
||||
All LLM calls go through the local claude_session bridge (Opus 4.8 @ xhigh),
|
||||
same architectural rule as the other extractors (local MCP only).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
import logging
|
||||
from uuid import UUID
|
||||
from legal_mcp import config
|
||||
from legal_mcp.config import parse_llm_json
|
||||
from legal_mcp.services import claude_session
|
||||
from legal_mcp.services import db, embeddings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
TREATMENT_POSITIVE = {"followed", "explained"}
|
||||
TREATMENT_NEGATIVE = {"distinguished", "criticized", "questioned", "overruled"}
|
||||
TREATMENT_NEUTRAL = {"mentioned"}
|
||||
_VALID_TREATMENT = TREATMENT_POSITIVE | TREATMENT_NEGATIVE | TREATMENT_NEUTRAL
|
||||
|
||||
def is_positive(t: str) -> bool: return t in TREATMENT_POSITIVE
|
||||
def is_negative(t: str) -> bool: return t in TREATMENT_NEGATIVE
|
||||
|
||||
def _coerce_treatment(raw: dict) -> str:
|
||||
t = str((raw or {}).get("treatment", "")).strip().lower()
|
||||
return t if t in _VALID_TREATMENT else "mentioned"
|
||||
|
||||
|
||||
def accept_match(best: tuple[str, float] | None, floor: float = config.HALACHA_CORROBORATION_MATCH_FLOOR) -> str | None:
|
||||
"""Return the halacha_id iff similarity clears the floor (INV-COR3)."""
|
||||
if not best:
|
||||
return None
|
||||
halacha_id, sim = best
|
||||
return halacha_id if sim >= floor else None
|
||||
|
||||
|
||||
def aggregate(links: list[dict], min_cites: int = config.HALACHA_CORROBORATION_MIN_CITES) -> dict:
|
||||
"""Aggregate per-halacha corroboration (INV-COR4/COR2).
|
||||
|
||||
links: [{"source_id": str, "treatment": str}, ...] already matched to ONE halacha.
|
||||
positive_sources = count of DISTINCT source_id whose treatment is positive.
|
||||
has_negative = any negative treatment present.
|
||||
corroborated = positive_sources >= min_cites AND not has_negative.
|
||||
"""
|
||||
positive = {l["source_id"] for l in links if is_positive(l["treatment"])}
|
||||
has_negative = any(is_negative(l["treatment"]) for l in links)
|
||||
return {
|
||||
"positive_sources": len(positive),
|
||||
"has_negative": has_negative,
|
||||
"corroborated": len(positive) >= min_cites and not has_negative,
|
||||
}
|
||||
|
||||
|
||||
_TREATMENT_PROMPT = """אתה משפטן בכיר. נתון ציטוט של פסק/החלטה קודמים בתוך החלטה מאוחרת.
|
||||
סווג כיצד ההחלטה המאוחרת **מטפלת** בתקדים המצוטט, לפי אחת מהקטגוריות:
|
||||
- followed — אימצה והחילה את ההלכה.
|
||||
- explained — הסבירה/הזכירה בלי לחלוק.
|
||||
- distinguished — אבחנה (קבעה שלא חל בנסיבות).
|
||||
- criticized — מתחה ביקורת בלי לבטל.
|
||||
- questioned — הטילה ספק.
|
||||
- overruled — דחתה/ביטלה את ההלכה.
|
||||
- mentioned — אזכור-אגב בלי טיפול.
|
||||
החזר JSON בלבד: {"treatment": "<קטגוריה>"}.
|
||||
"""
|
||||
|
||||
async def classify_treatment(cited_citation: str, context: str) -> str:
|
||||
"""Return one treatment label for how `context` treats `cited_citation`."""
|
||||
user = f"תקדים מצוטט: {cited_citation}\n\n--- ההקשר המצטט ---\n{context}\n--- סוף ---"
|
||||
try:
|
||||
result = await claude_session.query_json(
|
||||
user, system=_TREATMENT_PROMPT,
|
||||
model=config.HALACHA_EXTRACT_MODEL or None,
|
||||
effort=config.HALACHA_EXTRACT_EFFORT or None,
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning("classify_treatment failed: %s", e)
|
||||
return "mentioned"
|
||||
return _coerce_treatment(result if isinstance(result, dict) else {})
|
||||
|
||||
|
||||
async def build_for_precedent(case_law_id: str | UUID) -> dict:
|
||||
"""For one cited precedent: classify+match+store each incoming citation. Idempotent."""
|
||||
if isinstance(case_law_id, str):
|
||||
case_law_id = UUID(case_law_id)
|
||||
cits = await db.incoming_citations_for_precedent(case_law_id)
|
||||
linked = 0
|
||||
for c in cits:
|
||||
ctx = (c.get("context") or "").strip()
|
||||
if not ctx:
|
||||
continue
|
||||
vecs = await embeddings.embed_texts([ctx], input_type="query")
|
||||
best = await db.nearest_halacha_for_vector(case_law_id, vecs[0])
|
||||
halacha_id = accept_match(best)
|
||||
if not halacha_id:
|
||||
continue
|
||||
treatment = await classify_treatment(c.get("citing_case_law_id") or c.get("citing_decision_id") or "", ctx)
|
||||
await db.store_corroboration(
|
||||
halacha_id, c["source_id"],
|
||||
c.get("citing_case_law_id"), c.get("citing_decision_id"),
|
||||
treatment, best[1], ctx,
|
||||
)
|
||||
linked += 1
|
||||
return {"citations": len(cits), "linked": linked}
|
||||
@@ -1129,6 +1129,27 @@ ALTER TABLE case_law ADD COLUMN IF NOT EXISTS indexed_hash text;
|
||||
"""
|
||||
|
||||
|
||||
SCHEMA_V24_SQL = """
|
||||
-- X11: citation corroboration (treatment + halacha-level link)
|
||||
ALTER TABLE precedent_internal_citations
|
||||
ADD COLUMN IF NOT EXISTS treatment TEXT DEFAULT '';
|
||||
|
||||
CREATE TABLE IF NOT EXISTS halacha_citation_corroboration (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
halacha_id UUID NOT NULL REFERENCES halachot(id) ON DELETE CASCADE,
|
||||
citing_case_law_id UUID REFERENCES case_law(id) ON DELETE CASCADE,
|
||||
citing_decision_id UUID REFERENCES decisions(id) ON DELETE SET NULL,
|
||||
source_citation_id UUID NOT NULL,
|
||||
treatment TEXT NOT NULL,
|
||||
match_score NUMERIC(4,3) DEFAULT 0,
|
||||
match_context TEXT DEFAULT '',
|
||||
created_at TIMESTAMPTZ DEFAULT now(),
|
||||
UNIQUE (halacha_id, source_citation_id)
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_hcc_halacha ON halacha_citation_corroboration(halacha_id);
|
||||
"""
|
||||
|
||||
|
||||
async def _run_schema_migrations(pool: asyncpg.Pool) -> None:
|
||||
async with pool.acquire() as conn:
|
||||
await conn.execute(SCHEMA_SQL)
|
||||
@@ -1155,7 +1176,8 @@ async def _run_schema_migrations(pool: asyncpg.Pool) -> None:
|
||||
await conn.execute(SCHEMA_V21_SQL)
|
||||
await conn.execute(SCHEMA_V22_SQL)
|
||||
await conn.execute(SCHEMA_V23_SQL)
|
||||
logger.info("Database schema initialized (v1-v23)")
|
||||
await conn.execute(SCHEMA_V24_SQL)
|
||||
logger.info("Database schema initialized (v1-v24)")
|
||||
|
||||
|
||||
async def init_schema() -> None:
|
||||
@@ -3364,6 +3386,81 @@ async def update_halacha(
|
||||
return dict(row) if row else None
|
||||
|
||||
|
||||
async def nearest_halacha_for_vector(case_law_id: UUID, vec: list[float]) -> tuple[str, float] | None:
|
||||
"""Best-matching halacha of `case_law_id` for a context embedding (cosine)."""
|
||||
pool = await get_pool()
|
||||
row = await pool.fetchrow(
|
||||
"SELECT id::text AS id, 1 - (embedding <=> $2) AS sim "
|
||||
"FROM halachot WHERE case_law_id = $1 AND embedding IS NOT NULL "
|
||||
"ORDER BY embedding <=> $2 LIMIT 1",
|
||||
case_law_id, vec,
|
||||
)
|
||||
return (row["id"], float(row["sim"])) if row else None
|
||||
|
||||
|
||||
async def incoming_citations_for_precedent(case_law_id: UUID) -> list[dict]:
|
||||
"""All incoming citations (both graphs) with their context + source id."""
|
||||
pool = await get_pool()
|
||||
rows = await pool.fetch(
|
||||
"SELECT id::text AS source_id, source_case_law_id::text AS citing_case_law_id, "
|
||||
" NULL::text AS citing_decision_id, match_context AS context "
|
||||
"FROM precedent_internal_citations WHERE cited_case_law_id = $1 "
|
||||
"UNION ALL "
|
||||
"SELECT id::text, NULL, decision_id::text, context_text "
|
||||
"FROM case_law_citations WHERE case_law_id = $1",
|
||||
case_law_id,
|
||||
)
|
||||
return [dict(r) for r in rows]
|
||||
|
||||
|
||||
async def store_corroboration(
|
||||
halacha_id: str,
|
||||
source_id: str,
|
||||
citing_case_law_id,
|
||||
citing_decision_id,
|
||||
treatment: str,
|
||||
score: float,
|
||||
context: str,
|
||||
) -> None:
|
||||
from uuid import UUID as _UUID
|
||||
pool = await get_pool()
|
||||
# asyncpg requires UUID objects for uuid-typed columns; convert non-None strings.
|
||||
h_id = _UUID(halacha_id) if isinstance(halacha_id, str) else halacha_id
|
||||
s_id = _UUID(source_id) if isinstance(source_id, str) else source_id
|
||||
cl_id = _UUID(citing_case_law_id) if (citing_case_law_id and isinstance(citing_case_law_id, str)) else citing_case_law_id
|
||||
d_id = _UUID(citing_decision_id) if (citing_decision_id and isinstance(citing_decision_id, str)) else citing_decision_id
|
||||
await pool.execute(
|
||||
"INSERT INTO halacha_citation_corroboration "
|
||||
"(halacha_id, citing_case_law_id, citing_decision_id, source_citation_id, treatment, match_score, match_context) "
|
||||
"VALUES ($1,$2,$3,$4,$5,$6,$7) "
|
||||
"ON CONFLICT (halacha_id, source_citation_id) DO UPDATE SET "
|
||||
"treatment=EXCLUDED.treatment, match_score=EXCLUDED.match_score",
|
||||
h_id, cl_id, d_id, s_id, treatment, score, context,
|
||||
)
|
||||
|
||||
|
||||
async def list_corroboration_for_halacha(halacha_id: UUID) -> list[dict]:
|
||||
"""Return all corroboration rows for one halacha, ordered by match_score DESC."""
|
||||
pool = await get_pool()
|
||||
rows = await pool.fetch(
|
||||
"SELECT treatment, match_score, match_context, citing_case_law_id::text, "
|
||||
" citing_decision_id::text, created_at "
|
||||
"FROM halacha_citation_corroboration WHERE halacha_id = $1 "
|
||||
"ORDER BY match_score DESC", halacha_id,
|
||||
)
|
||||
return [
|
||||
{
|
||||
"treatment": r["treatment"],
|
||||
"match_score": float(r["match_score"]) if r["match_score"] is not None else None,
|
||||
"match_context": r["match_context"],
|
||||
"citing_case_law_id": r["citing_case_law_id"],
|
||||
"citing_decision_id": r["citing_decision_id"],
|
||||
"created_at": r["created_at"].isoformat() if r["created_at"] else None,
|
||||
}
|
||||
for r in rows
|
||||
]
|
||||
|
||||
|
||||
async def search_precedent_library_semantic(
|
||||
query_embedding: list[float],
|
||||
practice_area: str = "",
|
||||
|
||||
46
mcp-server/tests/test_corroboration.py
Normal file
46
mcp-server/tests/test_corroboration.py
Normal file
@@ -0,0 +1,46 @@
|
||||
from __future__ import annotations
|
||||
import pytest
|
||||
from legal_mcp.services import corroboration as cor
|
||||
|
||||
@pytest.mark.parametrize("raw,expected", [
|
||||
({"treatment": "followed"}, "followed"),
|
||||
({"treatment": "OVERRULED"}, "overruled"), # case-insensitive
|
||||
({"treatment": "bananas"}, "mentioned"), # unknown -> neutral default
|
||||
({}, "mentioned"), # missing -> neutral default
|
||||
])
|
||||
def test_coerce_treatment(raw, expected):
|
||||
assert cor._coerce_treatment(raw) == expected
|
||||
|
||||
def test_treatment_polarity():
|
||||
assert cor.is_positive("followed") and cor.is_positive("explained")
|
||||
assert cor.is_negative("distinguished") and cor.is_negative("overruled")
|
||||
assert not cor.is_positive("mentioned") and not cor.is_negative("mentioned")
|
||||
|
||||
def test_match_accepts_above_threshold():
|
||||
assert cor.accept_match(("h1", 0.62), floor=0.50) == "h1"
|
||||
|
||||
def test_match_rejects_below_threshold():
|
||||
assert cor.accept_match(("h1", 0.41), floor=0.50) is None
|
||||
|
||||
def test_match_rejects_empty():
|
||||
assert cor.accept_match(None, floor=0.50) is None
|
||||
|
||||
def _link(src, treatment):
|
||||
return {"source_id": src, "treatment": treatment}
|
||||
|
||||
def test_aggregate_counts_distinct_positive():
|
||||
links = [_link("d1","followed"), _link("d1","explained"), _link("d2","followed")]
|
||||
agg = cor.aggregate(links, min_cites=2)
|
||||
assert agg["positive_sources"] == 2 # d1 counted once (INV-COR4 independence)
|
||||
assert agg["has_negative"] is False
|
||||
assert agg["corroborated"] is True
|
||||
|
||||
def test_aggregate_negative_blocks():
|
||||
links = [_link("d1","followed"), _link("d2","followed"), _link("d3","distinguished")]
|
||||
agg = cor.aggregate(links, min_cites=2)
|
||||
assert agg["has_negative"] is True
|
||||
assert agg["corroborated"] is False # any negative -> not corroborated (INV-COR2)
|
||||
|
||||
def test_aggregate_below_threshold():
|
||||
agg = cor.aggregate([_link("d1","followed")], min_cites=2)
|
||||
assert agg["corroborated"] is False # single source insufficient (INV-COR4)
|
||||
Reference in New Issue
Block a user