feat(corroboration): approval_action decision fn + kill-switch (INV-COR2/COR4, X11 Phase 2)
- HALACHA_CORROBORATION_AUTO_APPROVE config (default ON, Dafna validated 2026-06-01) - approval_action(agg, has_overruled): overruled→demote, corroborated→approve, else None - 4 offline unit tests; Phase 2 plan + TaskMaster #75 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,290 @@
|
||||
# X11 Citation Corroboration — Phase 2 (Wire the approval gate) Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Turn the Phase 1 **signal** into an **approval action**. A halacha that is *corroborated* by ≥N distinct positive judicial citations (0 negatives) is auto-approved with citation provenance; a halacha that a later citing court *overruled* is demoted back to the chair gate. Then **backfill** the signal+approval across the whole corpus.
|
||||
|
||||
**Gate cleared:** Phase 1's "Out of scope" deferred auto-approval as *"Sensitive — gated on Dafna validating the signal from Phase 1 first."* Dafna validated the signal and approved enabling it (2026-06-01). This plan builds the **active** wiring (default ON, env-tunable kill-switch).
|
||||
|
||||
**Architecture:** No schema change — Phase 1's `halacha_citation_corroboration` table already holds the provenance. We add:
|
||||
1. a pure decision function `approval_action(agg, has_overruled)` (unit-tested, INV-COR2/COR4),
|
||||
2. DB transitions that move *only* the legal states (`pending_review → approved` on corroboration; `approved → pending_review` on overruled) — never touching `published`/`rejected`,
|
||||
3. `reconcile_approvals(case_law_id)` called at the tail of `build_for_precedent`,
|
||||
4. a corpus `build_all()` backfill driver + a **write** MCP tool `corroboration_rebuild`.
|
||||
|
||||
**Tech Stack:** Python 3.12, asyncpg, FastMCP, `claude_session` (local Opus 4.8), pytest (offline deterministic).
|
||||
|
||||
**Spec:** [docs/spec/X11-citation-corroboration.md](../../spec/X11-citation-corroboration.md) §4 step 5, §5 (INV-COR2/COR4/COR5/COR6), §6 (INV-G10 amendment).
|
||||
|
||||
---
|
||||
|
||||
## Invariant mapping (what each rule forces here)
|
||||
|
||||
- **INV-COR4** — auto-approve requires `positive_sources ≥ N` distinct sources ∧ `has_negative == False`. `aggregate()` (Phase 1) already computes this; Phase 2 only *acts* on `corroborated == True`.
|
||||
- **INV-COR2** — negative treatment never approves; **overruled** demotes. We split "negative" (blocks approval — already handled by `aggregate`) from **overruled** (actively *demotes an already-approved* halacha back to the chair).
|
||||
- **INV-COR5** — the chair gate is preserved for the uncorroborated tail and all negatives. We only transition the two legal states; uncorroborated halachot are never touched.
|
||||
- **INV-COR6** — provenance is retained: `reviewer` records the corroboration basis; the `halacha_citation_corroboration` rows remain the auditable evidence.
|
||||
- **INV-G10 (amended §6)** — the human gate's *authority source* is cumulative judicial treatment for the corroborated subset; chair gate stays mandatory for the tail. Auto-approval is therefore not "AI judgment" but recorded human (citing-court) judgment (INV-COR1).
|
||||
|
||||
**Demotion scope decision (precise reading of §4 step 5):** *any* negative blocks auto-approval (via `aggregate.has_negative`), but only **overruled** actively demotes a halacha that is already approved. `distinguished`/`criticized`/`questioned` block new auto-approval but do not un-approve an existing chair/confidence approval — that stronger action is reserved for `overruled`, and surfaced to the chair via the read tool.
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Config kill-switch
|
||||
|
||||
**Files:** Modify `mcp-server/src/legal_mcp/config.py`
|
||||
|
||||
- [ ] **Step 1:** After `HALACHA_CORROBORATION_MIN_CITES` (config.py:69) add:
|
||||
```python
|
||||
# X11 Phase 2: gate corroboration → approval. Default ON (Dafna validated the
|
||||
# Phase 1 signal, 2026-06-01). Set to "false" to disable the auto-approve/demote
|
||||
# wiring while keeping the signal (Phase 1) intact.
|
||||
HALACHA_CORROBORATION_AUTO_APPROVE = os.environ.get(
|
||||
"HALACHA_CORROBORATION_AUTO_APPROVE", "true"
|
||||
).strip().lower() in ("1", "true", "yes", "on")
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Commit** `feat(config): HALACHA_CORROBORATION_AUTO_APPROVE kill-switch (X11 Phase 2)`
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Pure decision function `approval_action` (TDD)
|
||||
|
||||
The whole approval policy distilled to one deterministic, offline-testable function.
|
||||
|
||||
**Files:** Modify `corroboration.py`; Test `tests/test_corroboration.py`
|
||||
|
||||
- [ ] **Step 1: Failing test** — append to `tests/test_corroboration.py`:
|
||||
```python
|
||||
def test_approval_action_corroborated_approves():
|
||||
agg = {"positive_sources": 2, "has_negative": False, "corroborated": True}
|
||||
assert cor.approval_action(agg, has_overruled=False) == "approve"
|
||||
|
||||
def test_approval_action_overruled_demotes_even_if_corroborated():
|
||||
# overruled wins over a positive count (INV-COR2 strong form)
|
||||
agg = {"positive_sources": 3, "has_negative": True, "corroborated": False}
|
||||
assert cor.approval_action(agg, has_overruled=True) == "demote"
|
||||
|
||||
def test_approval_action_single_source_noop():
|
||||
agg = {"positive_sources": 1, "has_negative": False, "corroborated": False}
|
||||
assert cor.approval_action(agg, has_overruled=False) is None
|
||||
|
||||
def test_approval_action_negative_nonoverruled_noop():
|
||||
# distinguished blocks approval but does not demote (no overruled)
|
||||
agg = {"positive_sources": 2, "has_negative": True, "corroborated": False}
|
||||
assert cor.approval_action(agg, has_overruled=False) is None
|
||||
```
|
||||
|
||||
- [ ] **Step 2:** Run to verify FAIL (`approval_action` undefined).
|
||||
|
||||
- [ ] **Step 3:** Implement in `corroboration.py`:
|
||||
```python
|
||||
def approval_action(agg: dict, has_overruled: bool) -> str | None:
|
||||
"""Decide the corroboration→approval action for ONE halacha (INV-COR2/COR4).
|
||||
|
||||
- 'demote' : a later court overruled it → back to the chair gate (overruled
|
||||
outranks any positive count).
|
||||
- 'approve' : corroborated (≥N distinct positives, 0 negatives).
|
||||
- None : leave as-is (single source, non-overruled negative, or tail).
|
||||
"""
|
||||
if has_overruled:
|
||||
return "demote"
|
||||
if agg.get("corroborated"):
|
||||
return "approve"
|
||||
return None
|
||||
```
|
||||
|
||||
- [ ] **Step 4:** Run to verify PASS (all). **Commit** `feat(corroboration): approval_action decision fn (INV-COR2/COR4, X11 Phase 2)`
|
||||
|
||||
---
|
||||
|
||||
## Task 3: DB transitions (legal states only)
|
||||
|
||||
**Files:** Modify `mcp-server/src/legal_mcp/services/db.py`
|
||||
|
||||
- [ ] **Step 1:** Add near `update_halacha` (db.py:3480-ish):
|
||||
```python
|
||||
async def approve_halacha_by_corroboration(
|
||||
halacha_id: UUID, n_sources: int, min_cites: int,
|
||||
) -> bool:
|
||||
"""Approve a halacha on citation corroboration — ONLY if it is currently
|
||||
awaiting the chair (pending_review). Never touches 'published'/'rejected'/
|
||||
already-'approved' (INV-COR5: chair gate preserved for everything else).
|
||||
Returns True iff a row transitioned."""
|
||||
pool = await get_pool()
|
||||
reviewer = f"corroborated ({n_sources} judicial citations ≥ {min_cites})"
|
||||
row = await pool.fetchrow(
|
||||
"UPDATE halachot SET review_status='approved', reviewer=$2, "
|
||||
"reviewed_at=now(), updated_at=now() "
|
||||
"WHERE id=$1 AND review_status='pending_review' RETURNING id",
|
||||
halacha_id, reviewer,
|
||||
)
|
||||
return row is not None
|
||||
|
||||
|
||||
async def demote_halacha_overruled(halacha_id: UUID) -> bool:
|
||||
"""Demote an APPROVED halacha back to the chair gate because a later citing
|
||||
court overruled it (INV-COR2). Only acts on 'approved' → 'pending_review';
|
||||
leaves 'published'/'rejected'/'pending_review' untouched. The reviewer note
|
||||
records why it is back in the queue. Returns True iff a row transitioned."""
|
||||
pool = await get_pool()
|
||||
row = await pool.fetchrow(
|
||||
"UPDATE halachot SET review_status='pending_review', "
|
||||
"reviewer='flagged: overruled by later citation (X11)', "
|
||||
"reviewed_at=NULL, updated_at=now() "
|
||||
"WHERE id=$1 AND review_status='approved' RETURNING id",
|
||||
halacha_id,
|
||||
)
|
||||
return row is not None
|
||||
|
||||
|
||||
async def list_corroboration_grouped(case_law_id: UUID) -> dict[str, list[dict]]:
|
||||
"""Per-halacha corroboration links for a cited precedent, in the
|
||||
{source_id, treatment} shape `aggregate()` consumes. Distinct citing source
|
||||
keyed by case_law/decision id (falls back to the citation row id)."""
|
||||
pool = await get_pool()
|
||||
rows = await pool.fetch(
|
||||
"SELECT hcc.halacha_id::text AS halacha_id, "
|
||||
" COALESCE(hcc.citing_case_law_id::text, hcc.citing_decision_id::text, "
|
||||
" hcc.source_citation_id::text) AS source_id, "
|
||||
" hcc.treatment "
|
||||
"FROM halacha_citation_corroboration hcc "
|
||||
"JOIN halachot h ON h.id = hcc.halacha_id "
|
||||
"WHERE h.case_law_id = $1",
|
||||
case_law_id,
|
||||
)
|
||||
out: dict[str, list[dict]] = {}
|
||||
for r in rows:
|
||||
out.setdefault(r["halacha_id"], []).append(
|
||||
{"source_id": r["source_id"], "treatment": r["treatment"]}
|
||||
)
|
||||
return out
|
||||
|
||||
|
||||
async def precedents_with_halachot_and_incoming_citations() -> list[str]:
|
||||
"""case_law ids that have at least one halacha AND at least one incoming
|
||||
citation (either graph) — the backfill target set."""
|
||||
pool = await get_pool()
|
||||
rows = await pool.fetch(
|
||||
"SELECT c.id::text FROM case_law c "
|
||||
"WHERE EXISTS (SELECT 1 FROM halachot h WHERE h.case_law_id=c.id) "
|
||||
" AND (EXISTS (SELECT 1 FROM precedent_internal_citations p "
|
||||
" WHERE p.cited_case_law_id=c.id) "
|
||||
" OR EXISTS (SELECT 1 FROM case_law_citations cc "
|
||||
" WHERE cc.case_law_id=c.id))",
|
||||
)
|
||||
return [r["id"] for r in rows]
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Commit** `feat(db): corroboration approve/demote transitions + backfill query (X11 Phase 2)`
|
||||
|
||||
---
|
||||
|
||||
## Task 4: `reconcile_approvals` + wire into `build_for_precedent` + `build_all`
|
||||
|
||||
**Files:** Modify `corroboration.py`
|
||||
|
||||
- [ ] **Step 1:** Add to `corroboration.py`:
|
||||
```python
|
||||
async def reconcile_approvals(case_law_id: str | UUID) -> dict:
|
||||
"""Apply the corroboration→approval policy for every halacha of a precedent.
|
||||
No-op (returns disabled) when the kill-switch is off. INV-COR2/COR4/COR5."""
|
||||
if not config.HALACHA_CORROBORATION_AUTO_APPROVE:
|
||||
return {"approved": 0, "demoted": 0, "disabled": True}
|
||||
if isinstance(case_law_id, str):
|
||||
case_law_id = UUID(case_law_id)
|
||||
grouped = await db.list_corroboration_grouped(case_law_id)
|
||||
approved = demoted = 0
|
||||
for halacha_id, links in grouped.items():
|
||||
agg = aggregate(links)
|
||||
has_overruled = any(l["treatment"] == "overruled" for l in links)
|
||||
action = approval_action(agg, has_overruled)
|
||||
if action == "approve":
|
||||
if await db.approve_halacha_by_corroboration(
|
||||
UUID(halacha_id), agg["positive_sources"],
|
||||
config.HALACHA_CORROBORATION_MIN_CITES,
|
||||
):
|
||||
approved += 1
|
||||
elif action == "demote":
|
||||
if await db.demote_halacha_overruled(UUID(halacha_id)):
|
||||
demoted += 1
|
||||
return {"approved": approved, "demoted": demoted, "disabled": False}
|
||||
```
|
||||
|
||||
- [ ] **Step 2:** At the end of `build_for_precedent`, replace the `return` with:
|
||||
```python
|
||||
appr = await reconcile_approvals(case_law_id)
|
||||
return {"citations": len(cits), "linked": linked,
|
||||
"approved": appr["approved"], "demoted": appr["demoted"]}
|
||||
```
|
||||
|
||||
- [ ] **Step 3:** Add the corpus driver:
|
||||
```python
|
||||
async def build_all() -> dict:
|
||||
"""Backfill: build the signal + apply approvals for every precedent that has
|
||||
halachot and incoming citations. Idempotent (ON CONFLICT on the link table;
|
||||
transitions only fire on the legal state)."""
|
||||
ids = await db.precedents_with_halachot_and_incoming_citations()
|
||||
totals = {"precedents": 0, "citations": 0, "linked": 0,
|
||||
"approved": 0, "demoted": 0}
|
||||
for cid in ids:
|
||||
r = await build_for_precedent(cid)
|
||||
totals["precedents"] += 1
|
||||
for k in ("citations", "linked", "approved", "demoted"):
|
||||
totals[k] += r.get(k, 0)
|
||||
logger.info("corroboration backfill %s: %s", cid, r)
|
||||
return totals
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Commit** `feat(corroboration): reconcile_approvals + build_all backfill (X11 Phase 2)`
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Write MCP tool `corroboration_rebuild`
|
||||
|
||||
**Files:** Modify `mcp-server/src/legal_mcp/server.py`
|
||||
|
||||
- [ ] **Step 1:** Add near `halacha_corroboration` (server.py:926):
|
||||
```python
|
||||
@mcp.tool()
|
||||
async def corroboration_rebuild(case_law_id: str = "") -> dict:
|
||||
"""בנה/רענן את ה-corroboration ויישם אישור-אוטומטי. ריק = כל הקורפוס (backfill);
|
||||
מזהה-תקדים = תקדים בודד. כותב halacha_citation_corroboration + מעדכן review_status
|
||||
(corroborated→approved, overruled→pending_review). X11 Phase 2."""
|
||||
from legal_mcp.services import corroboration as cor
|
||||
if case_law_id.strip():
|
||||
return await cor.build_for_precedent(case_law_id.strip())
|
||||
return await cor.build_all()
|
||||
```
|
||||
|
||||
- [ ] **Step 2:** Verify import/registration:
|
||||
```bash
|
||||
cd mcp-server && .venv/bin/python -c "from legal_mcp import server; print('corroboration_rebuild' in [t.name for t in server.mcp._tool_manager.list_tools()])"
|
||||
```
|
||||
Expected `True`.
|
||||
|
||||
- [ ] **Step 3: Commit** `feat(mcp): corroboration_rebuild write tool (X11 Phase 2)`
|
||||
|
||||
---
|
||||
|
||||
## Task 6: Backfill the corpus + verify
|
||||
|
||||
- [ ] **Step 1:** Snapshot approved/pending counts before.
|
||||
- [ ] **Step 2:** Run `build_all()` from the venv (`DOTENV_PATH=/home/chaim/.env DATA_DIR=…`). Expect ~12 precedents, no exception, a small number of `approved`/`demoted`.
|
||||
- [ ] **Step 3:** Verify: every halacha approved-by-corroboration has `reviewer LIKE 'corroborated %'`; no `published`/`rejected` changed; corroboration rows carry treatment+score. Spot-check one approved halacha via `halacha_corroboration`.
|
||||
- [ ] **Step 4: Commit** any data-audit note under `data/audit/`.
|
||||
|
||||
---
|
||||
|
||||
## Out of scope (Phase 2 backlog — deliberately deferred)
|
||||
|
||||
- **Enrichment (INV-COR3 secondary):** sharpen `rule_statement` from citing framing — **proposal-only**, must not silently rewrite an approved rule. Bigger design; separate plan.
|
||||
- **Treatment backfill of `case_law_citations.citation_type`** (default `'support'`) — orthogonal to corroboration (which classifies treatment fresh per citation into its own column).
|
||||
|
||||
---
|
||||
|
||||
## Self-Review
|
||||
|
||||
**Spec coverage:** INV-COR2 (overruled demote split from generic negative-block; Task 2/3/4), INV-COR4 (acts only on `corroborated`; Task 2/4), INV-COR5 (only legal-state transitions, tail untouched; Task 3 WHERE clauses), INV-COR6 (reviewer provenance + retained link rows; Task 3), INV-G10 amended (authority = citing courts; not AI; Task 2 comment). ✔
|
||||
**Safety:** kill-switch default ON but env-disable-able; transitions are directional and bounded by `review_status` WHERE clauses (cannot touch chair-final states); demotion moves toward *more* human review. ✔
|
||||
**Idempotency:** link table `ON CONFLICT` (Phase 1); approve only fires on `pending_review`, demote only on `approved` → re-runs converge. ✔
|
||||
Reference in New Issue
Block a user