feat: Stage A finalizers + #35/#36/#37 — critical-gap closure
Some checks failed
Build & Deploy / build-and-deploy (push) Has been cancelled

Four parallel sub-agents closed the remaining critical gaps from the
26/05 Stage A/B sprint. Each block independently tested; aggregated here.

## #30/#31 finalizers (sub-agent A)
* Auto-derive practice_area in case_create from case_number prefix
  (1xxx→rishuy_uvniya, 8xxx→betterment_levy, 9xxx→compensation_197);
  default for CaseCreateRequest is now "" (the DB constraint catches
  any stray "appeals_committee").
* practice_area.py: derive_subtype now handles axis-B domain values
  (rishuy_uvniya/betterment_levy/compensation_197) without parsing the
  case number; new helper derive_domain_practice_area().
* Halacha re-extraction verified unnecessary — all 6 reclassified
  records already had is_binding=false and approved halachot.
* Regression tests: 6 cases in tests/test_corpus_constraints.py
  covering practice_area enum, internal-committee chair/district,
  external-upload arar prefix, MCP guard.
* UI: district input → Select dropdown (7 districts) in
  precedent-edit-sheet.tsx, preserving legacy free-text values.

## #37 בל"מ subtypes (sub-agent B)
* 3 new appeal_subtypes: extension_request_{building_permit,
  betterment_levy,compensation}. APPEALS_COMMITTEE_SUBTYPES extended,
  SUBTYPES_BY_AREA mappings added.
* New helpers: is_blam_subject(), is_blam_subtype(),
  derive_subtype_with_blam(case_number, subject, practice_area).
  case_create now uses it to auto-detect "בקשה להארכת מועד" subjects.
* 3 methodology templates under docs/methodology/extension-request-*.md.
* paperclip_client.py mapping updated for the 3 new subtypes
  (extension_request_building_permit→CMP, the other two→CMPA).
* Frontend: bilingual "בל"מ" badge + filter dropdown on cases list +
  detail header; appeal-type-bars collapseBlam() merges בל"מ into its
  parent domain for aggregate bars.
* Wizard auto-detects בל"מ from subject during case creation.
* 3 Berlinger cases (1017/1018/1019-03-26) migrated to
  appeal_subtype=extension_request_building_permit via psql.

## #35 missing_precedents feature (sub-agent C)
* Schema V13: missing_precedents table (citation, case_id, party,
  legal_topic, status, linked_case_law_id, claim_quote, ...) +
  FK constraints + 3 indexes. Applied via psql + idempotent migration.
* 6 db.py service functions, 3 MCP tools, 6 FastAPI endpoints
  (POST/GET/PATCH/DELETE/upload — upload routes by citation prefix
  to ingest_internal_decision or ingest_precedent).
* Next.js page /missing-precedents with 5 status tabs + filters +
  sidebar badge counter + detail drawer with metadata edit + smart
  upload form that switches fields per committee/court.
* Bootstrap: 7 rows imported from the JSON file
  (3 citations × cases, all status=closed with linked_case_law_id).
* legal-researcher.md: new §2ב.5 with missing_precedent_create
  usage + dedup semantics + tool grant.

## #36 legal_arguments aggregation (sub-agent D)
* Schema V14: legal_arguments + legal_argument_propositions M:M.
  Applied via psql.
* New service argument_aggregator.py with two functions —
  aggregate_claims_to_arguments() (Claude CLI / claude_session) and
  get_legal_arguments(). Graceful llm_unavailable handling when CLI
  is missing (containers).
* 2 MCP tools + 2 API endpoints (POST .../aggregate-arguments as
  BackgroundTask, GET .../legal-arguments).
* Frontend: shadcn Accordion + new legal-arguments-panel.tsx with
  hierarchical (party → priority badge → arguments) display, "טיעונים"
  tab on the case page, "חשב/חשב מחדש" buttons.
* scripts/backfill_legal_arguments.py + SCRIPTS.md entry — dry-run
  found 8 candidate cases including 1017/1018/1019.

## Open follow-ups (intentionally deferred)
* npm run api:types in web-ui (CLAUDE.md flow) — recommended before
  the next UI commit; not required for backend deployment.
* Run backfill_legal_arguments.py --apply once the container picks up
  the new aggregator service.
* webhook on missing-precedents upload-close to Paperclip (optional).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-05-26 08:34:40 +00:00
parent af651d0135
commit f3cc9ca9d4
33 changed files with 4588 additions and 37 deletions

View File

@@ -28,6 +28,7 @@
| `voyage_rerank_corpus_poc.py` | python | POC #5 — voyage-3 vs rerank-2 על קורפוס מלא (785 docs). הכרעה: +4.5% mean@3 כללי, +11.6% על P queries (practical) | בנצ'מרק חד-פעמי, אישר את שלב B |
| `multimodal_backfill.py` | python | Backfill voyage-multimodal-3 page embeddings על מסמכי תיקים קיימים. idempotent (skips by default), forces `MULTIMODAL_ENABLED=true` ל-run, רץ מהקונטיינר. שלב C — ראה `docs/voyage-upgrades-plan.md` | ידני per-case (`python multimodal_backfill.py 8174-24 8137-24`) |
| `backfill_chunk_pages.py` | python | Backfill `page_number` ב-`document_chunks` קיימים. legacy chunker לא tracked עמודים → `page_number=NULL` חוסם boost של multimodal hybrid (text+image join על אותו עמוד). re-extracts כל PDF (re-OCR אם צריך, ~$0.0015/page), מחשב page_offsets, ומעדכן chunks. idempotent | ידני per-case (`python backfill_chunk_pages.py 8174-24 8137-24`) |
| `backfill_legal_arguments.py` | python | Backfill `legal_arguments` לתיקים עם `claims` קיימים (TaskMaster #36). מקבץ פרופוזיציות גולמיות לטיעונים משפטיים מובחנים (~6-12 לכל צד) דרך `argument_aggregator.aggregate_claims_to_arguments` (Claude CLI). תומך `--dry-run`/`--apply`/`--force`/`--case <num>...`. **חייב לרוץ מהמכונה המקומית** (לא קונטיינר) — `claude_session` דורש Claude CLI | ידני per-case (`python scripts/backfill_legal_arguments.py --apply --case 1017-03-26`) |
## תיקיית `.archive/` — סקריפטים שהושלמו

View File

@@ -0,0 +1,164 @@
#!/usr/bin/env python3
"""Backfill aggregated legal_arguments for existing cases.
For every case that has rows in ``claims`` but none in ``legal_arguments``,
run ``argument_aggregator.aggregate_claims_to_arguments``.
Usage (must use mcp-server venv — pgvector + asyncpg are vendored there):
PY=/home/chaim/legal-ai/mcp-server/.venv/bin/python
# Default = dry-run (lists what would be processed):
$PY scripts/backfill_legal_arguments.py
# Process all cases that need it:
$PY scripts/backfill_legal_arguments.py --apply
# Re-aggregate even cases that already have arguments:
$PY scripts/backfill_legal_arguments.py --apply --force
# Only process specific cases:
$PY scripts/backfill_legal_arguments.py --apply --case 1017-03-26 1018-03-26
The script must run from the local dev machine (not the container) because
``argument_aggregator`` calls ``claude_session`` which needs the Claude CLI.
"""
from __future__ import annotations
import argparse
import asyncio
import os
import sys
from pathlib import Path
from uuid import UUID
# Make the mcp-server source importable as ``legal_mcp``.
REPO_ROOT = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(REPO_ROOT / "mcp-server" / "src"))
# Default DB connection (overridable via env / .env on the dev box).
if "POSTGRES_URL" not in os.environ:
pg_user = os.environ.get("POSTGRES_USER", "legal_ai")
pg_pw = os.environ.get("POSTGRES_PASSWORD", "")
pg_host = os.environ.get("POSTGRES_HOST", "127.0.0.1")
pg_port = os.environ.get("POSTGRES_PORT", "5433")
pg_db = os.environ.get("POSTGRES_DB", "legal_ai")
os.environ["POSTGRES_URL"] = (
f"postgres://{pg_user}:{pg_pw}@{pg_host}:{pg_port}/{pg_db}"
)
async def _list_cases_needing_backfill(force: bool) -> list[dict]:
"""Find cases that have claims but no aggregated arguments (or all,
when ``force`` is True)."""
from legal_mcp.services import db
pool = await db.get_pool()
async with pool.acquire() as conn:
rows = await conn.fetch(
"""
SELECT c.id, c.case_number, c.status,
COUNT(DISTINCT cl.id) AS claim_count,
COUNT(DISTINCT la.id) AS arg_count
FROM cases c
LEFT JOIN claims cl ON cl.case_id = c.id
LEFT JOIN legal_arguments la ON la.case_id = c.id
WHERE c.archived_at IS NULL
GROUP BY c.id, c.case_number, c.status
HAVING COUNT(DISTINCT cl.id) > 0
ORDER BY c.case_number
"""
)
out: list[dict] = []
for r in rows:
d = dict(r)
if force or d["arg_count"] == 0:
out.append(d)
return out
async def _process_case(case: dict, force: bool) -> dict:
from legal_mcp.services import argument_aggregator
case_id = UUID(str(case["id"]))
case_number = case["case_number"]
print(
f"[backfill] {case_number}: {case['claim_count']} claims, "
f"{case['arg_count']} existing args — aggregating (force={force})...",
flush=True,
)
try:
result = await argument_aggregator.aggregate_claims_to_arguments(
case_id, force=force,
)
except Exception as e: # noqa: BLE001
return {
"case_number": case_number,
"status": "error",
"error": str(e),
}
print(
f"[backfill] {case_number}: status={result.get('status')} "
f"total={result.get('total')} by_party={result.get('by_party')}",
flush=True,
)
return {"case_number": case_number, **result}
async def main() -> int:
parser = argparse.ArgumentParser(
description="Backfill legal_arguments for cases with extracted claims.",
)
parser.add_argument(
"--apply", action="store_true",
help="Actually run aggregation (default: dry-run).",
)
parser.add_argument(
"--force", action="store_true",
help="Re-aggregate even cases that already have arguments.",
)
parser.add_argument(
"--case", nargs="*", default=[],
help="Only process these case numbers (e.g. --case 1017-03-26 1018-03-26).",
)
args = parser.parse_args()
cases = await _list_cases_needing_backfill(force=args.force)
if args.case:
wanted = set(args.case)
cases = [c for c in cases if c["case_number"] in wanted]
if not cases:
print("[backfill] No cases need processing.")
return 0
print(f"[backfill] {len(cases)} case(s) to process:")
for c in cases:
print(
f" - {c['case_number']:<14} status={c['status']:<20} "
f"claims={c['claim_count']:<4} args={c['arg_count']}",
)
if not args.apply:
print("\n[backfill] dry-run — pass --apply to actually run.")
return 0
print()
results: list[dict] = []
for case in cases:
r = await _process_case(case, force=args.force)
results.append(r)
print("\n[backfill] === Summary ===")
for r in results:
print(
f" {r['case_number']:<14} status={r.get('status', 'unknown'):<22} "
f"total={r.get('total', 0)}",
)
errors = [r for r in results if r.get("status") == "error"]
return 1 if errors else 0
if __name__ == "__main__":
sys.exit(asyncio.run(main()))