feat(corpus): corpus redesign — eliminate halacha queue, verified-by-citation layer, rank-at-retrieval (#153)
Implements chaim's 2026-06-20 directive (5 steps; step 6 deferred): 1. No review queue — HALACHA_NO_REVIEW_QUEUE=true (auto-approve all → background); migration cleared 2,416 pending_review → approved. 2. Verified layer — halachot.verified/cite_count from chair citations (db.refresh_verified_layer + scripts/build_verified_layer.py runs citator on ALL committee decisions). 2,775 verified / 137 precedents. 3. Retrieval ranks verified ≫ background — HALACHA_VERIFIED_BOOST in both semantic + lexical halacha queries; filter now includes background (<> rejected). 5. Disabled destructive panel cap/novelty — HALACHA_PANEL_REGIME_ENABLED=false (8508/1049/1200 proved it lost 22-30 genuine principles incl. Lustrenik). 4. Ingest contract — going-forward already queues metadata; backfill_practice_area.py + 206 re-queued to the metadata drain. Source of truth: docs/precedent-corpus-redesign/00-final-synthesis.md. Quality flags are 97% false-positive (nli-audit) → no longer gate. UI queue removal → Claude Design gate. 429 tests green (no regressions). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -89,3 +89,15 @@ importance (דפנה≫יו"ר-אחר≫סמכות). שום עיקרון לא נ
|
||||
4. **תיקון-חוזה-קליטה** (practice_area) — היגיינת-מקור.
|
||||
5. **רוויזיית-PR#304** — לבטל cap+novelty (הרסניים). הפאנל/דגלים לכל-היותר סיגנל-דירוג.
|
||||
6. (נדחה) V41/conformal/הטמעת-קאנון-חסר.
|
||||
|
||||
## 7. סטטוס-מימוש (2026-06-20)
|
||||
| צעד | סטטוס |
|
||||
|---|---|
|
||||
| 5 — ביטול cap/novelty | ✅ `HALACHA_PANEL_REGIME_ENABLED=false` (חזרה לחילוץ-עשיר-לרקע) |
|
||||
| 1 — ביטול תור | ✅ `HALACHA_NO_REVIEW_QUEUE=true` (auto-approve הכל) + migration 2,416→0 pending |
|
||||
| 2 — שכבת-מאומת | ✅ `verified`/`cite_count` + `db.refresh_verified_layer` + `build_verified_layer.py`; **2,775 מאומתים / 137 פס"ד** |
|
||||
| 3 — אחזור מאומת≫רקע | ✅ boost ב-2 שאילתות-האחזור (`HALACHA_VERIFIED_BOOST`); אומת חי (מאומתים צפים) |
|
||||
| 4 — חוזה-קליטה | ✅ going-forward מחווט (ingest queues metadata); 206 הוזנו ל-drain + `backfill_practice_area.py` (backfill חסום-מכסה זמנית) |
|
||||
| 6 — V41/conformal/קאנון-חסר | נדחה (כמתוכנן) |
|
||||
|
||||
429 בדיקות ירוקות (אפס רגרסיות). **שינוי-UI (הסרת תור-ההלכות מ-/precedents) → דרך שער Claude Design.**
|
||||
|
||||
@@ -162,6 +162,23 @@ HALACHA_AUTO_APPROVE_THRESHOLD = float(
|
||||
os.environ.get("HALACHA_AUTO_APPROVE_THRESHOLD", "0.80")
|
||||
)
|
||||
|
||||
# Corpus redesign (#153, chaim 2026-06-20): ELIMINATE the halacha review queue.
|
||||
# When on (default), extraction never produces 'pending_review' — every extracted
|
||||
# principle lands as 'approved' = available BACKGROUND (no human review, ever).
|
||||
# Trust/ranking comes from chair citation (halachot.verified/cite_count), not from
|
||||
# an approval gate. The nli-audit found the quality flags are 97% false-positive,
|
||||
# so gating on them only created a phantom backlog (2,402 items). Set false to
|
||||
# restore the legacy confidence+flags auto-approve gate.
|
||||
HALACHA_NO_REVIEW_QUEUE = os.environ.get("HALACHA_NO_REVIEW_QUEUE", "true").lower() == "true"
|
||||
|
||||
# Corpus redesign (#153): retrieval ranks VERIFIED (chair-cited) principles above
|
||||
# the unranked BACKGROUND. Added to the halacha similarity score (cosine 0-1): a flat
|
||||
# boost if the source precedent was chair-cited, plus a small per-citation increment
|
||||
# (capped). 0 disables (pure similarity). Tunable; calibrate against the canon.
|
||||
HALACHA_VERIFIED_BOOST = float(os.environ.get("HALACHA_VERIFIED_BOOST", "0.12"))
|
||||
HALACHA_CITE_BOOST_PER = float(os.environ.get("HALACHA_CITE_BOOST_PER", "0.01"))
|
||||
HALACHA_CITE_BOOST_CAP = int(os.environ.get("HALACHA_CITE_BOOST_CAP", "10"))
|
||||
|
||||
# ── Tri-model panel extraction regime (legal-principles-redesign, #152) ──────
|
||||
# chaim 2026-06-19: replace single-model auto-approve with a 3-model panel that
|
||||
# deep-analyzes each decision. 3 models (Claude local + DeepSeek + Gemini) each
|
||||
@@ -179,10 +196,13 @@ HALACHA_PANEL_MAX_NEW = int(os.environ.get("HALACHA_PANEL_MAX_NEW", "5"))
|
||||
# a floor misses genuine cross-model agreement → undercounts votes → over-culls.
|
||||
# Calibrate against the gold-set in Phase C before the production cull.
|
||||
HALACHA_PANEL_MATCH_COSINE = float(os.environ.get("HALACHA_PANEL_MATCH_COSINE", "0.80"))
|
||||
# When on (default), extraction uses the decision-level 3-model panel regime above
|
||||
# instead of the legacy per-chunk single-model auto-approve. Set false to fall back
|
||||
# to the legacy path (e.g. if all three judges are unreachable).
|
||||
HALACHA_PANEL_REGIME_ENABLED = os.environ.get("HALACHA_PANEL_REGIME_ENABLED", "true").lower() == "true"
|
||||
# DEFAULT OFF (#153, chaim 2026-06-20). The panel regime caps extraction to MAX_NEW
|
||||
# and filters by novelty — empirically PROVEN destructive (8508/1049/1200 each lost
|
||||
# 22-30 genuine principles incl. the core Lustrenik rule). The corpus redesign keeps
|
||||
# ALL extracted principles as an unranked BACKGROUND layer (trust comes from chair
|
||||
# citation, not extraction); so extraction reverts to the legacy rich per-chunk path.
|
||||
# The panel code is retained (dormant) for optional dedup, never for capping.
|
||||
HALACHA_PANEL_REGIME_ENABLED = os.environ.get("HALACHA_PANEL_REGIME_ENABLED", "false").lower() == "true"
|
||||
|
||||
# Importance layer (#153) — principle-level gold matching. OUR_CHAIR's citations
|
||||
# (tier-1 gold, protective) vs other chairs' (tier-2 weight). Match threshold: a
|
||||
|
||||
@@ -1669,6 +1669,20 @@ ALTER TABLE halachot
|
||||
CREATE INDEX IF NOT EXISTS idx_halachot_gold ON halachot(gold_chair, gold_digest)
|
||||
WHERE gold_chair OR gold_digest;
|
||||
|
||||
-- Corpus redesign (#153, chaim 2026-06-20: "trusted = citation, not review").
|
||||
-- Two-layer retrieval model:
|
||||
-- • verified = the principle's SOURCE precedent was actually cited by a chair
|
||||
-- (precedent_internal_citations). The ONLY trust signal — never
|
||||
-- from human review (the halacha review queue is eliminated).
|
||||
-- • cite_count = # distinct chair decisions citing the source precedent → the
|
||||
-- importance/ranking signal (verified ≫ background at retrieval).
|
||||
-- Refreshed by db.refresh_verified_layer() (scripts/build_verified_layer.py), and
|
||||
-- grows automatically as new chair decisions are ingested (active-learning).
|
||||
ALTER TABLE halachot
|
||||
ADD COLUMN IF NOT EXISTS verified BOOLEAN NOT NULL DEFAULT false,
|
||||
ADD COLUMN IF NOT EXISTS cite_count INT NOT NULL DEFAULT 0;
|
||||
CREATE INDEX IF NOT EXISTS idx_halachot_verified ON halachot(verified) WHERE verified;
|
||||
|
||||
-- halacha_citation_corroboration (X11) gains canonical_id so the signal
|
||||
-- aggregates at the principle level rather than the per-instance level.
|
||||
-- Backfill: UPDATE halacha_citation_corroboration SET canonical_id =
|
||||
@@ -5148,7 +5162,8 @@ async def store_halachot(case_law_id: UUID, halachot: list[dict]) -> int:
|
||||
async with pool.acquire() as conn:
|
||||
for i, h in enumerate(halachot):
|
||||
confidence = float(h.get("confidence", 0.0))
|
||||
auto_approve = confidence >= threshold
|
||||
# #153: no review queue → everything is available background (approved).
|
||||
auto_approve = config.HALACHA_NO_REVIEW_QUEUE or confidence >= threshold
|
||||
review_status = "approved" if auto_approve else "pending_review"
|
||||
reviewer = (
|
||||
f"auto-approved (confidence ≥ {threshold:.2f})"
|
||||
@@ -5353,7 +5368,9 @@ async def store_halachot_for_chunk(
|
||||
instance_type = "citation"
|
||||
|
||||
confidence = float(h.get("confidence", 0.0))
|
||||
auto_approve = confidence >= threshold and not flags
|
||||
# #153: no review queue → everything is available background (approved);
|
||||
# quality flags become ranking signals, not an approval gate (nli 97% FP).
|
||||
auto_approve = config.HALACHA_NO_REVIEW_QUEUE or (confidence >= threshold and not flags)
|
||||
review_status = "approved" if auto_approve else "pending_review"
|
||||
reviewer = (
|
||||
f"auto-approved (confidence ≥ {threshold:.2f})"
|
||||
@@ -6417,6 +6434,38 @@ async def gold_coverage_stats() -> dict:
|
||||
return dict(row)
|
||||
|
||||
|
||||
async def refresh_verified_layer() -> dict:
|
||||
"""Recompute the verified/cite_count layer from chair citations (#153).
|
||||
|
||||
'verified' = the principle's SOURCE precedent was cited by a chair (any
|
||||
committee decision). 'cite_count' = # distinct chair decisions citing it. This
|
||||
is the ONLY trust signal — never human review. Idempotent (full recompute).
|
||||
Returns {verified_principles, verified_precedents}.
|
||||
"""
|
||||
pool = await get_pool()
|
||||
async with pool.acquire() as conn:
|
||||
async with conn.transaction():
|
||||
await conn.execute(
|
||||
"UPDATE halachot SET verified=false, cite_count=0 "
|
||||
"WHERE verified OR cite_count>0")
|
||||
await conn.execute(
|
||||
"WITH cc AS ("
|
||||
" SELECT pic.cited_case_law_id AS id, "
|
||||
" count(DISTINCT pic.source_case_law_id) AS n "
|
||||
" FROM precedent_internal_citations pic "
|
||||
" JOIN case_law src ON src.id = pic.source_case_law_id "
|
||||
" WHERE src.source_kind='internal_committee' "
|
||||
" AND pic.cited_case_law_id IS NOT NULL "
|
||||
" GROUP BY pic.cited_case_law_id) "
|
||||
"UPDATE halachot h SET verified=true, cite_count=cc.n, updated_at=now() "
|
||||
"FROM cc WHERE h.case_law_id = cc.id")
|
||||
row = await conn.fetchrow(
|
||||
"SELECT count(*) FILTER (WHERE verified) AS vp, "
|
||||
" count(DISTINCT case_law_id) FILTER (WHERE verified) AS vc "
|
||||
"FROM halachot")
|
||||
return {"verified_principles": row["vp"], "verified_precedents": row["vc"]}
|
||||
|
||||
|
||||
async def list_canonical_instances(canonical_id: "UUID") -> list[dict]:
|
||||
"""List all halachot (instances) sharing a canonical_id — used by the UI accordion."""
|
||||
pool = await get_pool()
|
||||
@@ -6943,7 +6992,7 @@ async def search_precedent_library_semantic(
|
||||
"""
|
||||
pool = await get_pool()
|
||||
halacha_filters = [
|
||||
"h.review_status IN ('approved', 'published')",
|
||||
"h.review_status <> 'rejected'", # #153: include background; rank verified higher
|
||||
f"cl.source_kind = '{source_kind}'",
|
||||
"cl.searchable = true",
|
||||
]
|
||||
@@ -7007,18 +7056,22 @@ async def search_precedent_library_semantic(
|
||||
c_params.append(chair_name)
|
||||
c_idx += 1
|
||||
|
||||
# #153: verified (chair-cited) principles float above background.
|
||||
vboost = (f"(CASE WHEN h.verified THEN {config.HALACHA_VERIFIED_BOOST} ELSE 0 END "
|
||||
f"+ LEAST(h.cite_count, {config.HALACHA_CITE_BOOST_CAP}) * {config.HALACHA_CITE_BOOST_PER})")
|
||||
halacha_sql = f"""
|
||||
SELECT h.id AS halacha_id, h.case_law_id, h.rule_statement,
|
||||
h.reasoning_summary, h.supporting_quote, h.page_reference,
|
||||
h.practice_areas, h.subject_tags, h.confidence, h.rule_type,
|
||||
cl.case_number, cl.case_name, cl.court, cl.date AS decision_date,
|
||||
cl.precedent_level, cl.chair_name, cl.district,
|
||||
1 - (h.embedding <=> $1) AS score
|
||||
h.verified, h.cite_count,
|
||||
(1 - (h.embedding <=> $1)) + {vboost} AS score
|
||||
FROM halachot h
|
||||
JOIN case_law cl ON cl.id = h.case_law_id
|
||||
WHERE {' AND '.join(halacha_filters)}
|
||||
AND h.embedding IS NOT NULL
|
||||
ORDER BY h.embedding <=> $1
|
||||
ORDER BY score DESC
|
||||
LIMIT $2
|
||||
"""
|
||||
|
||||
@@ -7182,7 +7235,7 @@ async def search_precedent_library_lexical(
|
||||
|
||||
pool = await get_pool()
|
||||
halacha_filters = [
|
||||
"h.review_status IN ('approved', 'published')",
|
||||
"h.review_status <> 'rejected'", # #153: include background; rank verified higher
|
||||
f"cl.source_kind = '{source_kind}'",
|
||||
"cl.searchable = true",
|
||||
]
|
||||
@@ -7247,18 +7300,23 @@ async def search_precedent_library_lexical(
|
||||
c_params.append(chair_name)
|
||||
c_idx += 1
|
||||
|
||||
# #153: verified (chair-cited) principles float above background.
|
||||
vboost = (f"(CASE WHEN h.verified THEN {config.HALACHA_VERIFIED_BOOST} ELSE 0 END "
|
||||
f"+ LEAST(h.cite_count, {config.HALACHA_CITE_BOOST_CAP}) * {config.HALACHA_CITE_BOOST_PER})")
|
||||
halacha_sql = f"""
|
||||
SELECT h.id AS halacha_id, h.case_law_id, h.rule_statement,
|
||||
h.reasoning_summary, h.supporting_quote, h.page_reference,
|
||||
h.practice_areas, h.subject_tags, h.confidence, h.rule_type,
|
||||
cl.case_number, cl.case_name, cl.court, cl.date AS decision_date,
|
||||
cl.precedent_level, cl.chair_name, cl.district,
|
||||
h.verified, h.cite_count,
|
||||
GREATEST(
|
||||
ts_rank_cd(h.rule_tsv, plainto_tsquery('simple', $1)),
|
||||
ts_rank_cd(cl.meta_tsv, plainto_tsquery('simple', $1))
|
||||
)
|
||||
+ CASE WHEN cl.meta_tsv @@ plainto_tsquery('simple', $1)
|
||||
THEN 1.0 ELSE 0.0 END AS score
|
||||
THEN 1.0 ELSE 0.0 END
|
||||
+ {vboost} AS score
|
||||
FROM halachot h
|
||||
JOIN case_law cl ON cl.id = h.case_law_id
|
||||
WHERE {' AND '.join(halacha_filters)}
|
||||
|
||||
@@ -65,6 +65,9 @@
|
||||
| `halacha_panel_calibrate.py` | python | **כיול + מדידת הפאנל** (Trust-or-Escalate, ICLR 2025). `--source live` (ברירת-מחדל): מריץ את שאלת-ה-KEEP על מדגם-הזהב ומודד מול `is_holding` precision+coverage+**split-rate** לכל מדיניות + false-keep/false-drop (מייבא שופטים מ-`halacha_panel_approve`, **חובה מקומי**). **#133/FU-5** — `--source captured`: **אפס-עלות** (בלי re-vote/LLM) — מצליב סבבים שמורים (FU-1) מול הכרעות-יו"ר (FU-2) דרך `db.panel_rounds_vs_chair` ומדווח split-rate+auto-precision **לכל סבב** (מגמת הלולאה: ככל שהרובריקה משתפרת precision נשמר ו-split יורד); משתף את `analyze_pairs` של FU-4 (מקור-יחיד). שתי המדידות מדווחות **anon-stability** (מבחן-אנונימיזציה #81.7) כמטריקת-בריאות נגד echo-chamber. `--batch`/`--limit`/`--concurrency`. | ידני — לפני חיווט `--apply` (live) / תקופתי — מעקב-לולאה (captured) |
|
||||
| `halacha_rubric_distill.py` | python | **#133/FU-4 — זיקוק-רובריקה PROPOSE-ONLY.** מצליב `halacha_panel_rounds` (FU-1, הצבעות+נימוקים) מול הכרעות-היו"ר (FU-2, seeds ב-`halacha_goldset` batch `chair-live`) דרך `db.panel_rounds_vs_chair` (read-only), מנתח דטרמיניסטית **כשלים שיטתיים** (false-keep/false-drop, פיצולים-שהוכרעו, שיעור-מחלוקת-עם-היו"ר לכל שופט), ומציע `KEEP_SYSTEM` v2 + exemplars מופשטים (claude_session מקומי, אפס עלות) כ**דוח-diff** ל-`data/learning/rubric-proposal-<ts>.md`. **לעולם לא auto-apply** — אימוץ v2 = עריכה אנושית של הקבוע דרך PR (INV-LRN1); exemplars מופשטים בלבד (INV-LRN5); הסיגנל היחיד = הכרעת-יו"ר, לא הצבעות-פאנל (anti-echo). מתחת ל-12 זוגות → "אין מספיק נתונים". `--no-llm` (סטטיסטיקה בלבד) / `--limit N`. **חובה מקומי**. | תקופתי — אחרי שהצטברו הכרעות-יו"ר על מחלוקות-פאנל |
|
||||
| `backfill_canonical_halachot.py` | python | **V41 — הקמת מודל ההלכות הקנוניות (חד-פעמי + idempotent).** (1) בונה רכיבים-קשורים (connected components) מ-`equivalent_halachot` (transitive closure — union-find). (2) לכל אשכול: בוחר נציג-קנוני (הכי הרבה corroboration → confidence → earliest), יוצר שורת `canonical_halachot`, ומעדכן `canonical_id` + `instance_type` לכל חברי האשכול. (3) לסינגלטונים (ללא קישורי-שוויון): 1:1 canonical. (4) מאכלס `halacha_citation_corroboration.canonical_id` מ-`halachot.canonical_id`. `--dry-run` (ברירת-מחדל, מחשב ומדווח בלבד) / `--apply` (כותב) / `--verbose`. לאחר הרצה: `canonical_statement` = ניסוח-נציג (pending_synthesis); עוקב: `backfill_canonical_synthesis.py` (Phase 4) יסנתז ניסוח-רחב דרך LLM. הרץ: `mcp-server/.venv/bin/python scripts/backfill_canonical_halachot.py --apply`. | **חד-פעמי** (לאחר deploy V41) / idempotent לפי צורך |
|
||||
| `build_verified_layer.py` | python | **#153 — בניית שכבת-המאומת מאזכורים.** "מאומת=אזכור, לא ביקורת" (chaim 2026-06-20): מריץ את ה-citator (`extract_internal_citations`) על **כל** החלטות-הוועדה (לא רק דפנה), ואז `db.refresh_verified_layer` שמחשב `halachot.verified`/`cite_count` מ-`precedent_internal_citations` (verified=פס"ד-המקור צוטט ע"י יו"ר; cite_count=# החלטות מצטטות). idempotent, גדל-אוטומטית עם החלטות חדשות. regex/embeddings בלבד, אפס-LLM. `--no-citator` (refresh בלבד). הרץ: `mcp-server/.venv/bin/python scripts/build_verified_layer.py`. | חוזר (אחרי קליטת החלטות-יו"ר) |
|
||||
| `backfill_practice_area.py` | python | **#153 step 4 — חוזה-קליטה.** 87% מהפסיקה-החיצונית בלי practice_area → אחזור-מסונן-תחום מחמיצן. מריץ סיווג קיים (`derive_domain_practice_area` דטרמיניסטי → `precedent_metadata_extractor.extract_and_apply` LLM) על הלא-מסווגים. מרוסן (extract_metadata=claude/sonnet). `--dry-run`/`--apply`/`--limit`/`--no-throttle`. **going-forward כבר מחווט** (ingest.py:233 מתזמן metadata); זה ל-backfill הקיימים (חלופה: להזין ל-`metadata_extraction_requested_at` ולתת ל-drain). | חד-פעמי (backfill) |
|
||||
| `compute_principle_gold.py` | python | **#153 (נטוש)** — גישת זיהוי-זהב ברמת-עיקרון דרך התאמת-`match_context`→הלכה. **הוחלף** ע"י "מאומת=אזכור" (`build_verified_layer.py`) אחרי שההתאמה נכשלה (match_context=רשימת-הפניות). נשמר לעיון. | deprecated |
|
||||
| `cull_principles.py` | python | **#152 Phase C — סינון רטרואקטיבי של קורפוס-העקרונות דרך פאנל-3 (הפיך).** מריץ על כל עיקרון 'original' קיים את אותו משטר שה-extractor משתמש בו להבא (`services/panel_extraction.panel_keep_score`, G2): 3 שופטים (Claude מקומי + DeepSeek + Gemini) מצביעים keep+score → כלל-האישור (3 קולות→שורד · 2 וציון≥0.85→שורד · 2 ו<0.85→יו"ר · ≤1→נדחה) → תקרת `HALACHA_PANEL_MAX_NEW`=5 לכל החלטה לפי ציון (`apply_cap`). נדחה → `halachot.review_status='rejected'` + ה-canonical שלו `rejected` (הפיך, גיבוי-CSV ב-`data/audit/` לפני כל כתיבה). מרוסן ב-`usage_limits` (עוצר-רך בתקרת-שימוש, resumable). `--dry-run` (ברירת-מחדל) / `--apply` / `--sample N` (החלטות אקראיות) / `--limit N` / `--no-throttle` / `--verbose`. **חובה מקומי** (3 שופטים). הרץ: `cd mcp-server && HOME=/home/chaim .venv/bin/python ../scripts/cull_principles.py --apply`. | **חד-פעמי** (סינון ראשוני) + ניתן-לחזרה |
|
||||
| `backfill_canonical_synthesis.py` | python | **V41 Phase 4 — סינתזת-LLM ל-`canonical_statement` (idempotent + resumable).** עובר על canonicals ב-`review_status='pending_synthesis'` (רב-instance ראשונים) ומזקק לכל אחד ניסוח אחד כללי ומעוגן בציטוטי-המופעים (INV-AH) דרך `services/canonical_synthesis.py` (מסלול-יחיד, G2). שערים: עיגון/הימנעות, **drift-floor** (cosine מול המקור, ברירת-מחדל 0.80 — סטייה גדולה→נשמר המקור), ואיסור ציטוטי-תיק חדשים. בכל מקרה הסטטוס מתקדם ל-`pending_review` לשער-היו"ר (G10/INV-LRN6). מודל Opus (`HALACHA_CANONICAL_SYNTH_MODEL`). מרוסן ע"י `usage_limits` (עוצר-רך בתקרת-שימוש, resumable). `--dry-run` (ברירת-מחדל) / `--apply` / `--sample N` (מדגם אקראי לבדיקה) / `--limit N` / `--no-throttle` / `--verbose`. CSV-audit ל-`data/audit/canonical-synthesis-*.csv`. **חובה מקומי** (claude_session). הרץ: `cd mcp-server && HOME=/home/chaim .venv/bin/python ../scripts/backfill_canonical_synthesis.py --apply`. שוטף: כלי-MCP `canonical_synthesize_pending`. | **חד-פעמי** (המסה הראשונית) + idempotent לחדשים |
|
||||
| `halacha_batch_reconcile.py` | python | **#82.7** — dedup חוצה-פסקים offline (שמרני, **dry-run בלבד**). dedup-on-insert משווה רק תוך-פסק; כאן סף מחמיר (cosine ≥0.95, `--cosine`) ולא-הרסני: מאתר זוגות הלכות near-duplicate בין פסקים שונים (pgvector `<=>` exact) עם איתות לקסיקלי (Jaccard/Levenshtein) ומדווח ל-CSV ב-`data/audit/` לסקירת היו"ר. לא מדלג/ממזג/מוחק. `--include-pending`. **`--link`** רושם את הזוגות שנמצאו כ-`equivalent_halachot` (parallel authority, #84.2 — **deprecated post-V41** — השתמש ב-`backfill_canonical_halachot.py --apply` במקום). רץ עם venv של mcp-server. | **deprecated** — הוחלף ב-`backfill_canonical_halachot.py` (V41). נשמר לצורכי audit |
|
||||
|
||||
98
scripts/backfill_practice_area.py
Normal file
98
scripts/backfill_practice_area.py
Normal file
@@ -0,0 +1,98 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Backfill practice_area for external precedents (#153, step 4 — ingest contract).
|
||||
|
||||
87% of external court rulings (209/239) lack practice_area, so area-scoped retrieval
|
||||
misses them. The classifier infrastructure already exists
|
||||
(precedent_metadata_extractor.extract_and_apply → practice_area + metadata); it just
|
||||
never ran on these rows. This runs it on the unclassified, throttled by usage_limits.
|
||||
|
||||
Deterministic shortcut first (derive_domain_practice_area from our case-number scheme,
|
||||
free); only rows it can't resolve go to the LLM classifier.
|
||||
|
||||
cd ~/legal-ai/mcp-server
|
||||
HOME=/home/chaim .venv/bin/python ../scripts/backfill_practice_area.py --dry-run
|
||||
HOME=/home/chaim .venv/bin/python ../scripts/backfill_practice_area.py --apply
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "mcp-server", "src"))
|
||||
|
||||
from legal_mcp.services import db, precedent_metadata_extractor # noqa: E402
|
||||
from legal_mcp.services.practice_area import derive_domain_practice_area # noqa: E402
|
||||
|
||||
try:
|
||||
from legal_mcp.services import usage_limits
|
||||
except Exception: # pragma: no cover
|
||||
usage_limits = None
|
||||
|
||||
|
||||
def _over_ceiling() -> tuple[bool, str]:
|
||||
if usage_limits is None:
|
||||
return False, ""
|
||||
u = usage_limits.subscription_usage()
|
||||
if u is None:
|
||||
return False, ""
|
||||
over, _r, detail = usage_limits.ceiling_status(u)
|
||||
return over, detail
|
||||
|
||||
|
||||
async def _run(apply: bool, limit: int | None, throttle: bool) -> int:
|
||||
pool = await db.get_pool()
|
||||
rows = await pool.fetch(
|
||||
"SELECT id, case_number FROM case_law "
|
||||
"WHERE source_kind='external_upload' AND COALESCE(practice_area,'')='' "
|
||||
" AND COALESCE(full_text,'')<>'' ORDER BY created_at")
|
||||
if limit:
|
||||
rows = rows[:limit]
|
||||
print(f"[{'APPLY' if apply else 'DRY-RUN'}] {len(rows)} unclassified external precedents\n", flush=True)
|
||||
det = llm = stopped = 0
|
||||
by_area: dict[str, int] = {}
|
||||
for n, r in enumerate(rows, 1):
|
||||
# 1) deterministic from our case-number scheme (free)
|
||||
area = derive_domain_practice_area(r["case_number"] or "")
|
||||
if area:
|
||||
det += 1
|
||||
by_area[area] = by_area.get(area, 0) + 1
|
||||
if apply:
|
||||
await pool.execute("UPDATE case_law SET practice_area=$2 WHERE id=$1", r["id"], area)
|
||||
continue
|
||||
# 2) LLM classifier (throttled)
|
||||
if throttle:
|
||||
over, detail = _over_ceiling()
|
||||
if over:
|
||||
print(f"\n⏸ usage ceiling ({detail}) — stopping at {n-1}. Re-run to resume.", flush=True)
|
||||
stopped = 1
|
||||
break
|
||||
if apply:
|
||||
res = await precedent_metadata_extractor.extract_and_apply(r["id"])
|
||||
pa = (res or {}).get("practice_area") or ""
|
||||
if pa:
|
||||
llm += 1
|
||||
by_area[pa] = by_area.get(pa, 0) + 1
|
||||
else:
|
||||
llm += 1
|
||||
if n % 20 == 0:
|
||||
print(f" …{n}/{len(rows)}", flush=True)
|
||||
print(f"\n── summary ── deterministic: {det} · LLM: {llm} · by_area: {by_area}"
|
||||
f"{' (stopped early)' if stopped else ''}")
|
||||
if not apply:
|
||||
print("dry-run — nothing written. Re-run with --apply.")
|
||||
return 0
|
||||
|
||||
|
||||
def main() -> int:
|
||||
p = argparse.ArgumentParser(description="Backfill practice_area for external precedents (#153)")
|
||||
p.add_argument("--apply", action="store_true")
|
||||
p.add_argument("--limit", type=int, default=None)
|
||||
p.add_argument("--no-throttle", action="store_true")
|
||||
a = p.parse_args()
|
||||
return asyncio.run(_run(a.apply, a.limit, not a.no_throttle))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
53
scripts/build_verified_layer.py
Normal file
53
scripts/build_verified_layer.py
Normal file
@@ -0,0 +1,53 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Build the verified principle layer from chair citations (#153, corpus redesign).
|
||||
|
||||
"Trusted = citation, not review" (chaim 2026-06-20). A principle is `verified` iff
|
||||
its SOURCE precedent was actually cited by a chair (any committee decision); never
|
||||
from human review. This:
|
||||
1. Runs the citator (`extract_internal_citations`) over ALL committee decisions —
|
||||
not just דפנה's — so other chairs' citations populate the graph too (tier-2).
|
||||
2. Recomputes halachot.verified / cite_count from precedent_internal_citations.
|
||||
|
||||
Idempotent. Run after ingesting new chair decisions (or wire into the ingest path)
|
||||
so the verified layer grows automatically. EMBEDDING/REGEX-only for the citator,
|
||||
no LLM.
|
||||
|
||||
cd ~/legal-ai/mcp-server
|
||||
HOME=/home/chaim .venv/bin/python ../scripts/build_verified_layer.py # full
|
||||
HOME=/home/chaim .venv/bin/python ../scripts/build_verified_layer.py --no-citator # refresh only
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "mcp-server", "src"))
|
||||
|
||||
from legal_mcp.services import citation_extractor, db # noqa: E402
|
||||
|
||||
|
||||
async def _run(run_citator: bool) -> int:
|
||||
if run_citator:
|
||||
print("→ extracting citations from ALL committee decisions (citator)…", flush=True)
|
||||
res = await citation_extractor.extract_all_internal_committee()
|
||||
print(f" citator: {res}", flush=True)
|
||||
print("→ refreshing verified/cite_count from chair citations…", flush=True)
|
||||
stats = await db.refresh_verified_layer()
|
||||
print(f"\n── verified layer ──")
|
||||
print(f" verified principles: {stats['verified_principles']}")
|
||||
print(f" verified precedents: {stats['verified_precedents']}")
|
||||
return 0
|
||||
|
||||
|
||||
def main() -> int:
|
||||
p = argparse.ArgumentParser(description="Build verified principle layer (#153)")
|
||||
p.add_argument("--no-citator", action="store_true",
|
||||
help="skip citation extraction; only recompute verified/cite_count")
|
||||
a = p.parse_args()
|
||||
return asyncio.run(_run(run_citator=not a.no_citator))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
Reference in New Issue
Block a user