Bake the 2026-06-03 strict-cleanup rubric into the extraction pipeline so the
corpus stays clean at the source instead of accumulating duplicates, obiter
dicta, truncated quotes and thin restatements that clog the review queue.
#81 — quality gate:
- New pure module halacha_quality.py with unit-tested validators:
non-decision/obiter (Wambaugh markers), truncated-quote (mid-word cut),
thin-restatement (rule≈quote), quote-unverified.
- Validators run in halacha_extractor._process; a non-decision is re-typed
obiter; flags persist in new halachot.quality_flags column.
- Auto-approve now requires confidence>=threshold AND no quality flags;
flagged items route to pending_review regardless of confidence.
- Both extraction prompts hardened: reject undecided dicta, exclude
case-specific applications, require abstraction, forbid over-splitting.
#82 — dedup-on-insert (store_halachot_for_chunk):
- Within the same precedent, skip a halacha whose normalized supporting_quote
already exists, or whose rule-embedding has cosine>=HALACHA_DEDUP_COSINE
(0.93) against an already-stored one. Makes re-runs idempotent.
Migration: halachot.quality_flags TEXT[] (additive, idempotent ALTER).
Tests: 19 new unit tests; full suite 156 passed. Validated end-to-end against
dev DB (dedup skips dups, flag blocks auto-approve, re-run inserts 0).
Calibration: flags fire on only ~10% of current survivors (low false-positive).
Spec: docs/halacha-strict-rubric.md
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#34 don't manufacture doubt about unambiguous statutes (s.19(ג)(2));
#35 writer/QA two-sources-of-truth sync gap (DB vs drafts/decision.md).
Output of the weekly-feedback-analysis job, pending commit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
חיווט ספ-המערכת לסוכני-Paperclip כך שכל סוכן חייב לקרוא את 00-constitution
תחילה, ואז את ספ-התחום הרלוונטי לתפקידו (לפי טבלת X4 §2) — לפני עבודה מהותית.
- HEARTBEAT.md: סעיף עליון "קריאת-ספ — קודם החוקה (00), אז ספ-התחום" לפני §0–§8,
עם טבלת תפקיד→ספ ל-8 הסוכנים.
- 8 קבצי-סוכן (ceo/proofreader/researcher/analyst/writer/qa/exporter/hermes):
סעיף "קרא לפני פעולה (INV-AG1)" בראש הגוף.
- X4-agents.md: שדה "אכיפה" של INV-AG1 → "מחוּוט (פרוצדורלי)"; §5 → "בוצע".
אכיפה פרוצדורלית בכוונה — invariant פרויקטלי-תפעולי, אין שער-קוד שמכריח קריאה.
prereq לסוכני-התהליך (תת-פרויקט 5). gap-audit נשמר כ-snapshot (כמו FU-8a).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Covers GAP-11 (INV-RET4/G8) and GAP-14 (INV-QA1/G10). Retrieval quality was
never measured (only telemetry observation) and the halacha review backlog was
invisible (the 10/19 gap was found by accident).
Unit B — backlog visibility (pure code, container):
- metrics.halacha_backlog(conn) → {pending_review, approved, rejected, published,
total, oldest_pending_at}; surfaced in metrics.get_dashboard() (get_metrics MCP
tool) and /api/system/diagnostics. Live count revealed 178 pending / 1552 total,
oldest from 2026-05-03 — previously invisible.
Unit A — retrieval eval harness (host-side scripts):
- scripts/eval_gold_bootstrap.py — seeds data/eval/gold-set.jsonl. Two sources:
citations (cited==relevant via search_relevance_feedback — empty until decisions
cite precedents) and known_item (query=case_name → relevant=self; a real
citation-free signal, the methodology #52 checked by hand). Idempotent; preserves
source='chair' rows.
- scripts/eval_retrieval.py — runs the production retrieval path (search_library /
search_internal) over the gold-set; computes precision@k, recall@k, MRR, nDCG@k
(k=5,10); aggregates overall + per-corpus + per-practice_area; writes a report and
a delta vs committed baseline.json (which records the retrieval_config it reflects).
--self-test unit-checks the metric math offline.
Gold-set strategy = hybrid (chair decision): bootstrap + chair review. The citation
source is empty today (0 cited precedents in decisions), so the seed is known-item
(77 queries: 54 internal_decisions + 23 precedent_library). The gold-set is
PROVISIONAL until Dafna reviews it (the domain chair-gate).
Baseline (production config: multimodal+rerank on): R@10=0.987, MRR=0.837,
nDCG@10=0.872. Finding: MULTIMODAL_ENABLED=true slightly lowers known-item recall
(image-page results displace exact name matches) — relevant to #15. precedent_library
weaker than internal (R@10 0.957 vs 1.0) — one external precedent unfindable by name.
"CI gate" realized as discipline (re-runnable harness + committed baseline + run
before/after any retrieval-layer change) — retrieval needs prod DB + Voyage, no CI
runner has that access.
Spec: docs/superpowers/specs/2026-05-31-fu5-eval-harness-design.md
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Deterministic migration of ~52 internal_committee rows whose case_number holds
a full citation → normalized bare number (citation_formatted already correct).
DB analysis (2026-05-31): clean 1-token extraction, 0 key-collisions, 0
citation↔case_number mismatches, no month-padding dups. Chair-gated reversible
migration (backup→dry-run→approve→apply). One edge for chair: 8047/23 ערר vs בל"מ.
External (#68/FU-2c) split out — its citation_formatted is inconsistent.
Verified all 11 case_law FKs use id(UUID), not case_number → rename is FK-safe.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
content_hash/indexed_hash change detection + reindex_case_law from stored
full_text (no re-OCR) + drift health-check. Verified vs 3+ sources (content-
hash change detection, RAG re-embed-on-edit). #61.2 multimodal backfill closed:
42 rows are text-ingested (document_id NULL, no source PDF) — page-images
impossible without a PDF to render.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
X2/X3/X4 invariants are facts about this system's own integration/ops (no external
authority); they use מקור-סמכות=project runbooks, tied to a global engineering invariant.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Per chair clarification: the ≥3-authoritative-source verification protocol governs
ENGINEERING/architecture decisions only (G1–G10). Legal-domain content (G11) is the
authority of the chair + project docs (block-schema, decision-methodology, lessons,
skills/decision) — NOT externally triple-sourced.
- §2/§4/§5 scoped to engineering invariants; added the two-authority distinction
- G11 reframed: source-of-authority = chair + project docs; removed FJC/South Bucks/
1958-statute as "sources to verify" and the UNVERIFIED flag
- Removed the "open items — primary-source verification" section (the over-application)
- Pruned now-orphaned legal sources from the appendix (kept NCSC/CEPEJ/FJC for G9/G10)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
13 tasks across 3 phases (keystone constitution → lifecycle files → cross-cutting),
each verification-gated (≥3 sources or UNVERIFIED+escalate) with review checkpoints.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
