Adds docs/spec/X14-storage-minio.md — the domain spec + phased plan for
migrating binary document storage from the local data/ tree to the
already-deployed MinIO service (Coolify svc `minio`).
Captures: disk inventory, scattered file-I/O map (~8 services, no central
layer), DB path columns, MinIO deploy state, Paperclip = API-consumer only.
Defines 7 domain invariants (INV-STG1..7) and a 7-phase execution plan.
Chair decisions (2026-06-08): git-per-case keeps text/metadata + MinIO holds
binaries (INV-STG7); WORM Object-Lock on FINAL decisions only (INV-STG4);
internal Docker network for legal-ai↔MinIO.
Invariants: keeps G2 (single storage path replaces scattered I/O);
INV-STG1..7 new. Spec-only PR — no code/behavior change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Gemini key is stored in Infisical as GOOGLE_GEMINI_API_KEY
(nautilus /external-apis/gemini). Align the panel to read that canonical name
first, falling back to bare GEMINI_API_KEY for back-compat — so an
Infisical→.env sync keeps working.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
משלים את #141 בצד-לקוח: שדה digest_kind ב-Digest type (hand-written), ותג
"עדכון" ב-DigestCard לגיליונות announcement (לא-הכרעות). decision = ברירת-מחדל
ללא תג. זורם דרך /api/digests (digest_kind כבר ב-_DIGEST_COLS).
build (webpack) עובר, lint נקי בקבצי digests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The halacha extraction queue was stuck (same class as the metadata issue): 26
precedents requested extraction with no drainer, plus 1 orphaned in 'processing'
(status=processing, requested_at cleared → never re-picked by the queue).
- db.requeue_stale_processing_extractions(kind): re-stamp orphaned 'processing'
rows (requested_at IS NULL) so they re-drain; halacha extractor force=False
resumes from chunk checkpoints (no duplicates).
- process_pending_extractions calls it at the top — fully unattended, safe under
the global advisory lock. Mirrors the digests-drain self-heal.
- legal-halacha-drain.config.cjs: pm2 cron (every 2h, conservative — Claude is
slow/rate-limited and each run adds to the chair's pending_review queue).
drain_halacha_queue.py stays on claude_session (high reasoning quality for
holding/ratio; NOT moved to Gemini). SCRIPTS.md.
The chair-approval gate (INV-G10) is untouched — this only produces halachot;
Daphna still approves each in /approvals.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Records the now-complete corpus citation graph: why native not Obsidian (G2),
the 6 opt-in node layers (precedent/topic/practice-area · halacha · gaps ·
digests), node size/color semantics, the Graph Analysis metrics
(PageRank/betweenness/community via web/graph_metrics.py), navigation, the
/api/graph/* endpoints, the key files, a how-to-extend recipe, the invariants
(G2/G5/UI2/UI4), and the PR history.
Adds docs/corpus-graph.md + a reference-table row in legal-ai/CLAUDE.md.
Docs only — no code change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`npm run api:types` — brings the generated src/lib/api/types.ts up to date
with the live FastAPI schema (UI1: types derive from the OpenAPI SSoT). The
file had drifted; this regen captures the corpus-graph endpoints/models
(/api/graph/corpus, /api/graph/facets, /api/graph/node/{id}/neighborhood;
CorpusGraph / GraphNode / GraphFacets) plus accumulated changes from other
merged work. web-ui build passes against the regenerated types.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The /precedents metadata queue was stuck — 24 rows requested, nothing draining
them — and the agentic claude CLI hit error_max_turns on what is a single
structured text→JSON task (slow + flaky). Metadata extraction is bounded
extraction, the wrong fit for an agentic loop.
- gemini_session.py: query_json drop-in (gemini-2.5-flash, JSON mode, httpx —
no new SDK dep). Reads GEMINI_API_KEY (~/.env; SoT Infisical
nautilus:/external-apis/gemini). Host-side only — no LLM from the container.
- precedent_metadata_extractor: claude_session.query_json → gemini_session.
Validated live: rich, accurate fields (case_name/summary/appeal_subtype/tags).
- process_pending_extractions: kind-aware cooldown — metadata 2s (Gemini, fast),
halacha keeps 30s (Claude rate limits).
- drain_metadata_queue.py + legal-metadata-drain.config.cjs (pm2 cron */15) so
the queue never clogs again. SCRIPTS.md.
- X8 INV-FP5 updated: per-task engine choice (Gemini=bounded metadata,
claude_session=agentic halacha), both host-side, single canonical queue (G2).
Agentic/voice-sensitive work (writing, analysis, halacha) stays on claude_session
(Daphna's subscription). Gemini cost ≈ $0.10/1M tokens — negligible.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Enables the previously-disabled "הלכות" toggle. Each approved/published halacha
of a displayed precedent becomes a hal:<id> node linked to its parent
precedent (extracted_from); two cross-rule edges when both endpoints are in
view: corroborates (a later ruling cites the rule —
halacha_citation_corroboration) and equivalent (same principle from another
committee — equivalent_halachot). Node size = corroboration in-degree.
Backend (web/graph_api.py — read-only, G2):
- _halacha_nodes_and_edges(): halachot WHERE case_law_id in view AND
review_status IN (approved, published), LIMIT 600; rule_type carried in the
source_kind slot, rule_statement in note. Wired into both build functions
(gated via node_types). Metrics still exclude halacha edges (only cites/
precedent-typed feed PageRank). Validated: 185 halachot on the top-30
precedents; 20 corroboration + 5 equivalent edges in the corpus.
Frontend:
- graph.ts: GraphEdgeType += extracted_from.
- graph-filter-panel: "הלכות" toggle enabled (was disabled "שלב ב׳").
- graph-canvas: amber halacha nodes; edge colours — extracted_from (faint
amber), corroborates (amber), equivalent (violet).
- graph-node-panel: halacha branch — אזכורים + סוג כלל + rule text; "open in
library" deep-links to the parent precedent.
- graph-view: halacha added to node + edge legends.
web-ui build + lint pass. Invariants: G2 (SELECT-only), UI2 (no model change —
reuses note/source_kind/case_law_id slots).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Periodic safety net for the multi-judge approval panel: samples panel-approved
halachot, re-runs the same 3-judge KEEP vote, and surfaces any that now lean
DROP — candidate false-keeps a human should glance at. Report-only by default;
--flag reopens flips to pending_review. Baseline 0/15 on the 2026-06-07 batch.
Closes the loop the literature prescribes (Trust-or-Escalate / selective
prediction): monitor the auto-decision error rate rather than trusting it
blindly. Reuses halacha_panel_approve's judges (single source of truth).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
אותו יומון יכול להגיע כשני PDF שונים (re-send/forward → בייטים שונים →
content_hash dedup מפספס), אבל yomon_number ייחודי → ה-update ב-enrich מתנגש
על uq_digests_yomon_number. עכשיו enrich תופס את ההתנגשות, מוחק את השורה
הכפולה (היומון כבר קיים), ומחזיר status='duplicate' — כך ה-cron לא מנסה אותה
שוב ושוב. סוגר לולאת-retry אינסופית פוטנציאלית במערכת הלא-מאוישת.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Final corpus-graph PR. Connects the graph to the chair's workflow and rounds
out the Obsidian-grade interactions.
Backend (web/graph_api.py): neighborhood depth cap 2 → 3 (still bounded by
NODE_CAP_MAX).
Frontend:
- URL deep-link: /graph?focus=cl:<id> is read on mount and written on focus
change (router.replace, scroll:false). GraphView wrapped in <Suspense> per
Next 16's useSearchParams requirement.
- "הצג בגרף" button on the precedent detail page → /graph?focus=cl:<id>.
- Depth slider (1–3) in the focused overlay → useNodeNeighborhood(id, depth).
- Export PNG: grabs the rendered <canvas> from the area ref → toDataURL →
download; failures surface a toast (UI4).
- Rich node panel: precedent nodes fetch headnote/summary via the existing
usePrecedent hook (Skeleton while pending, error surfaced — UI4).
- Edge-type legend (ציטוט / נושא-תחום / יומון) added under the node legend.
Deferred (noted for a later pass): expand-in-place merge, search→camera-center.
web-ui build + lint pass. Invariants: G2 (depth change is read-only), UI4
(PNG + detail errors surfaced, not swallowed). api:types post-deploy.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
drain_digests רץ תחת flock (drainer יחיד), אז כל שורה 'processing' בתחילת ריצה
היא שריד מריצה קודמת שנקטעה באמצע-שורה (סשן/מכסה). מאפסים אותה ל-'pending'
לריצה חוזרת — סוגר את הפער האחרון ל-resume אוטומטי מלא ללא התערבות.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Chaim's idea: surface the downloaded "כל יום" digests in the graph. Each digest
COVERS the ruling it analyses — a corpus precedent when we have it (16), or a
synthesized gap node from its underlying_citation when we don't (269). So the
digest layer doubles as a discovery signal: it makes visible that the daily
feed overwhelmingly covers rulings NOT yet in the corpus.
Backend (web/graph_api.py — read-only, G2):
- "digest" added to VALID_NODE_TYPES (off by default).
- _digest_nodes_and_edges(): dig:<id> nodes from completed digests, `covers`
edge → cl:precedent (linked_case_law_id in view) or → gap:<underlying_citation>
(synthesized, deduped against the gap layer — real in-degree wins). Carries
concept_tag (label), headline_holding (note), underlying_court/date.
- _add_digests() appends the layer with gap dedup. Wired into both build
functions. GraphNode += note, digest_id. Gated via node_types (no app.py
change). Validated: 16 covers→precedent, 269 covers→gap.
Frontend:
- graph.ts: GraphNodeType += "digest"; GraphEdgeType += "covers"; node fields.
- graph-filter-panel: toggle "יומונים (כל יום)" (off by default).
- graph-canvas: digest = teal node (r=4); `covers` edges teal.
- graph-node-panel: digest branch — concept + holding + court/date + link to
/digests.
web-ui build + lint pass. Invariants: G2 (SELECT-only), UI2. api:types post-deploy.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Turns the graph into a gap-finder: the 247 unresolved internal citations
(a corpus precedent cites a ruling NOT in the corpus) collapse to 230 distinct
"gap" nodes — each sized by how many corpus precedents cite it, i.e. the
most-wanted missing precedent.
Backend (web/graph_api.py — read-only, G2):
- "gap" added to VALID_NODE_TYPES (NOT default → off unless requested).
- New _gap_nodes_and_edges(): gap:<normalized citation> nodes from
precedent_internal_citations WHERE cited_case_law_id IS NULL, sized by global
in-degree; cites edges only from precedents present in the view (dangling-edge
invariant holds). Best-effort enrichment from missing_precedents via exact
normalized-citation match → gap_status + missing_precedent_id. Validated:
230 gaps, top ע"א 3213/97 (cited 5×), 230/230 matched to missing_precedents.
- GraphNode += gap_status, missing_precedent_id. Metrics correctly exclude gap
edges (target not a precedent). No app.py change (gated via node_types).
Frontend:
- graph.ts: GraphNodeType += "gap"; node fields.
- graph-filter-panel: toggle "חוסרי מחקר (פסיקה חסרה)" (off by default).
- graph-canvas: gaps render as faint hollow dashed circles, never recoloured
by color-by; sized by citation count.
- graph-node-panel: gap branch — "מצוטטת ע״י N פסיקות" + status badge + link
to /missing-precedents.
web-ui build + lint pass. Invariants: G2 (SELECT-only), UI2 (model grows on
explicit Pydantic). api:types post-deploy.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The chair cannot review every pending halacha. Three independent-lineage judges
(Opus via claude_session · DeepSeek · Gemini-2.5-flash — #1 on LegalBench) vote
on the COARSE axis we proved reliable across models (92%): "is this a genuine,
keepable rule?". Only an agreed verdict acts; every split escalates to the chair
(INV-G10). Buckets: clean→KEEP?; nli_unsupported→entailment re-adjudication;
extraction-defects→re-extraction.
halacha_panel_calibrate.py calibrates the voting policy on the gold-set's
is_holding (the coarse label) per Trust-or-Escalate (ICLR 2025): unanimous →
94.9% precision / 78% coverage; majority → 92.9% / 99%; ZERO false-drops in
both (the panel never rejects a good rule). Chosen policy (chair-approved):
clean→majority-2/3, nli→asymmetric (majority-reject, unanimous-approve),
defects→re-extraction. Reversible (--apply backs up review_status+flags first).
Sources: Panel-of-LLM-Evaluators (PoLL) · Trust-or-Escalate (ICLR 2025,
arXiv:2407.18370) · selective-prediction / learning-to-defer.
Invariants: upholds G10 (human gate — splits escalate, panel only collapses the
queue) and G9 (provenance — reviewer records the panel + policy). Read paths only
in calibrate; --apply writes review_status/quality_flags reversibly with backup.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
COURT_FETCH_SHARED_SECRET + LEGAL_CHAT_SHARED_SECRET migrated to Infisical
nautilus:/legal-ai (2026-06-07). Updated the pm2 config comments: the stale
"migrate to Infisical once the MCP server is back" TODO is now done; local
env files remain the runtime source, Infisical is the SoT/record.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ה-cron של drain_digests הוא מנגנון ה-resume (pending-based, idempotent, host-side,
לא תלוי בסשן). חיזוק: אם enrich נכשל באמצע (מכסת claude נגמרה) השורה נשארה
'completed' עם שדות ריקים → לא היתה מטופלת שוב. עכשיו drain מאפס בתחילתו כל
digest 'completed' עם concept_tag ריק *וגם* underlying_citation ריק (= חילוץ
שמעולם לא נחת; שורה תקינה תמיד מכילה לפחות מראה-מקום) → pending לריצה חוזרת.
כך כל קטיעה/מכסה מתאוששת אוטומטית בריצת ה-cron הבאה.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds legal-metadata filtering and the payload to color by it (foundation for
the color-by selector in the analytics PR).
Backend (web/graph_api.py, web/app.py) — read-only, G2:
- GraphNode += court, date (ISO) — precedents carry them for filter/color-by.
- build_corpus_graph += server-side WHERE filters (G5): court, precedent_level,
chair, district, year_from, year_to (EXTRACT(YEAR FROM date)). Neighborhood
query also selects court/date.
- New GET /api/graph/facets (response_model GraphFacets, UI2) → distinct
courts/levels/chairs/districts so the UI doesn't hardcode Hebrew strings.
Frontend:
- graph.ts: GraphNode += court/date; GraphFilters += the six params;
buildParams; useGraphFacets() hook.
- graph-filter-panel: an "advanced" Accordion with court/precedent_level/chair/
district Selects (from facets) + year-from/year-to Selects.
- graph-view: new controls wired into filters; facets fetched and passed down.
Verified read-only against the live DB (precedent_level=עליון&year_from=2015
filters correctly; facets populated: 36 courts / 3 levels / 19 chairs / 4
districts). web-ui build + lint pass.
Invariants: G2 (SELECT-only via db.get_pool), G5 (filters server-side),
UI2 (explicit response_models). api:types to be regenerated post-deploy.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The spec said "supreme → Tier-0"; reality (PR #124) routes by נט-format
availability — נט המשפט (Tier-1) serves all courts incl. Supreme-with-נט-format,
and only serial-only Supreme falls to the (still-unbuilt) Tier-0 → manual.
Updated §0 source-distinction, §1 routing diagram, §5 risks (Tier-0 limitation
+ scheduled drain). Docs-only.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Live drain surfaced three issues:
1. Tier-0 needed `h2` (httpx http2) — added to the court-fetch extra.
2. Supreme cases that carry a נט-format number (e.g. בר"מ 72182-06-25) were
routed to the unvalidated Tier-0 and failed, even though נט המשפט serves
Supreme cases too. classify() now parses the file-month-year triple for
Supreme prefixes; the orchestrator routes by triple-availability:
נט-format present → Tier-1 (validated, all courts)
serial-only Supreme (עע"מ 5886/24) → Tier-0
neither → clear "no public route" failure
Validated live: בר"מ 72182-06-25 fetched via Tier-1 (5-page PDF).
3. A non-`RuntimeError` fetch exception (the h2 import error) left jobs stuck
in 'running'. The fetch block now catches any Exception → _record_failure
(INV-CF2/CF3), so a job always reaches a terminal state.
+ test_supreme_with_net_format_triple. Suite 11/11.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- scripts/drain_court_fetch.py: drives orchestrator.drain_pending (host-only;
no-op when queue empty). Mirrors drain_halacha_queue.py.
- scripts/legal-court-fetch-drain.config.cjs: pm2 cron (hourly :17, one-shot),
COURT_FETCH_DRAIN_CRON override.
- fix: orchestrator default service URL 127.0.0.1 → 10.0.1.1 (the service binds
the docker0 gateway; the host can't reach it on loopback). Found live — the
first drain failed "connection refused" until corrected.
- SCRIPTS.md entries.
Validated end-to-end in PRODUCTION on a real digest: עת"מ 43830-12-24
(החברה להגנת הטבע) fetched from נט המשפט → case_law (79 chunks, source_url),
digest relinked (INV-DIG3 closed), halacha queued pending_review. job=done.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
