Five enhancements to the precedent retrieval stack:
* **#44 HNSW indexes** for precedent_chunks + halachot (replacing IVFFlat
lists=50). Build time ~3s combined. Better recall@10 with pgvector 0.8.2.
* **#45 Halacha sweep** — 96 pending halachot at conf>=0.78 promoted to
approved (1141 → 1237). Cluster at conf=0.78 spot-checked OK. Applied
via psql only — env HALACHA_AUTO_APPROVE_THRESHOLD unchanged (0.80).
* **#43 MMR diversity** — search_precedent_library_hybrid now caps at
``max_per_case_law=2`` (default). Prevents one precedent dominating
top-10 when many of its chunks/halachot rank high. New helper
``_diversify_by_case_law`` in hybrid_search.py.
* **#46 Dynamic halacha boost** — replaces the static ``score+=0.05``
with ``score+=confidence*0.06``. Calibrated so avg-confidence (~0.85)
stays at +0.05; high-conf halachot get a slight extra lift, low-conf
ones get less. Behaviour preserved at the mean.
* **#41 BM25/tsvector hybrid + RRF**. Schema V12 adds STORED tsvector
columns ``precedent_chunks.content_tsv`` and ``halachot.rule_tsv``
(using simple config — Postgres has no Hebrew stemmer) + GIN indexes.
New ``db.search_precedent_library_lexical`` mirrors the semantic
function with ts_rank_cd over plainto_tsquery. ``hybrid_search``
runs sem+lex in parallel and fuses via RRF before rerank. Toggle:
env ``BM25_HYBRID_ENABLED`` (default true), graceful fallback to
semantic-only on lexical failure.
#40 (VOYAGE_RERANK_ENABLED) was already true in Coolify env; no change.
#42 (Claude Haiku query expansion) deferred — latency + cost concerns
warrant a separate plan; the bm25 lexical leg already recovers most of
the exact-string recall #42 was meant to address.
Closes TaskMaster #41, #43-#46.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Defense in depth — the MCP wrapper guard catches researcher uploads, but
the HTTP API (/api/precedent-library/upload) bypasses the wrapper and
calls services.precedent_library.ingest_precedent directly. The guard
now also lives in the service, so HTTP uploads of ערר/בל"מ citations
to the external corpus get rejected at the source.
Companion to DB constraint case_law_external_arar_check (applied via
psql) — three independent layers now enforce the same invariant.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Born-digital Hebrew PDFs from legal software often encode gershayim (״)
as double-yod (יי), producing the same corruption patterns as OCR.
The fixer was only called after Google Cloud Vision OCR — digitally
created PDFs that passed quality checks received no correction.
Changes:
- Apply _fix_hebrew_quotes() in the direct extraction path
- Add 'בליימ' → 'בל"מ' (בקשה להארכת מועד — systematic corruption in 1017-03-26)
- Add 'תמייא' → 'תמ"א' (תכנית מתאר ארצית)
- Update docstring to reflect the broader scope
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Wrap date.fromisoformat() in try/except in case_update tool — prevents
unhandled ValueError from surfacing as 500; FastAPI now catches it as 422
- Add DialogDescription (sr-only) to 5 dialogs missing aria-describedby:
documents-panel preview + delete, drafts-panel delete + feedback, link-related-dialog
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fields existed in DB and Precedent type but were missing from:
- PrecedentUpdateRequest (backend model)
- update_case_law allowed set (db layer)
- PrecedentPatch (frontend type)
- precedent-edit-sheet form state, inputs, and patch payload
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Archived cases have archived_at IS NOT NULL — they are not "stuck",
they are done. The stale query was missing this filter.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Document new daphna-procedural-patterns.md cataloging the
"appraiser clarification request" interim-decision pattern observed in
8174-24 — structure only, not phrasing (case is an outlier example).
- daphna-decision-tree.md §0.5: gating question before main tree
- legal-ceo.md voice docs table: register procedural patterns doc
- legal-writer.md: mandatory consultation when pattern_tag is set,
with explicit warning against copying 8174-24 wording
Approved via interaction request_confirmation (CMPA-15) 2026-05-17.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The legal-analyst agent was generating a longer placeholder form
[ימולא ע"י יו"ר הוועדה — עמדה/הנחיה לגבי סוגיה זו שתשמש את סוכן הכתיבה]
which _is_placeholder() did not match (substring check fails because ] is
further along in the longer form). Result: UI showed "✓ עמדה נקבעה" (green)
for all 4 issues even though no chair direction had been entered.
Fixes:
1. research_md.py: add regex fallback — any text starting with [ימולא is a placeholder
2. legal-analyst.md: template now emits the standard short placeholder only
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- עדכון טבלת מצב: כל המודלים מסונכרנים (instructions = DB)
- החלפת טבלת בעיות בטבלת סטטוס תיקונים עם commit references
- הוסף טבלת שינויים נוספים מהסשן
- הערה: Skills CMPA=6 עיצוב מכוון, verify מאשר "0 need sync"
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Model drift (instructions → match DB):
- CEO: claude-sonnet-4-6 → claude-opus-4-6 (DB runs opus; CEO needs opus quality)
- מנתח/כותב/מגיה: claude-opus-4-7 → claude-opus-4-6 (DB runs 4-6; no 4-7 in adapter)
legal-proofreader.md:
- {issue-id} placeholder → $PAPERCLIP_TASK_ID בשני המקומות (done + blocked)
legal-researcher.md:
- הוסף reference ל-HEARTBEAT.md בראש הקובץ
legal-qa.md:
- הבהרת שיטת בדיקת corpus_queries_logged: grep ידני בלבד, לא validate_decision
CLAUDE.md (curator):
- הוסף תהליך אישור הצעות curator: comment → חיים מאשר → commits ל-SKILL.md/lessons.md
maxConcurrentRuns CEO: כבר 2 ב-DB — לא נדרש שינוי
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
CEO (legal-ceo.md):
- הסרת company UUID ו-project UUID קשוחים בדוגמת יצירת issue
- שימוש ב-$PAPERCLIP_COMPANY_ID לחברה
- project_id נשלף דינמית מה-issue ההורה דרך $PAPERCLIP_TASK_ID
researcher (legal-researcher.md):
- הוסף mcp__legal-ai__search_internal_decisions לרשימת tools
- הוסף סעיף 2ב.2א המסביר את ההבדל: search_decisions = דפנה בלבד;
search_internal_decisions = כל ועדות הערר בכל המחוזות
- הוראות מתי להשתמש + אזהרת היררכיה (ועדת ערר < מחוזי)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
7-agent parallel audit of all Paperclip agents (CEO, analyst,
researcher, writer, QA, exporter, proofreader, curator).
Found 12 issues including 3 critical:
- Exporter: V vs v naming mismatch in DOCX versioning
- Exporter: case.status not updated to exported after export
- Researcher: section ז missing from case 8174-24
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Document emit_case_status_webhook flow and plugin integration
- Document stale-case-reminder and weekly-feedback-analysis jobs
- Fix paperclip_api.py vs paperclip_client.py (both exist, api.py is current)
- Add warning: weekly-feedback-job CEO has no issueId
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
CEO wakes for weekly-feedback-job via agents.invoke without issueId,
so $PAPERCLIP_TASK_ID is empty. Removed steps 4-5 (comment + close
issue) from handler — now file-write only with stdout logging.
Also commits pending docs and agent instructions from prior session.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
If pc_wake_ceo fails, the endpoint now raises HTTP 502 and skips the
case_update to processing — preventing cases from silently getting stuck
with no CEO running. Also adds `processing` to CEO routing table and
updates case_list docstring with full status list.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Fix keyboard navigation bug: React was reusing the submit button DOM element
when transitioning "הבא" → "צור תיק", retaining focus and causing Enter to
auto-submit step 3. Added key props to force element replacement.
- CaseEditDialog now covers all wizard fields: appellants, respondents,
property_address, permit_number (in addition to existing title, subject,
hearing_date, expected_outcome, notes).
- When case title changes, Paperclip project name is updated in background
via new update_project_name() in paperclip_client.py.
- Extended CaseUpdateRequest, case_update MCP tool, and caseUpdateSchema
to carry the new fields end-to-end.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace deprecated datetime.utcnow() with datetime.now(timezone.utc)
to avoid Python 3.12+ DeprecationWarning.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- P3-T1: --check-instructions flag + check_instructions() prints a table of all
agents' instructionsFilePath with status (✅ OK / ❌ MISSING / ⚠ NOT SET),
size, mtime, and ⚠ DRIFT when file has changed since last sync
- P3-T2: --apply now runs a pre-flight check on master agents and aborts if any
instruction file is missing, before touching the DB or calling any API
- P3-T3: get_claude_md_mtime() helper; --apply stamps claude_md_mtime and
claude_md_last_synced into each mirror agent's metadata via the PATCH call
- P3-T4: alias check-agents added to ~/.bashrc
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
GET /api/cases/stale?days=N — returns cases not updated in N days (default 3)
that are not in 'final' or 'new' status, with days_stale count.
GET /api/chair-feedback/weekly-summary?days=N — returns chair feedback from
the last N days (default 7) as a Hebrew bullet-list summary for CEO agent.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
asyncpg returns JSONB columns as raw JSON strings when no type codec is
configured (only pgvector is registered in _init_connection). The stored
value is a correct JSONB array (jsonb_typeof=array confirmed), but
asyncpg decodes it as str. Parse it explicitly in the GET handler so
the frontend receives the correct Python list/dict.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
asyncpg cannot encode a Python list as JSONB directly (expects str).
Passing str with ::jsonb causes double-encoding (stored as JSONB string).
Solution: json.dumps() the value → pass as text → PostgreSQL parses
with ::text::jsonb cast, storing it as the correct JSONB array/object.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Pass req.value directly to asyncpg instead of json.dumps(req.value).
When a Python string was passed with ::jsonb, asyncpg encoded it as a
JSONB string (not an array), causing the frontend spread operator to
split it into individual characters — one textarea per character.
Also fix typo in DISCUSSION_RULES default: "אסה" → "מאסה".
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two bugs caused all 5 interim blocks to fail with "Claude CLI failed
(exit 1): unknown error":
1. source_context was embedded BOTH inside the prompt template (via
{source_context}) AND prepended again in write_block — doubling every
block's context size (232K chars × 2 = 465K chars).
2. _build_source_context loaded all 9 case documents for every block
regardless of relevance.
Fixes:
- Remove the duplicate source_context prepend in write_block; the
template already contains it via {source_context}
- Add per-block document filtering (_BLOCK_DOC_TYPES): block-he/zayin →
empty, block-chet → protocol only, block-tet → appraisals only
- Add 400K char guard before calling claude -p with a descriptive error
(vs opaque "exit 1: unknown error")
- Add prompt-size warning and size info in claude_session error messages
Result: block-he 0 chars, block-zayin 0 chars, block-vav ~172K,
block-chet ~45K, block-tet ~300K (all under 400K limit)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- DB: add 'all_committees' virtual source_kind covering internal_committee
+ external_upload appeals_committee rows in one query
- DB: stats now count all case_law rows (not just external_upload),
fixing the precedents_total that excluded 44 internal-committee records
- UI: courts table filters to source_type=court_ruling only;
committees table uses the new all_committees query
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Both CourtRow and CommitteeRow citation cells are now Next.js Links
→ /precedents/{id}, letting users navigate directly from the list.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace useEffect-based form hydration with React's approved derived-state
pattern (setState-during-render). This eliminates the one-frame flash where
the precedent_level Select showed "—" before useEffect fired, and fixes
cases where the same record reference returned from TanStack cache caused
useEffect to not re-run after save+invalidate.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add ability to mark case_law records as related (e.g. same appeal
through ועדת ערר → מנהלי → עליון):
- DB: case_law_relations join table (bidirectional, V11 migration)
- DB CRUD: add/remove/get_case_law_relations
- Service: get_precedent() now returns related_cases[]
- MCP: precedent_link_cases + precedent_unlink_cases tools
- REST: POST/DELETE /api/precedent-library/{id}/relations
- UI: RelatedCasesSection on detail page with search dialog and unlink
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
A/B test (2026-05-05) showed DeepSeek V4-Pro is 2-3x faster and ~20x cheaper
than Sonnet for style/lexicon pattern analysis, with comparable quality.
Adds adapters/deepseek-paperclip-adapter/ package, documents adapter requirements
(env injection, run-id headers), updates CLAUDE.md with adapter integration notes,
and records lessons from ערר 1200-25 (block order for 1xxx, "להלן מתוך" pattern,
expanded factual background, bridge planning analysis, flat heading structure).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The metadata extractor occasionally stuffs the practice_area enum
(``betterment_levy``, ``rishuy_uvniya``, ``compensation_197``) into
the free-text ``appeal_subtype`` column. The edit sheet then showed the
raw English string in the "תת-סוג" input.
When initialising the form, run the value through ``appealSubtypeLabel``
which maps known practice-area enum values to their Hebrew label and
returns anything else unchanged. The user can then edit normally; on
save the Hebrew sticks, so the next view is also clean.
reextract_metadata / reextract_halachot extract & apply but never cleared
metadata_extraction_requested_at / halacha_extraction_requested_at —
only the bulk worker (process_pending_extractions) did. Result: clicking
"חלץ מטא-דאטה" on the edit sheet (or calling precedent_extract_metadata
directly) left the row stuck in the queue forever, with the UI badge
showing "ממתין לחילוץ" even after extraction succeeded.
Mirror the worker's behaviour: on success ('completed' / 'no_changes' /
'no_halachot'), call db.clear_extraction_request to drain the queue.
Coolify deploy required for the FastAPI container; local MCP server
needs a process restart for the change to take effect (long-running).
Earlier commit afcc481 opened request_metadata_extraction and
request_halacha_extraction to all source kinds — but
list_pending_extraction_requests still hard-filtered to external_upload.
Result: stamping a queue request on an internal_committee row succeeded
silently, but the worker (and the queue badge) never saw it. Even with
the auto-wakeup added in c7132ba the CEO would wake, find 0 pending
items, and exit.
Drop the legacy filter so the queue listing matches the writer side.
Coolify deploy required for the FastAPI container to pick this up.
The "חלץ מטא-דאטה" / "חלץ הלכות" buttons in the UI used to only stamp
the queue (set metadata_extraction_requested_at / halacha_extraction_requested_at)
and rely on a human running `mcp__legal-ai__precedent_process_pending` from
local Claude Code to drain it.
That left the user with an unintuitive two-step flow: click button → run
local MCP tool. Meanwhile, the upload endpoint already does the right
thing — after ingest succeeds it calls `pc_wake_for_precedent_extraction`,
which creates a Paperclip issue, assigns it to the CEO, and wakes them
to run `precedent_process_pending` automatically.
Add the same wakeup call to the manual request-metadata / request-halachot
endpoints. Now clicking the button is sufficient — the CEO picks it up
and drains the queue without manual intervention.
Best-effort: matches the upload flow's failure semantics. The queue stamp
still happens even if the wakeup fails, so the user can fall back to the
manual MCP tool when needed. The wakeup outcome is included in the
response under `wakeup` for observability.
Coolify deploy required for the FastAPI container to pick this up.
When a precedent has not had successful halacha extraction yet, show a
small wand icon between the edit and delete buttons. Clicking it queues
the precedent for the local MCP worker (request-halachot endpoint).
Visibility rule (`needsHalachaExtraction`): show when text extraction is
complete AND halacha status is "pending without requested_at" (never
tried) or "failed" (allow retry). Hide while processing, after
completion, or when already queued — to avoid duplicate requests.
Pairs with the metadata-extract button on the edit sheet.
The "חלץ מטא-דאטה" / "חלץ הלכות" buttons in the UI were returning 404
for any precedent with `source_kind != 'external_upload'`. The original
restriction was meant to keep LLM extraction off internal-committee
imports (their metadata supposedly came from the case file system),
but the same precedent rows can still need re-extraction when ingest
produces broken data — e.g. the corrupted `subject_tags` value
`['[','"','ה','י',...]` that motivated this change (an early ingest
stored a JSON literal into a TEXT[] column, which Postgres split into
single chars).
Two changes here:
1. db.request_metadata_extraction / request_halacha_extraction:
drop the `AND source_kind='external_upload'` filter. The extractor
already preserves user values (only fills empty fields), so this
is safe.
2. precedent_metadata_extractor.extract_and_apply: detect the
character-by-character corruption above and treat it as empty so
the freshly-extracted tags actually replace the broken ones.
Heuristic: 3+ elements where every element is at most 2 chars
(legitimate tags are multi-character Hebrew words).
Coolify deploy required for the FastAPI container to pick this up.
The Hermes Knowledge Curator's hermes-curator.md says it must be able to
read both DOCX and PDF final decisions. The original implementation
hardcoded the .docx extension only. Extend to try .docx → .pdf → .doc →
.rtf → .txt → .md, returning the first match. extractor.extract_text
already supports all six formats, so no extractor changes needed.
If none found, the not_found response now includes the tried_extensions
list so the caller knows what was attempted.
Verified on case 1130-25 (.docx still picked first) and tested via
`curator-cmp mcp test legal-ai`.
The Knowledge Curator (Hermes) couldn't read סופי-{case}.docx because
document_get_text only works on rows in the documents table — the final
file is just a copy in the case's exports/ directory, not a tracked
document. CMP-71 hit this and produced an unproductive interaction
asking the user how to fix the access issue.
Add a new MCP tool that:
- Locates exports/סופי-{case_number}.docx via config.find_case_dir
- Extracts text using the existing extractor service (python-docx based)
- Returns JSON with status + text + page_count + truncation info
- Optional max_chars cap for large decisions
Smoke test on case 1130-25: 400-char preview returns proper Hebrew text
beginning with "לפנינו ערר על החלטת הוועדה המקומית...".
The local MCP server reloads on next Hermes spawn (stdio mode), so the
tool is immediately available — no Coolify deploy needed.
Curator's promptTemplate (DB-stored) updated to use the new tool as the
primary path for reading the final.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
