Compare commits
6 Commits
worktree-r
...
worktree-m
| Author | SHA1 | Date | |
|---|---|---|---|
| adb9586055 | |||
| 09c737bddf | |||
| 95571e0f05 | |||
| 1f2ff4f1ec | |||
| 6c870ac691 | |||
| ff3a2f398c |
@@ -52,6 +52,41 @@
|
||||
- **אפס עלות API** — claude CLI משתמש ב-claude.ai subscription של chaim. הנחת היסוד של `claude_session.py` (claude CLI מקומי בלבד) נשמרת.
|
||||
- Coolify dependency: ה-Service Definition של legal-ai חייב להכיל `extra_hosts: host.docker.internal:host-gateway` (אחרת ה-proxy יקבל ConnectError).
|
||||
|
||||
### legal-mcp-http — שרת ה-MCP ב-HTTP (מאוגוסט 2026)
|
||||
- פורט: `127.0.0.1:8790` (loopback בלבד) · קונפיג: [`scripts/legal-mcp-http.config.cjs`](../scripts/legal-mcp-http.config.cjs)
|
||||
- **הדלת של סוכני הפלטפורמה** אל 108 כלי ה-MCP. סשן אינטראקטיבי מגיע לאותו שרת ב-stdio דרך `~/.claude.json` — **אותו קוד, שתי תחבורות** (G2).
|
||||
- שער Bearer (`MCP_HTTP_SHARED_SECRET`); השרת **מסרב לעלות** בלי טוקן. בדיקה: ללא טוקן → `401`, עם טוקן → `200`.
|
||||
- **⚠️ שינוי קוד ב-`mcp-server/` לא נכנס לתוקף עד `pm2 restart legal-mcp-http`.** זה תהליך ארוך-חיים, בניגוד ל-stdio שנטען מחדש בכל סשן. אחרי מיזוג שנוגע ב-`mcp-server/` — להפעיל מחדש, אחרת הסוכנים ירוצו על קוד ישן בשקט.
|
||||
|
||||
> #### ✅ תצורת האחסון והאחזור מיושרת לקונטיינר (2026-08-05)
|
||||
> `STORAGE_BACKEND=s3` ו-`MULTIMODAL_ENABLED=true` — זהים לקונטיינר, כך שמסמך
|
||||
> שסוכן כותב דרך MCP הוא זה שה-web קורא, ושתי הדלתות מדרגות את אותו קורפוס
|
||||
> באותה צורה (G2).
|
||||
>
|
||||
> **הסודות:** `MINIO_ENDPOINT` · `MINIO_ACCESS_KEY` · `MINIO_SECRET_KEY` נוצרו
|
||||
> ב-Infisical תחת `/apps/legal-ai` (tag `credentials`), ונטענים ל-`~/.legal-mcp-http.env`
|
||||
> (chmod 600) — אותו דפוס כמו `MCP_HTTP_SHARED_SECRET`. **רוטציה:** לעדכן
|
||||
> ב-Infisical → למשוך מחדש לקובץ → `pm2 restart legal-mcp-http`.
|
||||
>
|
||||
> **⚠️ מלכודת — `aioboto3` מיובא עצלנית.** מסלול ה-S3 מייבא אותו רק בקריאה
|
||||
> הראשונה, ולכן venv בלי החבילה **עולה בהצלחה** ונשבר רק בפעולת-הקובץ הראשונה
|
||||
> — כשל שקט עד לשימוש. הוא מוצהר ב-`mcp-server/pyproject.toml`, אבל לא היה
|
||||
> מותקן ב-venv המקומי. אחרי כל בנייה מחדש של ה-venv:
|
||||
> `~/legal-ai/mcp-server/.venv/bin/pip install -e ~/legal-ai/mcp-server`.
|
||||
>
|
||||
> **אימות לפני הפעלה** (הרץ מהמארח, לא מהקונטיינר):
|
||||
> ```bash
|
||||
> cd ~/legal-ai/mcp-server && set -a && . ~/.legal-mcp-http.env && set +a
|
||||
> STORAGE_BACKEND=s3 HOME=/home/chaim DOTENV_PATH=/home/chaim/.env \
|
||||
> .venv/bin/python -c "import asyncio;from legal_mcp.services import storage
|
||||
> async def m():
|
||||
> st=storage.get_storage()
|
||||
> print(len(await st.list_keys('', bucket=storage.Bucket.DOCUMENTS)))
|
||||
> asyncio.run(m())"
|
||||
> ```
|
||||
> מצופה: מספר האובייקטים בדלי (היו 2,283 ב-2026-08-05). שגיאת
|
||||
> `ModuleNotFoundError: aioboto3` = ה-venv חסר את התלות.
|
||||
|
||||
---
|
||||
|
||||
## מבנה תיקיות
|
||||
|
||||
@@ -20,6 +20,7 @@ from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import time
|
||||
from uuid import UUID
|
||||
|
||||
from legal_mcp.services import (
|
||||
@@ -74,7 +75,92 @@ def _passes_floor(hit: dict) -> bool:
|
||||
_RETRIEVAL_BUDGET_S = 22.0
|
||||
|
||||
|
||||
async def build_view(case_number: str) -> dict:
|
||||
#: Cached views, keyed by case number → (fingerprint, expires_at|None, view).
|
||||
#: Assembling a view costs 20+ seconds of Voyage embeds and vector searches, and
|
||||
#: the result changes only when its inputs do. Keyed by a FINGERPRINT of those
|
||||
#: inputs rather than a guessed TTL: a chair who verifies a precedent must see it
|
||||
#: reflected on the next load, not after a timer expires.
|
||||
_view_cache: dict[str, tuple[str, float | None, dict]] = {}
|
||||
|
||||
#: One lock per case so two concurrent loads of the same page don't both compute.
|
||||
#: Reload-during-a-slow-load is how the original outage compounded into Postgres
|
||||
#: deadlocks; the second reader now waits for the first and gets its result.
|
||||
_view_locks: dict[str, asyncio.Lock] = {}
|
||||
|
||||
#: A partial view (retrieval budget exhausted) is still worth caching — otherwise
|
||||
#: every reload re-pays 22 seconds — but only briefly, so the missing suggestions
|
||||
#: get another chance while the corpus is quieter.
|
||||
_PARTIAL_TTL_S = 90.0
|
||||
|
||||
|
||||
async def _inputs_fingerprint(case_id: UUID) -> str:
|
||||
"""Cheap signature of everything the view is derived from.
|
||||
|
||||
Changes when: arguments are re-aggregated, the chair attaches or verifies a
|
||||
precedent, or the corpus grows (new rulings change the suggestions). One
|
||||
round-trip — the point is to be far cheaper than the 20s it guards.
|
||||
"""
|
||||
pool = await db.get_pool()
|
||||
async with pool.acquire() as conn:
|
||||
row = await conn.fetchrow(
|
||||
"""
|
||||
SELECT
|
||||
(SELECT count(*)::text || ':' || coalesce(max(updated_at)::text, '-')
|
||||
FROM legal_arguments WHERE case_id = $1) AS args,
|
||||
(SELECT count(*)::text || ':' || coalesce(max(updated_at)::text, '-')
|
||||
FROM case_precedents WHERE case_id = $1) AS attached,
|
||||
(SELECT count(*)::text || ':' || coalesce(max(created_at)::text, '-')
|
||||
FROM case_law WHERE searchable = true) AS corpus
|
||||
""",
|
||||
case_id,
|
||||
)
|
||||
return f"{row['args']}|{row['attached']}|{row['corpus']}"
|
||||
|
||||
|
||||
def invalidate(case_number: str = "") -> None:
|
||||
"""Drop cached views — one case, or all when called with no argument.
|
||||
|
||||
Callers that mutate the inputs may use this for immediacy, but correctness
|
||||
does not depend on it: the fingerprint check catches any change on the next
|
||||
read regardless of whether anyone remembered to invalidate (G1 — the guard
|
||||
lives at the source of truth, not in every writer's memory).
|
||||
"""
|
||||
if case_number:
|
||||
_view_cache.pop(case_number, None)
|
||||
else:
|
||||
_view_cache.clear()
|
||||
|
||||
|
||||
async def build_view(case_number: str, *, use_cache: bool = True) -> dict:
|
||||
"""Assemble the citation-verification view, served from cache when unchanged."""
|
||||
if not use_cache:
|
||||
return await _build_view_uncached(case_number)
|
||||
|
||||
lock = _view_locks.setdefault(case_number, asyncio.Lock())
|
||||
async with lock:
|
||||
case = await db.get_case_by_number(case_number)
|
||||
if not case:
|
||||
return {"status": "case_not_found", "case_number": case_number,
|
||||
"arguments": []}
|
||||
cid = case["id"]
|
||||
fp = await _inputs_fingerprint(UUID(cid) if isinstance(cid, str) else cid)
|
||||
|
||||
hit = _view_cache.get(case_number)
|
||||
if hit is not None:
|
||||
cached_fp, expires_at, view = hit
|
||||
fresh = cached_fp == fp and (expires_at is None or time.monotonic() < expires_at)
|
||||
if fresh:
|
||||
return {**view, "cached": True}
|
||||
|
||||
view = await _build_view_uncached(case_number)
|
||||
if view.get("status") == "ok":
|
||||
expires_at = (None if view.get("retrieval_complete", True)
|
||||
else time.monotonic() + _PARTIAL_TTL_S)
|
||||
_view_cache[case_number] = (fp, expires_at, view)
|
||||
return {**view, "cached": False}
|
||||
|
||||
|
||||
async def _build_view_uncached(case_number: str) -> dict:
|
||||
case = await db.get_case_by_number(case_number)
|
||||
if not case:
|
||||
return {"status": "case_not_found", "case_number": case_number, "arguments": []}
|
||||
|
||||
125
mcp-server/tests/test_citation_view_cache.py
Normal file
125
mcp-server/tests/test_citation_view_cache.py
Normal file
@@ -0,0 +1,125 @@
|
||||
"""The citation view is cached by input fingerprint, not by a guessed TTL.
|
||||
|
||||
Assembling it costs 20+ seconds of Voyage embeds and vector searches. A TTL
|
||||
would force a choice between staleness and cost: a chair who verifies a
|
||||
precedent must see it on the very next load, and a timer cannot promise that.
|
||||
Keying on a fingerprint of the inputs (arguments · attachments · corpus) makes
|
||||
freshness a property of the data rather than of the clock.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
|
||||
import pytest
|
||||
|
||||
from legal_mcp.services import case_citation_verification as ccv
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _clean_cache():
|
||||
ccv.invalidate()
|
||||
yield
|
||||
ccv.invalidate()
|
||||
|
||||
|
||||
def _stub(monkeypatch, fingerprint: str, calls: list, *, complete: bool = True):
|
||||
"""Point build_view at a fake case, a controllable fingerprint, and a
|
||||
counted assembler, so these tests never touch the corpus."""
|
||||
async def fake_case(_cn):
|
||||
return {"id": "00000000-0000-0000-0000-000000000001"}
|
||||
|
||||
async def fake_fp(_cid):
|
||||
return fingerprint
|
||||
|
||||
async def fake_build(cn):
|
||||
calls.append(cn)
|
||||
return {"status": "ok", "case_number": cn, "arguments": [],
|
||||
"retrieval_complete": complete, "summary": {}}
|
||||
|
||||
monkeypatch.setattr(ccv.db, "get_case_by_number", fake_case)
|
||||
monkeypatch.setattr(ccv, "_inputs_fingerprint", fake_fp)
|
||||
monkeypatch.setattr(ccv, "_build_view_uncached", fake_build)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_second_load_is_served_from_cache(monkeypatch):
|
||||
calls: list = []
|
||||
_stub(monkeypatch, "fp-1", calls)
|
||||
first = await ccv.build_view("1069-04-26")
|
||||
second = await ccv.build_view("1069-04-26")
|
||||
assert first["cached"] is False
|
||||
assert second["cached"] is True
|
||||
assert len(calls) == 1, "the expensive assembly must run once"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_changed_inputs_invalidate_without_anyone_calling_invalidate(monkeypatch):
|
||||
"""The chair verifies a precedent → fingerprint moves → next load rebuilds.
|
||||
Correctness must not depend on a writer remembering to clear the cache."""
|
||||
calls: list = []
|
||||
_stub(monkeypatch, "fp-before", calls)
|
||||
await ccv.build_view("1069-04-26")
|
||||
|
||||
_stub(monkeypatch, "fp-after", calls) # e.g. a new case_precedents row
|
||||
again = await ccv.build_view("1069-04-26")
|
||||
assert again["cached"] is False
|
||||
assert len(calls) == 2
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_concurrent_loads_do_not_stampede(monkeypatch):
|
||||
"""Reload-during-a-slow-load is how the original outage compounded into
|
||||
Postgres deadlocks. The later readers must wait, not pile on."""
|
||||
calls: list = []
|
||||
|
||||
async def fake_case(_cn):
|
||||
return {"id": "00000000-0000-0000-0000-000000000001"}
|
||||
|
||||
async def fake_fp(_cid):
|
||||
return "fp-1"
|
||||
|
||||
async def slow_build(cn):
|
||||
calls.append(cn)
|
||||
await asyncio.sleep(0.05)
|
||||
return {"status": "ok", "case_number": cn, "arguments": [],
|
||||
"retrieval_complete": True, "summary": {}}
|
||||
|
||||
monkeypatch.setattr(ccv.db, "get_case_by_number", fake_case)
|
||||
monkeypatch.setattr(ccv, "_inputs_fingerprint", fake_fp)
|
||||
monkeypatch.setattr(ccv, "_build_view_uncached", slow_build)
|
||||
|
||||
await asyncio.gather(*(ccv.build_view("1069-04-26") for _ in range(6)))
|
||||
assert len(calls) == 1, "six concurrent loads, one assembly"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_partial_view_expires_so_missing_suggestions_get_retried(monkeypatch):
|
||||
"""A budget-exhausted view is cached to stop reload storms, but it must not
|
||||
become permanent — the arguments with no suggestions deserve another try."""
|
||||
calls: list = []
|
||||
_stub(monkeypatch, "fp-1", calls, complete=False)
|
||||
await ccv.build_view("1069-04-26")
|
||||
_fp, expires_at, _view = ccv._view_cache["1069-04-26"]
|
||||
assert expires_at is not None, "a partial view must carry an expiry"
|
||||
|
||||
monkeypatch.setattr(ccv.time, "monotonic", lambda: expires_at + 1)
|
||||
after = await ccv.build_view("1069-04-26")
|
||||
assert after["cached"] is False
|
||||
assert len(calls) == 2
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_complete_view_does_not_expire_on_a_timer(monkeypatch):
|
||||
calls: list = []
|
||||
_stub(monkeypatch, "fp-1", calls, complete=True)
|
||||
await ccv.build_view("1069-04-26")
|
||||
_fp, expires_at, _view = ccv._view_cache["1069-04-26"]
|
||||
assert expires_at is None, "freshness comes from the fingerprint, not a clock"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_use_cache_false_always_rebuilds(monkeypatch):
|
||||
calls: list = []
|
||||
_stub(monkeypatch, "fp-1", calls)
|
||||
await ccv.build_view("1069-04-26")
|
||||
await ccv.build_view("1069-04-26", use_cache=False)
|
||||
assert len(calls) == 2
|
||||
@@ -56,6 +56,26 @@ const env = {
|
||||
// transports read exactly the same corpus.
|
||||
DOTENV_PATH: "/home/chaim/.env",
|
||||
DATA_DIR: "/home/chaim/legal-ai/data",
|
||||
// Retrieval flags must match the container's, or the two doors into the same
|
||||
// corpus rank differently (G2 — a parallel path that drifts). The container
|
||||
// runs MULTIMODAL_ENABLED=true; without this the MCP door skipped the
|
||||
// page-image merge entirely, so an agent and the web UI could answer the same
|
||||
// question from different result sets. ~/.env sets neither, so both fell to
|
||||
// the code defaults (false) — the drift was silent.
|
||||
//
|
||||
// Blob storage must match the container too, for the same reason: a document
|
||||
// an agent writes through MCP has to be the one the web reads back. The MinIO
|
||||
// credentials now live in Infisical (/apps/legal-ai) and are loaded from the
|
||||
// runtime file below, exactly like MCP_HTTP_SHARED_SECRET.
|
||||
//
|
||||
// Verified from the host before enabling: put/get/delete round-trip against
|
||||
// https://s3.nautilus.marcusgroup.org, plus a read of an existing production
|
||||
// blob out of 2,283 objects. Note the S3 path imports aioboto3 LAZILY, so a
|
||||
// missing dependency does not fail at boot — it fails on the first blob
|
||||
// operation. It is declared in mcp-server/pyproject.toml; make sure the venv
|
||||
// actually has it (`.venv/bin/pip install -e mcp-server`).
|
||||
STORAGE_BACKEND: "s3",
|
||||
MULTIMODAL_ENABLED: "true",
|
||||
MCP_TRANSPORT: "streamable-http",
|
||||
MCP_HTTP_HOST: "127.0.0.1",
|
||||
MCP_HTTP_PORT: "8790",
|
||||
|
||||
Reference in New Issue
Block a user