Chaim
d95a36f310
The /precedents metadata queue was stuck — 24 rows requested, nothing draining them — and the agentic claude CLI hit error_max_turns on what is a single structured text→JSON task (slow + flaky). Metadata extraction is bounded extraction, the wrong fit for an agentic loop. - gemini_session.py: query_json drop-in (gemini-2.5-flash, JSON mode, httpx — no new SDK dep). Reads GEMINI_API_KEY (~/.env; SoT Infisical nautilus:/external-apis/gemini). Host-side only — no LLM from the container. - precedent_metadata_extractor: claude_session.query_json → gemini_session. Validated live: rich, accurate fields (case_name/summary/appeal_subtype/tags). - process_pending_extractions: kind-aware cooldown — metadata 2s (Gemini, fast), halacha keeps 30s (Claude rate limits). - drain_metadata_queue.py + legal-metadata-drain.config.cjs (pm2 cron */15) so the queue never clogs again. SCRIPTS.md. - X8 INV-FP5 updated: per-task engine choice (Gemini=bounded metadata, claude_session=agentic halacha), both host-side, single canonical queue (G2). Agentic/voice-sensitive work (writing, analysis, halacha) stays on claude_session (Daphna's subscription). Gemini cost ≈ $0.10/1M tokens — negligible. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
49 lines
1.7 KiB
Python
49 lines
1.7 KiB
Python
"""Drain the precedent metadata-extraction queue.
|
|
|
|
Calls ``process_pending_extractions(kind='metadata')`` in batches until the
|
|
queue is empty (two consecutive zero-progress rounds). Metadata extraction runs
|
|
on **Gemini Flash** (structured JSON) — fast and reliable, unlike the agentic
|
|
claude CLI which hit ``error_max_turns`` on this bounded task. A no-op (fast)
|
|
when the queue is empty.
|
|
|
|
Host-only (reads GEMINI_API_KEY + POSTGRES_URL from ~/.env via legal_mcp.config).
|
|
Scheduled by ``legal-metadata-drain`` (pm2 cron); also runnable by hand:
|
|
|
|
mcp-server/.venv/bin/python scripts/drain_metadata_queue.py [batch]
|
|
"""
|
|
|
|
import asyncio
|
|
import os
|
|
import sys
|
|
|
|
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "mcp-server", "src"))
|
|
|
|
from legal_mcp.services import precedent_library as pl
|
|
|
|
|
|
async def main() -> int:
|
|
batch = int(sys.argv[1]) if len(sys.argv) > 1 else 10
|
|
total = 0
|
|
empty_rounds = 0
|
|
rnd = 0
|
|
while empty_rounds < 2:
|
|
rnd += 1
|
|
out = await pl.process_pending_extractions(kind="metadata", limit=batch)
|
|
processed = out.get("processed", 0)
|
|
total += processed
|
|
print(f"[round {rnd}] processed={processed} total_pending={out.get('total_pending', 0)} "
|
|
f"status={out.get('status')}", flush=True)
|
|
for r in out.get("results", []):
|
|
print(f" {str(r.get('case_number',''))[:42]}: {r.get('status')}", flush=True)
|
|
if processed == 0:
|
|
empty_rounds += 1
|
|
await asyncio.sleep(3)
|
|
else:
|
|
empty_rounds = 0
|
|
print(f"===DONE=== metadata extracted (cumulative cases handled={total})", flush=True)
|
|
return 0
|
|
|
|
|
|
if __name__ == "__main__":
|
|
sys.exit(asyncio.run(main()))
|