fix(extraction): סינון cited_only מתור/מוני החילוץ (#140)
All checks were successful
G12 Leak-Guard / leak-guard (pull_request) Successful in 7s
Lint — undefined names / undefined-names (pull_request) Successful in 14s

31 שורות case_law עם source_kind='cited_only' (ציטוט-בלבד, ללא full_text/chunks)
נושאות halacha_extraction_status='pending' רק כברירת-מחדל ומזהמות את מונה ה-pending
ובמתזמר/בדף-התפעול — אין להן מה לחלץ.

תיקון (G1 — תיקון-במקור, G2 — מסנן יחיד משותף):
- db.EXTRACTION_ELIGIBLE_PREDICATE — מקור-אמת יחיד ל"שורה ברת-חילוץ" (source_kind
  <> 'cited_only' AND יש precedent_chunks). מוחל ב-list_pending_extraction_requests;
  #139 יעשה בו שימוש-חוזר ל-reconcile (אותו כלל, לא כפול).
- מוני-snapshot מסננים cited_only: halacha_drain_supervisor.db_snapshot,
  web/app.py meta+hal_ext (GROUP BY status).
- reconcile_metadata_status.py מורחב לכסות גם את תור-ההלכות: cited_only→'skipped'
  (אותו terminal-state כמו צד-המטא, תור-תאום, G2). בוצע על ה-DB החי: 31 הועברו
  ל-'skipped' (metadata כבר היה מיושב — אידמפוטנטי). התפלגות-אחרי: halacha
  pending=9 (עבודה אמיתית), skipped=31, completed=309.

בדיקות: test_extraction_queue_eligibility (predicate + list_pending מחיל אותו,
שני ה-kinds). כל 345 בדיקות mcp עוברות. guards נקיים.

Invariants: G1 (terminal-state אמיתי במקור), G2 (predicate יחיד, ללא תור מקביל),
INV-DM1 (stub לא-searchable אינו מועמד-חילוץ), G12 (leak-guard נקי).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-15 04:03:21 +00:00
parent 7043de0ac2
commit c348903e4b
6 changed files with 114 additions and 9 deletions

View File

@@ -1,4 +1,4 @@
"""Reconcile stale ``metadata_extraction_status='pending'`` rows (G1).
"""Reconcile stale ``*_extraction_status='pending'`` rows (G1).
The column defaults to 'pending', but only ``source_kind='external_upload'``
rows with extractable text genuinely need the Gemini metadata drain. Internal
@@ -13,6 +13,10 @@ This settles each row to a truthful terminal state at the source:
- external_upload w/ text but missing name/summary → stamp requested_at (real work → drain picks it up)
- cited_only (no text) → 'skipped' (terminal; nothing to extract)
Covers BOTH extraction queues (#140): the cited_only→'skipped' settle is applied
to ``halacha_extraction_status`` as well as ``metadata_extraction_status`` — same
phantom-backlog fix on the twin queue, one reconcile script (G2).
Idempotent and re-runnable (a healthy DB reports all-zero). The companion source
fix lives in db.create_internal_committee_decision (inserts 'completed' directly)
so internal rows never re-enter this state.
@@ -57,6 +61,15 @@ async def main() -> int:
"WHERE source_kind = 'cited_only' "
"AND metadata_extraction_status = 'pending'"
)
# Halacha side (#140): cited_only stubs inherit DEFAULT 'pending' but have no
# text/chunks to extract holdings from — settle them to the same terminal
# 'skipped' the metadata side uses, so they stop inflating the halacha
# pending counter / supervisor snapshot. Same source-fix, one reconcile (G2).
cited_hal = await pool.execute(
"UPDATE case_law SET halacha_extraction_status = 'skipped' "
"WHERE source_kind = 'cited_only' "
"AND halacha_extraction_status = 'pending'"
)
def n(tag: str) -> str:
try:
@@ -64,10 +77,11 @@ async def main() -> int:
except (AttributeError, IndexError):
return "?"
print(f"internal_committee → completed : {n(internal)}")
print(f"external_upload → completed : {n(external_done)}")
print(f"external_upload → requeued : {n(external_requeued)}")
print(f"cited_only → skipped : {n(cited)}")
print(f"internal_committee → completed : {n(internal)}")
print(f"external_upload → completed : {n(external_done)}")
print(f"external_upload → requeued : {n(external_requeued)}")
print(f"cited_only metadata → skipped : {n(cited)}")
print(f"cited_only halacha → skipped : {n(cited_hal)}")
rows = await pool.fetch(
"SELECT coalesce(metadata_extraction_status,'NULL') s, count(*) n "
@@ -76,6 +90,13 @@ async def main() -> int:
print("\nresulting metadata_extraction_status distribution:")
for r in rows:
print(f" {r['s']:<12} {r['n']}")
hal_rows = await pool.fetch(
"SELECT coalesce(halacha_extraction_status,'NULL') s, count(*) n "
"FROM case_law GROUP BY 1 ORDER BY 2 DESC"
)
print("\nresulting halacha_extraction_status distribution:")
for r in hal_rows:
print(f" {r['s']:<12} {r['n']}")
return 0