feat(halacha): rhetorical-role pre-filter — fallback excludes facts/arguments (#81.6)
All checks were successful
G12 Leak-Guard / leak-guard (pull_request) Successful in 5s
All checks were successful
G12 Leak-Guard / leak-guard (pull_request) Successful in 5s
חילוץ-הלכות מוגבל למקטעי הנמקה/הכרעה בלבד (INV-LRN2 quality-at-source). הפער שנסגר: מסלול ה-fallback (כשה-chunker לא תייג שום מקטע כ-extractable, כותרות לא-תקניות → הכול 'other') נפל קודם ל**כל** ה-chunks — והחזיר בדיוק את המקטעים שהמסנן הראשי מחריג (רקע עובדתי + טענות הצדדים). בלבול Facts↔Reasoning הוא מחלקת-השגיאה הדומיננטית (LegalSeg), כך שהזנת עובדות לחילוץ פוגעת ישירות ב-precision. - NON_REASONING_SECTIONS = (facts, appellant_claims, respondent_claims, intro) - _select_extractable_chunks(): מרכז את מדיניות-הבחירה (primary + fallback) בפונקציה אחת המשמשת גם את הבחירה הראשית וגם את ה-re-read לקביעת-סטטוס (G2 — מקור-אמת יחיד, אין מסלול מקביל). ה-fallback מחריג את NON_REASONING_SECTIONS ועדיין מגיע להנמקה שנחתה תחת 'other'. invariants: G1 (נרמול-במקור, לא תיקון-בקריאה) · G2 (אין מסלול מקביל) · INV-LRN2 (quality-at-source). tests: 4 חדשות (primary/fallback-excludes-args/all-nonreasoning/disjoint-sets) + 61 בדיקות-הלכה קיימות עוברות. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -64,6 +64,15 @@ EXTRACTION_FAILURE_THRESHOLD = 0.5
|
||||
# never contain holdings, only positions, so we skip them.
|
||||
EXTRACTABLE_SECTIONS = ("legal_analysis", "ruling", "conclusion")
|
||||
|
||||
# Sections confidently classified as NON-reasoning (parties' positions, the
|
||||
# factual background, the opening). The fallback path — taken when the chunker
|
||||
# labeled nothing as an extractable section (non-standard headings → 'other') —
|
||||
# excludes these so facts/arguments are NEVER fed into extraction, while
|
||||
# reasoning that merely landed under 'other' is still reached. Raises precision
|
||||
# on the dominant Facts↔Reasoning confusion class (#81.6; INV-LRN2
|
||||
# quality-at-source; LegalSeg / rhetorical-role labeling).
|
||||
NON_REASONING_SECTIONS = ("facts", "appellant_claims", "respondent_claims", "intro")
|
||||
|
||||
|
||||
# Two prompts — choose by source's is_binding flag.
|
||||
#
|
||||
@@ -498,6 +507,39 @@ async def extract(case_law_id: UUID | str, force: bool = False,
|
||||
await pool.release(lock_conn)
|
||||
|
||||
|
||||
async def _select_extractable_chunks(
|
||||
case_law_id: UUID,
|
||||
) -> tuple[list[dict], bool]:
|
||||
"""Pick the chunks that are candidates for halacha extraction (#81.6).
|
||||
|
||||
Rhetorical-role pre-filter (INV-LRN2 quality-at-source): only
|
||||
reasoning/decision sections feed extraction.
|
||||
|
||||
Primary: chunks labeled as an extractable section
|
||||
(``EXTRACTABLE_SECTIONS``). Fallback — taken only when the chunker labeled
|
||||
*nothing* extractable (non-standard headings collapse everything to
|
||||
'other') — is every chunk EXCEPT those confidently classified as
|
||||
non-reasoning (``NON_REASONING_SECTIONS``: facts / parties' arguments /
|
||||
intro). This preserves recall for reasoning that landed under 'other' while
|
||||
never feeding the factual background or the parties' positions into
|
||||
extraction. Previously the fallback took *all* chunks, re-admitting exactly
|
||||
the sections the primary filter excludes.
|
||||
|
||||
Returns ``(chunks, used_fallback)`` so the caller can log the fallback once.
|
||||
"""
|
||||
chunks = await db.list_precedent_chunks(
|
||||
case_law_id, section_types=EXTRACTABLE_SECTIONS,
|
||||
)
|
||||
if chunks:
|
||||
return chunks, False
|
||||
all_chunks = await db.list_precedent_chunks(case_law_id)
|
||||
filtered = [
|
||||
c for c in all_chunks
|
||||
if c.get("section_type") not in NON_REASONING_SECTIONS
|
||||
]
|
||||
return filtered, True
|
||||
|
||||
|
||||
async def _extract_impl(case_law_id: UUID, force: bool = False,
|
||||
effort: str | None = None) -> dict:
|
||||
"""Core extraction (caller holds the global advisory lock for the duration).
|
||||
@@ -514,22 +556,16 @@ async def _extract_impl(case_law_id: UUID, force: bool = False,
|
||||
|
||||
is_binding = bool(record.get("is_binding"))
|
||||
|
||||
# Try the targeted sections first (legal_analysis / ruling / conclusion).
|
||||
# If the chunker labeled everything as 'other' (common when a ruling
|
||||
# uses non-standard headings or the section markers aren't bracketed
|
||||
# cleanly), fall back to ALL chunks — better to over-include than to
|
||||
# silently skip a ruling that has reasoning under an unexpected label.
|
||||
chunks = await db.list_precedent_chunks(
|
||||
case_law_id, section_types=EXTRACTABLE_SECTIONS,
|
||||
)
|
||||
if not chunks:
|
||||
chunks = await db.list_precedent_chunks(case_law_id)
|
||||
if chunks:
|
||||
logger.info(
|
||||
"halacha_extractor: case_law=%s — no targeted sections, "
|
||||
"falling back to all %d chunks",
|
||||
case_law_id, len(chunks),
|
||||
)
|
||||
# Rhetorical-role pre-filter (#81.6, INV-LRN2): only reasoning/decision
|
||||
# sections are candidates. The fallback (no targeted section labeled)
|
||||
# still excludes facts/arguments/intro — see _select_extractable_chunks.
|
||||
chunks, used_fallback = await _select_extractable_chunks(case_law_id)
|
||||
if used_fallback and chunks:
|
||||
logger.info(
|
||||
"halacha_extractor: case_law=%s — no targeted sections, "
|
||||
"falling back to %d non-argument chunks (facts/arguments excluded)",
|
||||
case_law_id, len(chunks),
|
||||
)
|
||||
if not chunks:
|
||||
await db.set_case_law_halacha_status(case_law_id, "completed")
|
||||
return {"status": "no_chunks", "extracted": 0, "stored": 0}
|
||||
@@ -655,10 +691,10 @@ async def _extract_impl(case_law_id: UUID, force: bool = False,
|
||||
|
||||
await asyncio.gather(*[_process(c) for c in pending])
|
||||
|
||||
# Decide final status from what's LEFT (re-read checkpoints).
|
||||
after = await db.list_precedent_chunks(case_law_id, section_types=EXTRACTABLE_SECTIONS)
|
||||
if not after:
|
||||
after = await db.list_precedent_chunks(case_law_id)
|
||||
# Decide final status from what's LEFT (re-read checkpoints). Use the same
|
||||
# candidate-selection policy as above so the pending count matches the set
|
||||
# we actually extracted from (G2 — single source of truth, no parallel path).
|
||||
after, _ = await _select_extractable_chunks(case_law_id)
|
||||
still_pending = sum(1 for c in after if c.get("halacha_extracted_at") is None)
|
||||
total = len(await db.list_halachot(case_law_id=case_law_id, limit=10_000))
|
||||
|
||||
|
||||
Reference in New Issue
Block a user