feat(learning): FU-3 — uncertainty-sampling של תור-האישור לפי מחלוקת-הפאנל (#133)
All checks were successful
G12 Leak-Guard / leak-guard (pull_request) Successful in 6s
All checks were successful
G12 Leak-Guard / leak-guard (pull_request) Successful in 6s
תור-אישור-ההלכות הקיים (order_by_priority, #84.3) מקדם עכשיו את ההלכות שהפאנל התלבט עליהן: split קודם, אחר-כך incomplete — התוויות בעלות-הערך הגבוה ביותר ללולאת-הלמידה (הכרעת-היו"ר מפרקת אי-ודאות אמיתית ומזינה את זיקוק-ה-rubric ב-FU-4). uncertainty-sampling על סיגנל-המחלוקת האמיתי של הפאנל, לא רק confidence-החילוץ. - list_halachot: LEFT JOIN לאחרון-הסבבים (DISTINCT ON latest round_ts מ- halacha_panel_rounds) + מפתח-מיון ראשי CASE verdict split→0/incomplete→1/ else→2, לפני מפתחות #84.3 (corroboration→confidence→age). סבבים פה-אחד ופריטים-ללא-סבב נשארים בזנב עם הסדר הקיים. - panel_verdict נחשף בכל שורה (UI יכול לתייג "פיצול" + ביקורת-סדר). - שימוש חוזר בדגל order_by_priority הקיים ובטאב הקיים — בלי מסלול/דגל מקביל (G2). ה-UI כבר מבקש order_by_priority=true → אפס שינוי-UI, אין צורך בשער-עיצוב. - test_halacha_priority_panel_order.py: 3 בדיקות offline (SQL-capture) — מפתח-מחלוקת ראשי בעדיפות, FIFO ללא דליפת-CASE, panel_verdict נבחר. Invariants: INV-G10 (capture-only, לא משנה review_status) · G1/G2 · INV-IA (אותו שער/טאב). רגרסיה: 76 בדיקות עברו. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
85
mcp-server/tests/test_halacha_priority_panel_order.py
Normal file
85
mcp-server/tests/test_halacha_priority_panel_order.py
Normal file
@@ -0,0 +1,85 @@
|
||||
"""Tests for #133 / FU-3 — active uncertainty-sampling of the chair review queue.
|
||||
|
||||
When the chair queue is requested with order_by_priority, the items the 3-judge
|
||||
panel SPLIT on (and then INCOMPLETE rounds) must float to the top — those are
|
||||
the highest-value active-learning labels (the chair's call resolves a genuine
|
||||
ambiguity and feeds rubric distillation, FU-4). This reuses the existing
|
||||
order_by_priority flag (no parallel path, G2).
|
||||
|
||||
Runs fully OFFLINE: monkeypatches db.get_pool with a fake pool that captures the
|
||||
SQL passed to fetch, and asserts the ORDER BY / JOIN shape — no Postgres.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
|
||||
import pytest
|
||||
|
||||
from legal_mcp.services import db
|
||||
|
||||
|
||||
class _FakePool:
|
||||
"""Captures SQL passed to ``fetch``; returns no rows."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.queries: list[str] = []
|
||||
|
||||
async def fetch(self, sql: str, *args): # noqa: ANN002, ANN201
|
||||
self.queries.append(sql)
|
||||
return []
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def fake_pool(monkeypatch: pytest.MonkeyPatch) -> _FakePool:
|
||||
pool = _FakePool()
|
||||
|
||||
async def _get_pool() -> _FakePool:
|
||||
return pool
|
||||
|
||||
monkeypatch.setattr(db, "get_pool", _get_pool)
|
||||
return pool
|
||||
|
||||
|
||||
def _list_sql(pool: _FakePool) -> str:
|
||||
return next(q for q in pool.queries if "FROM halachot h" in q)
|
||||
|
||||
|
||||
def test_priority_order_ranks_panel_split_first(fake_pool: _FakePool) -> None:
|
||||
asyncio.run(
|
||||
db.list_halachot(review_status="pending_review", order_by_priority=True)
|
||||
)
|
||||
sql = _list_sql(fake_pool)
|
||||
# latest-verdict join is present …
|
||||
assert "FROM halacha_panel_rounds" in sql
|
||||
assert "DISTINCT ON (halacha_id)" in sql
|
||||
# … and the ORDER BY ranks split before incomplete before everything else,
|
||||
# AHEAD of the #84.3 corroboration/confidence/age keys.
|
||||
order = sql[sql.index("ORDER BY"):]
|
||||
assert "WHEN 'split' THEN 0" in order
|
||||
assert "WHEN 'incomplete' THEN 1" in order
|
||||
rank_pos = order.index("CASE pr.verdict")
|
||||
corr_pos = order.index("corroboration_negative")
|
||||
conf_pos = order.index("h.confidence")
|
||||
assert rank_pos < corr_pos < conf_pos, (
|
||||
"panel-disagreement rank must be the PRIMARY sort key, before the "
|
||||
"existing #84.3 corroboration/confidence ordering"
|
||||
)
|
||||
|
||||
|
||||
def test_fifo_order_has_no_panel_rank(fake_pool: _FakePool) -> None:
|
||||
"""Without order_by_priority the queue stays in deterministic FIFO order —
|
||||
the panel-rank CASE must not leak into the default ordering."""
|
||||
asyncio.run(db.list_halachot(review_status="pending_review"))
|
||||
sql = _list_sql(fake_pool)
|
||||
order = sql[sql.index("ORDER BY"):]
|
||||
assert "CASE pr.verdict" not in order
|
||||
assert "h.case_law_id, h.halacha_index" in order
|
||||
|
||||
|
||||
def test_panel_verdict_selected(fake_pool: _FakePool) -> None:
|
||||
"""panel_verdict is surfaced on each row so the UI can badge *why* an item
|
||||
is at the top of the queue (and so the order is auditable)."""
|
||||
asyncio.run(db.list_halachot(order_by_priority=True))
|
||||
sql = _list_sql(fake_pool)
|
||||
assert "pr.verdict AS panel_verdict" in sql
|
||||
Reference in New Issue
Block a user