feat(learning): prospective held-out style-distance trend (Path A)
A clean held-out test of voice learning was no longer runnable: every final-uploaded case already has its lessons folded, and lessons are stored universal/untagged so leave-one-out is impossible. Path A makes the test prospective instead — capture the generalization datapoint at the one moment it's clean. On final upload, after the draft↔final pair is created but BEFORE this case's lessons are folded (folding is a separate manual /training step), we snapshot style_distance (anti_pattern_total, golden-ratio max-deviation, change_percent) alongside the current voice-lesson pool size. Because the draft was written with only the PRIOR pool, each row is a clean "with N accumulated lessons, our draft on this unseen case scored X" datapoint. As the pool grows over cases, a downward trend = learning generalizes. - db: SCHEMA_V45 style_distance_history (append-only) + helpers voice_lesson_pool_sizes / record_style_distance_snapshot / get_style_distance_history. - app: best-effort capture in api_upload_final_decision (never fails the upload); GET /api/learning/style-distance-history for the trend. Reuses the existing style_distance service + appeal_type_rules pool — no parallel metric path. The 8 existing cases are already folded, so the table starts empty and fills from the next final (their clean window is past). Invariants: G2 (reuse style_distance/appeal_type_rules — one path), INV-LRN4 (measure the draft↔final gap; this is its trend surface). LLM-free (style_distance is deterministic) so it runs in the container. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
28
web/app.py
28
web/app.py
@@ -3813,6 +3813,25 @@ async def api_upload_final_decision(case_number: str, file: UploadFile = File(..
|
||||
# checking + missing-precedent flagging (07-learning §1.3). Surfaced, not silent.
|
||||
library = await _enroll_final_in_library(case, case_number, final_text, chair_name)
|
||||
|
||||
# Path A — prospective held-out snapshot. Right now the draft (decision_blocks)
|
||||
# was written with only the PRIOR lesson pool — this case's lessons fold later,
|
||||
# manually, in /training. So measuring style-distance HERE is a clean
|
||||
# generalization datapoint. Append-only; best-effort (never fails the upload).
|
||||
try:
|
||||
from legal_mcp.services import style_distance as _sd
|
||||
sd = await _sd.style_distance(case_number)
|
||||
if isinstance(sd, dict) and "error" not in sd:
|
||||
pool = await db.voice_lesson_pool_sizes()
|
||||
summ = sd.get("summary", {})
|
||||
await db.record_style_distance_snapshot(
|
||||
case_number, pair_id,
|
||||
pool.get("discussion_rules", 0), pool.get("transition_phrases", 0),
|
||||
summ.get("anti_pattern_total"), summ.get("ratio_max_deviation_pp"),
|
||||
summ.get("change_percent"),
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning("held-out style-distance snapshot failed for %s: %s", case_number, e)
|
||||
|
||||
case_dir = config.find_case_dir(case_number)
|
||||
if case_dir.exists():
|
||||
commit_and_push(case_dir, f"החלטה סופית של היו\"ר: {final_name}")
|
||||
@@ -4738,6 +4757,15 @@ async def api_learning_style_distance(case_number: str):
|
||||
return await _sd.style_distance(case_number)
|
||||
|
||||
|
||||
@app.get("/api/learning/style-distance-history")
|
||||
async def api_learning_style_distance_history():
|
||||
"""Path A — מגמת ה-held-out הפרוספקטיבי: snapshot של מרחק-הסגנון שנלכד בכל
|
||||
העלאת-סופי *לפני* הטמעת-לקחי-התיק, מול גודל-בריכת-הלקחים באותו רגע. ירידה
|
||||
ב-anti_pattern_total / change_percent ככל שהבריכה גדלה = הלמידה מכלילה."""
|
||||
items = await db.get_style_distance_history()
|
||||
return {"items": items, "count": len(items)}
|
||||
|
||||
|
||||
def _coerce_json(raw):
|
||||
if isinstance(raw, str):
|
||||
try:
|
||||
|
||||
Reference in New Issue
Block a user