feat(health): טקסונומיית-בריאות לסוכן — zombie/stalled/working/idle (#222)
מוסיף web/agent_health.py — מסווג-טהור classify_issue_health (agent-agnostic, testable) שממפה כל issue פתוח+משויך-סוכן למצב-בריאות אחד, ו-get_agent_health ב-paperclip_client שגוזר את הפרימיטיבים מ-heartbeat_runs (run חי=finished_at null בחלון) + agent_wakeup_requests (total + recovery-markers בחלון). מרכיב על אירועי #219 (אותם reasons של recovery). - zombie = פתוח+משויך-סוכן, אין run חי, יש wakeup-recovery → הstranded-child שאובחן ידנית (reference_recovery_loop_stranded_child) עולה אוטומטית. - stalled = ננער חוזר בלי progress; working = run חי; idle = המתנה בנונית. - Port: pc_get_agent_health (read-only, לא עטוף-טלמטריה לפי הכלל); endpoint GET /api/operations/agents/health (worst-first, +counts). - 7 בדיקות pytest (מסווג-טהור + fetch fake-asyncpg). אומת חי read-only מול Paperclip DB: 6 issues→idle נכון, אפס zombie. Invariants: מקיים G12 (מסווג אגנוסטי; fetch מהמעטפת; המגע מהשער), G2 (מקור-בריאות יחיד). UI ב-/operations = follow-up מגודר-שער-עיצוב (feedback_claude_design_gate). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
79
web/agent_health.py
Normal file
79
web/agent_health.py
Normal file
@@ -0,0 +1,79 @@
|
||||
"""Agent health taxonomy (TaskMaster #222) — one crisp state per stuck issue.
|
||||
|
||||
Classifies each open, agent-assigned issue into a single health state so the
|
||||
stranded-child / recovery-loop cases we diagnose by hand today
|
||||
([[reference_recovery_loop_stranded_child]], [[reference_paperclip_recovery_loops]])
|
||||
surface automatically in the dashboard — the ``gt feed --problems`` idea from
|
||||
Gastown, grounded in *our* failure modes.
|
||||
|
||||
Two layers, split for testability (same shape as web.agent_telemetry):
|
||||
- :func:`classify_issue_health` is a **pure function** over three primitives
|
||||
(live-run flag + wakeup counts). No DB, no platform symbols — trivially
|
||||
unit-tested and platform-agnostic (INV-G12).
|
||||
- The Paperclip-specific fetch that derives those primitives from
|
||||
``heartbeat_runs`` / ``agent_wakeup_requests`` lives in the shell
|
||||
(``web.paperclip_client.get_agent_health``); this module only decides *how to
|
||||
label* what the fetch measured, so the taxonomy survives a platform swap.
|
||||
|
||||
The wakeup/recovery signals it reads are the very ones #219 telemetry now emits
|
||||
(``agent.wakeup`` with recovery reasons) — this is the consumer that turns that
|
||||
stream into an at-a-glance verdict.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
# Severity order (worst first) — the dashboard sorts problems by this.
|
||||
HEALTH_STATES: tuple[str, ...] = ("zombie", "stalled", "working", "idle")
|
||||
|
||||
# Substrings of Paperclip wakeup ``reason`` values that mark a recovery-loop
|
||||
# wake (as opposed to a genuine new task). Sourced from the recovery-loop
|
||||
# forensics in memory: an issue that keeps getting these while agent-owned is a
|
||||
# zombie — assigned + "active" but nothing is really executing.
|
||||
RECOVERY_REASON_MARKERS: tuple[str, ...] = (
|
||||
"source_scoped_recovery",
|
||||
"stranded_assigned",
|
||||
"issue_continuation",
|
||||
"issue_reopened",
|
||||
"missing_disposition",
|
||||
"successful_run_missing_state",
|
||||
"waiting_on_review",
|
||||
)
|
||||
|
||||
# ≥ this many wakeups inside the fetch window, with no recovery marker and no
|
||||
# live run, reads as "poked repeatedly but not progressing" = stalled.
|
||||
STALL_WAKEUP_THRESHOLD = 3
|
||||
|
||||
|
||||
def is_recovery_reason(reason: str | None) -> bool:
|
||||
"""True if a wakeup ``reason`` is a recovery-loop marker (not a new task)."""
|
||||
if not reason:
|
||||
return False
|
||||
return any(marker in reason for marker in RECOVERY_REASON_MARKERS)
|
||||
|
||||
|
||||
def classify_issue_health(
|
||||
*,
|
||||
has_live_run: bool,
|
||||
recovery_wakeups: int,
|
||||
total_wakeups: int,
|
||||
) -> str:
|
||||
"""Label one open, agent-assigned issue. Pure — no I/O.
|
||||
|
||||
- ``working`` — a live run is executing right now.
|
||||
- ``zombie`` — no live run but recovery-loop wakeups are firing: the issue
|
||||
looks agent-owned + active yet nothing is really running (the stranded
|
||||
child we chase manually).
|
||||
- ``stalled`` — no live run, no recovery markers, but poked ``>=``
|
||||
:data:`STALL_WAKEUP_THRESHOLD` times in-window (repeated wakes, no
|
||||
progress).
|
||||
- ``idle`` — open + assigned but quiet: benign waiting.
|
||||
|
||||
Called only for issues that are already open and agent-assigned; unassigned
|
||||
or closed issues are not health-tracked (the fetch filters them out).
|
||||
"""
|
||||
if has_live_run:
|
||||
return "working"
|
||||
if recovery_wakeups >= 1:
|
||||
return "zombie"
|
||||
if total_wakeups >= STALL_WAKEUP_THRESHOLD:
|
||||
return "stalled"
|
||||
return "idle"
|
||||
Reference in New Issue
Block a user