Chaim
bfc352c7b5
מוסיף web/agent_health.py — מסווג-טהור classify_issue_health (agent-agnostic, testable) שממפה כל issue פתוח+משויך-סוכן למצב-בריאות אחד, ו-get_agent_health ב-paperclip_client שגוזר את הפרימיטיבים מ-heartbeat_runs (run חי=finished_at null בחלון) + agent_wakeup_requests (total + recovery-markers בחלון). מרכיב על אירועי #219 (אותם reasons של recovery). - zombie = פתוח+משויך-סוכן, אין run חי, יש wakeup-recovery → הstranded-child שאובחן ידנית (reference_recovery_loop_stranded_child) עולה אוטומטית. - stalled = ננער חוזר בלי progress; working = run חי; idle = המתנה בנונית. - Port: pc_get_agent_health (read-only, לא עטוף-טלמטריה לפי הכלל); endpoint GET /api/operations/agents/health (worst-first, +counts). - 7 בדיקות pytest (מסווג-טהור + fetch fake-asyncpg). אומת חי read-only מול Paperclip DB: 6 issues→idle נכון, אפס zombie. Invariants: מקיים G12 (מסווג אגנוסטי; fetch מהמעטפת; המגע מהשער), G2 (מקור-בריאות יחיד). UI ב-/operations = follow-up מגודר-שער-עיצוב (feedback_claude_design_gate). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
80 lines
3.3 KiB
Python
80 lines
3.3 KiB
Python
"""Agent health taxonomy (TaskMaster #222) — one crisp state per stuck issue.
|
|
|
|
Classifies each open, agent-assigned issue into a single health state so the
|
|
stranded-child / recovery-loop cases we diagnose by hand today
|
|
([[reference_recovery_loop_stranded_child]], [[reference_paperclip_recovery_loops]])
|
|
surface automatically in the dashboard — the ``gt feed --problems`` idea from
|
|
Gastown, grounded in *our* failure modes.
|
|
|
|
Two layers, split for testability (same shape as web.agent_telemetry):
|
|
- :func:`classify_issue_health` is a **pure function** over three primitives
|
|
(live-run flag + wakeup counts). No DB, no platform symbols — trivially
|
|
unit-tested and platform-agnostic (INV-G12).
|
|
- The Paperclip-specific fetch that derives those primitives from
|
|
``heartbeat_runs`` / ``agent_wakeup_requests`` lives in the shell
|
|
(``web.paperclip_client.get_agent_health``); this module only decides *how to
|
|
label* what the fetch measured, so the taxonomy survives a platform swap.
|
|
|
|
The wakeup/recovery signals it reads are the very ones #219 telemetry now emits
|
|
(``agent.wakeup`` with recovery reasons) — this is the consumer that turns that
|
|
stream into an at-a-glance verdict.
|
|
"""
|
|
from __future__ import annotations
|
|
|
|
# Severity order (worst first) — the dashboard sorts problems by this.
|
|
HEALTH_STATES: tuple[str, ...] = ("zombie", "stalled", "working", "idle")
|
|
|
|
# Substrings of Paperclip wakeup ``reason`` values that mark a recovery-loop
|
|
# wake (as opposed to a genuine new task). Sourced from the recovery-loop
|
|
# forensics in memory: an issue that keeps getting these while agent-owned is a
|
|
# zombie — assigned + "active" but nothing is really executing.
|
|
RECOVERY_REASON_MARKERS: tuple[str, ...] = (
|
|
"source_scoped_recovery",
|
|
"stranded_assigned",
|
|
"issue_continuation",
|
|
"issue_reopened",
|
|
"missing_disposition",
|
|
"successful_run_missing_state",
|
|
"waiting_on_review",
|
|
)
|
|
|
|
# ≥ this many wakeups inside the fetch window, with no recovery marker and no
|
|
# live run, reads as "poked repeatedly but not progressing" = stalled.
|
|
STALL_WAKEUP_THRESHOLD = 3
|
|
|
|
|
|
def is_recovery_reason(reason: str | None) -> bool:
|
|
"""True if a wakeup ``reason`` is a recovery-loop marker (not a new task)."""
|
|
if not reason:
|
|
return False
|
|
return any(marker in reason for marker in RECOVERY_REASON_MARKERS)
|
|
|
|
|
|
def classify_issue_health(
|
|
*,
|
|
has_live_run: bool,
|
|
recovery_wakeups: int,
|
|
total_wakeups: int,
|
|
) -> str:
|
|
"""Label one open, agent-assigned issue. Pure — no I/O.
|
|
|
|
- ``working`` — a live run is executing right now.
|
|
- ``zombie`` — no live run but recovery-loop wakeups are firing: the issue
|
|
looks agent-owned + active yet nothing is really running (the stranded
|
|
child we chase manually).
|
|
- ``stalled`` — no live run, no recovery markers, but poked ``>=``
|
|
:data:`STALL_WAKEUP_THRESHOLD` times in-window (repeated wakes, no
|
|
progress).
|
|
- ``idle`` — open + assigned but quiet: benign waiting.
|
|
|
|
Called only for issues that are already open and agent-assigned; unassigned
|
|
or closed issues are not health-tracked (the fetch filters them out).
|
|
"""
|
|
if has_live_run:
|
|
return "working"
|
|
if recovery_wakeups >= 1:
|
|
return "zombie"
|
|
if total_wakeups >= STALL_WAKEUP_THRESHOLD:
|
|
return "stalled"
|
|
return "idle"
|