fix(operations): disabling the halacha drain now stops a running process immediately
All checks were successful
G12 Leak-Guard / leak-guard (pull_request) Successful in 6s
All checks were successful
G12 Leak-Guard / leak-guard (pull_request) Successful in 6s
The /operations "disabled" toggle only wrote drain_controls.disabled, which the drain checks at STARTUP — so a drain already mid-run kept going until the queue emptied or the night window closed. Disabling did not stop a running drain. Three layers, immediate + backstops: - web/app.py operations_drain_toggle: on disable, also stop the running process immediately via the host pm2 bridge (_ops_pm2_control). Best-effort — a bridge failure doesn't fail the toggle. - halacha_drain_supervisor.py: each tick now reads the disabled flag (added to db_snapshot) and, when set, stops the drain and never re-triggers it — regardless of burst/window. Backstop if the UI path failed (≤ one tick). - drain_halacha_queue.py: re-check is_drain_disabled at the top of every round, so a drain disabled mid-run halts at the next round boundary. Per-chunk checkpoints mean the in-flight case loses nothing. SCRIPTS.md updated for both drain and supervisor. Invariants: G1 (fix at source — the disable control honoured along every path, not just at startup); G2 (no parallel control path — same drain_controls flag). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -62,6 +62,14 @@ async def main():
|
||||
empty_rounds = 0
|
||||
rnd = 0
|
||||
while empty_rounds < 2:
|
||||
# Re-check the /operations kill-switch each round so a drain disabled
|
||||
# mid-run halts at the next round boundary (not only at startup). The UI
|
||||
# toggle + supervisor stop the process outright; this is the in-process
|
||||
# backstop. Per-chunk checkpoints mean the current case loses nothing.
|
||||
if await db.is_drain_disabled("legal-halacha-drain"):
|
||||
print(f"===STOP=== disabled via /operations mid-run — halting "
|
||||
f"({total} cases this run; resumes when re-enabled)", flush=True)
|
||||
break
|
||||
if not _in_window():
|
||||
print(f"===STOP=== drain window closed ({_WINDOW_END:02d}:00) — "
|
||||
f"{total} cases this run; rest resumes next night", flush=True)
|
||||
|
||||
Reference in New Issue
Block a user