Skip to content

fix(sessions): close the delete/respawn race; deflake the webhook dispatch test - #166

Merged
defangdevs merged 1 commit into
masterfrom
fix/session-races
Aug 2, 2026
Merged

fix(sessions): close the delete/respawn race; deflake the webhook dispatch test#166
defangdevs merged 1 commit into
masterfrom
fix/session-races

Conversation

@defangdevs

Copy link
Copy Markdown
Owner

Master CI run 30740226645 (the #164 merge) failed two VM tests; both trace to session-lifecycle races.

sessions test — real module bug

Settings-page delete (delist → kill) raced start_session, which had already read the session's JSON and was mid-preparation: its tmux new-session resurrected the just-killed session as a live, delisted session — which the reconcile loop deliberately never kills (unmanaged sessions are tolerated by design), so the leak is permanent. Fix in start_session:

  • re-check the current sessions file immediately before tmux new-session;
  • verify again after the spawn and kill it if the session was delisted meanwhile (a delete landing later sees the live session and kills it itself — either interleaving now converges on dead);
  • ordered before mark_started, whose jq assignment would otherwise re-create a deleted session as a stub entry.

webhook test — flaky assertion from #164

The test read initialPrompt from sessions.json after the hook session appeared, racing the supervisor's own prompt consumption (mark_started nulls it within ~2s). Now the dispatch section stops agent-box-agent.service before the signed delivery (the spawn wrapper's file write is the contract and needs no supervisor), asserts the full prompt deterministically, then restarts the supervisor and asserts the consumption half (hasRun == true, initialPrompt == null, hook session live in tmux).

Checks run

  • nix run .#assemble committed; module-generated-up-to-date, multi-user, module-single-file, webhook-route green natively (aarch64).
  • VM tests (sessions, webhook, and the rest) run in this PR's CI.

No security, AWS, or migration impact; the delete race fix is behavior users already assumed (deleted sessions stay deleted).

Refs run 30740226645.

🤖 Generated with Claude Code

https://claude.ai/code/session_013VoCn5tqBY3hKKwmtCH1kx

…patch test

Master CI run 30740226645 failed both ways this race can bite:

1. sessions VM test: a session deleted through the settings page came
   back. The delete delists then kills, but start_session had already
   read the session's JSON and was preparing the spawn (a dozen jq
   calls, claude state seeding) — its tmux new-session then resurrected
   the just-killed session. The reconcile loop deliberately tolerates
   unmanaged sessions, so the leak is permanent and invisible to the
   supervisor. start_session now re-checks the CURRENT file immediately
   before spawning AND verifies again after: a delist landing before the
   post-check is honored by killing the fresh spawn; one landing after
   sees the session live and kills it itself. The post-check is ordered
   before mark_started, whose jq assignment would otherwise re-create a
   just-deleted session as a stub entry.

2. webhook VM test (added in #164): it read initialPrompt out of
   sessions.json after waiting for the hook-* session to appear — but
   the supervisor consumes a kickoff prompt (mark_started nulls it)
   within ~2s of spawning, so the assert raced the very consumption it
   sat next to; it passed on the PR run by timing luck and lost on
   master. The dispatch section now STOPS agent-box-agent.service before
   the signed delivery (the spawn wrapper writes sessions.json either
   way — that is the point of the file-based contract), asserts the full
   prompt deterministically, then restarts the supervisor and asserts
   the other half: the hook session comes up in tmux, hasRun flips true,
   and initialPrompt is consumed to null.

Verification: nix run .#assemble committed; module-generated-up-to-date,
multi-user, module-single-file, webhook-route pass natively on
aarch64-linux. The two VM tests run in this PR's CI on x86 — the
webhook test now exercises spawn-while-supervisor-down plus
consume-on-start, and the sessions test keeps stochastic coverage of
the delete race with the window shrunk from ~1s to the jq→tmux gap and
backstopped by the post-spawn kill.

Refs run 30740226645.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013VoCn5tqBY3hKKwmtCH1kx
@defangdevs
defangdevs merged commit 34836b3 into master Aug 2, 2026
1 check passed
@defangdevs
defangdevs deleted the fix/session-races branch August 2, 2026 09:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants