Skip to content

feat(app): add the session supervisor - #282

Merged
kfox merged 1 commit into
mainfrom
session-manager
Aug 14, 2026
Merged

feat(app): add the session supervisor#282
kfox merged 1 commit into
mainfrom
session-manager

Conversation

@kfox

@kfox kfox commented Aug 14, 2026

Copy link
Copy Markdown
Owner

Plan step 3 of the web console: the state machine a long-lived host runs. No HTTP — that is the next step.

SessionManager owns at most one Session at a time and serialises every transition behind one lock:

                 start(req)                    build ok
   idle ──────────────────▶ starting ────────────────────▶ running
     ▲                          │                              │
     │                    build raised              stop() │ threads exited
     │                          ▼                              ▼
     └──── teardown done ─── stopping ◀──────────────────────────
                                │
                  teardown done + build had failed
                                ▼
                              error ──start(req)──▶ starting

error is not sticky — it is where a failed build parks its diagnostic, and a machine that was unreachable a minute ago usually isn't now.

Design points worth the review

Everything slow runs off the caller's thread. build_session blocks for many seconds and teardown is not much cheaper, so start()/stop() return as soon as the transition is claimed. What deliberately does not run off-thread is validation: validate_configs is hardware-free, so a bad config can be refused synchronously. That is what the split in #279 was for.

The build seam is a publish callback, not just a return value. A build that fails after the stacks are up has already taken the hardware. Handing the session over the moment it exists keeps "no hardware is left held" on the single path out of a generation, rather than splitting it between the supervisor and every build function — which is why a failed build routes through stopping on its way to error.

start() never implicitly stops. Replacing a running show is switch(), so the one path that has to get stop → settle → start right is the one path that does it; start() while running raises SupervisorBusy (a 409 later). wait_for(state, generation=…) exists because "wait until running" is ambiguous across a switch — the show being replaced is running too.

Plus the parts that only matter once a session outlives the command that started it: a settle window (the U64's DMA service refuses new connections for a few seconds after one closes, and AVFoundation refuses to reopen a camera straight after release() — two facts, one timer, re-armed after a marker recovery since that opens a backend of its own), a reap poller for shows that end by themselves, a run marker that triggers a safe-state reset if the last run died mid-show, and a bounded generation-tagged log tail so a failure to start is readable somewhere other than the terminal.

Supervisor threads are daemon=False, unlike every other background thread here, for the same reason the playlist threads are: these are the threads that tear the hardware down.

Verification

make check green (ruff, mypy --strictserve.py joins the strict list — pyright, 3895 tests), make site-check OK. 29 new unit tests cover the transitions, the failed-build path, the cooldown, the reaper, switch, the run marker, safe-state recovery and the log buffer, all against injected fakes with no sleeping.

Hardware verification against the real U64 — 22 assertions over five generations, driving the real build_and_start/teardown, results in the PR thread.

Not in this PR

--serve, the [web] config section and the /api/* routes are the next step.

The one-shot CLI's process and its session have the same lifetime, so
nothing ever had to answer "which session is this?". A long-lived host
does: SessionManager owns at most one Session at a time and serialises
every transition behind one lock — idle -> starting -> running ->
stopping -> idle, with error as where a failed build parks its
diagnostic.

Build and teardown are injected, so the state machine is testable
against fakes with no hardware, no sockets and no sleeping. The build
seam is a publish callback rather than a bare return value: a build that
fails after the stacks are up has already taken the hardware, and
handing the session over the moment it exists keeps "no hardware is left
held" on the single path out of a generation instead of splitting it
across every build function. That is why a failed build routes through
stopping on its way to error.

Also here, because they only matter once a session outlives the command
that started it:

- A settle window between teardown and the next start. Two hardware
  facts, one timer: the U64's DMA service refuses new connections for a
  few seconds after one closes, and AVFoundation refuses to reopen a
  camera straight after release(). Re-armed after a marker recovery,
  which opens and closes a backend of its own.
- A reap poller. A non-looping show ends by itself; the CLI notices
  because it is parked in a join, a daemon has nothing watching.
- A run marker under the data dir, so a start that finds one knows the
  last run died mid-show and resets the machine first.
- A bounded, generation-tagged log tail, so a failure to start can be
  read somewhere other than the terminal.

start() never implicitly stops — replacing a show is switch(), so the
one path that has to get stop -> settle -> start right is the one path
that does it.

Nothing runs it yet: no flag, no config key, no endpoint.
@kfox

kfox commented Aug 14, 2026

Copy link
Copy Markdown
Owner Author

Hardware verification — 22 assertions over five generations against the real machine, driving the actual build_and_start / teardown (not the fakes the unit tests use). This is the part the suite explicitly can't reach: a backend reopened inside the settle window, teardown really releasing the DMA socket, the reaper noticing a non-looping show end on its own, and safe-state recovery from a leftover marker.

PASS  start() returned generation 1
PASS  reached running
PASS  run marker written while running
PASS  still running mid-show
PASS  reaped to idle when the show ended
PASS  run marker cleared on the way down
PASS  no error recorded
PASS  cooldown armed after teardown (3.0s)
PASS  start() returned generation 2
PASS  generation 2 running
PASS  start waited out the settle window
PASS  stop() accepted
PASS  stopped to idle
PASS  generation 3 running
PASS  safe-state recovery ran before the build
PASS  final stop to idle
PASS  unreachable start reached error
PASS  last_error set: 'StackBuildError: stack build failed (exit code 4)'
PASS  no marker left behind by a failed start
PASS  error is not sticky — start accepted
PASS  generation 5 running
PASS  stopped to idle

Transition log, one process, five generations in 41 seconds:

    0.00s  gen 1  starting
    3.14s  gen 1  running
   15.82s  gen 1  stopping      <- 12s show ended by itself; the reaper caught it
   15.84s  gen 1  idle
   15.84s  gen 2  starting
   21.98s  gen 2  running       <- sat out the 3s settle window first
   21.98s  gen 2  stopping
   22.12s  gen 2  idle
   22.12s  gen 3  starting
   31.28s  gen 3  running       <- leftover marker: safe-state reset, then settle, then build
   31.42s  gen 3  idle
   31.42s  gen 4  starting
   34.43s  gen 4  stopping      <- unreachable machine
   34.43s  gen 4  error
   34.43s  gen 5  starting
   40.58s  gen 5  running       <- error is not sticky
   40.70s  gen 5  idle

The DMA socket was opened and closed five times in that window with no wedge, which is the settle timer doing its job.

The marker-recovery case is the one that changed the code: safe-state opens and closes a backend of its own, and handing that straight to the build is exactly the socket-reuse hazard the cooldown exists for. _recover_if_unclean now reports whether it touched the hardware and the start re-arms the window before building — visible above as gen 3 taking 9s to come up against gen 2's 6s.

A frame captured off HDMI mid-run confirms the C64 is actually rendering under the supervisor, not just reporting running.

@codecov

codecov Bot commented Aug 14, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 88.51964% with 38 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.28%. Comparing base (8ebb527) to head (eaee3bf).
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
c64cast/app/serve.py 88.44% 31 Missing and 7 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #282      +/-   ##
==========================================
+ Coverage   82.18%   82.28%   +0.09%     
==========================================
  Files         144      145       +1     
  Lines       25235    25566     +331     
  Branches     3705     3731      +26     
==========================================
+ Hits        20739    21036     +297     
- Misses       3689     3715      +26     
- Partials      807      815       +8     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

@kfox
kfox merged commit 51616cf into main Aug 14, 2026
19 checks passed
@kfox
kfox deleted the session-manager branch August 14, 2026 00:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant