Skip to content

Fix host-trust supervision during provider hangs - #33

Merged
joecklau merged 2 commits into
developfrom
fix/host-trust-supervisor-isolation
Aug 13, 2026
Merged

Fix host-trust supervision during provider hangs#33
joecklau merged 2 commits into
developfrom
fix/host-trust-supervisor-isolation

Conversation

@joecklau

@joecklau joecklau commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Summary

Fix a pool-supervision failure where a hung Docker Sandboxes inventory or health operation could block the shared supervisor, starve 30-second host-trust lease renewal, and leave a runner GitHub-assignable after its 90-second guest lease expired.

  • Bound steady-state host-trust, liveness, inventory, diagnostics, reconciliation, over-capacity, and replacement work so one stuck provider operation cannot indefinitely starve lease maintenance.
  • Quarantine affected physical capacity and fence only the exact GitHub runner registration when host-trust transport or lease maintenance fails.
  • Hydrate and validate immutable runner IDs during restart reconciliation, rejecting zero-ID adoption and same-name identity replacement.
  • Extend the initial lease keeper with the same fail-closed quarantine and exact-registration fencing behavior.
  • Add regression coverage for lease failures, exact-ID fencing, restart identity recovery, and mismatched runner identities.
  • Include two incidental hygiene changes: generalize the agent scratch-path example and use a neutral example hostname in a Docker Sandboxes policy fixture.

Validation

  • go test ./internal/pool
  • go test -race ./internal/pool
  • go test ./... (one existing timing-sensitive Docker Sandboxes keepalive test failed only under full-suite load; isolated rerun passed 5/5)
  • Independent final verification found no P0-P2 blockers.
  • Live Docker Sandboxes canary: https://github.com/solutionforest/ephemeral-action-runner-test/actions/runs/31688472557 (passed in 19 seconds)
  • Live two-runner pool remained responsive beyond a full lease lifetime, with fresh lease expirations read back from both guests.

Checklist

  • I kept credentials, private keys, tokens, and machine-specific configuration out of this pull request.
  • I added or updated tests where behavior changed.
  • I updated relevant documentation. No operator-facing configuration or command changed; the operational behavior is covered by code comments and regression tests.
  • For provider or onboarding changes, I followed the Development and Extension Principles and documented and tested every intentional exception.
  • I read and followed the contributing guide and code of conduct.

Bound steady-state provider operations so hung inventory and health checks cannot starve host-trust lease renewal. Fence exact GitHub runner registrations when trust maintenance fails, preserve uncertain local capacity in quarantine, and validate immutable runner identities during restart reconciliation.
@joecklau
joecklau deployed to epar-live-ci August 13, 2026 10:45 — with GitHub Actions Active
@joecklau
joecklau deployed to epar-live-ci August 13, 2026 19:10 — with GitHub Actions Active
@joecklau
joecklau marked this pull request as ready for review August 13, 2026 19:27
@joecklau
joecklau merged commit b262286 into develop Aug 13, 2026
12 checks passed
@joecklau
joecklau deleted the fix/host-trust-supervisor-isolation branch August 13, 2026 19:28
@joecklau joecklau mentioned this pull request Aug 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants