Skip to content

Serialize all org publishers through one ticketed writer lock (Refs #4753) - #4858

Merged
EHotwagner merged 5 commits into
mainfrom
routine/4753-publication-writer-lock-20261024
Oct 24, 2026
Merged

EHotwagner merged 5 commits into
mainfrom
routine/4753-publication-writer-lock-20261024

Conversation

@EHotwagner

@EHotwagner EHotwagner commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Implements the root decision on #4753 (comment): one org-wide serialized publication writer, replacing the aggregate-resource profile, with the "genuine next Wizard release" prerequisite dropped. This is the .github half; producer repositories adopt the same step in their own PRs.

Mechanism (ADR-0097). GitHub concurrency groups are per repository, so no group can serialize publishers in different repositories. A producer's GITHUB_TOKEN cannot write elsewhere, the ledger App key lives in .github alone and the nuget.org Trusted Publishing policy pins each producer's own workflow, so a journal-CAS lock or a central workflow would need credentials or moves that do not exist. The lock therefore uses the public Actions job list: every publishing job has a step named Acquire org publication writer lock (composite action .github/actions/publication-writer-lock) and a display name containing [org-publication-writer] (the live Actions API often returns no step list, so participants are recognised by job-level name/status/started_at only); its ticket is the job started_at then job id; it proceeds only when no other in_progress participant in a registered repository has a smaller ticket (registry/publication-writers.json, read live from main). The oldest live job always proceeds, so there is no cycle. The job's end is the release, so a crashed holder cannot deadlock. A seven hour lease ignores ghosts, waiters fail closed after a bounded wait, and any unreadable registry, listing or own-job record never acquires. It grants no authority: candidate, journal CAS, revocation, protection and the live-main equality guard are unchanged.

Changes.

  • Replace the three per-repository publisher groups with the lock step: both Wizard writer jobs (recovery-diagnostic stays outside), the coherent publisher and its install-only suspension job, and the Telemetry Host publisher. Reseal the Wizard route by an exact inversion layer in tests/release-effect-sealing; update the Wizard writer-scope tests.
  • scripts/publication_writer_lock.py, the composite action, registry/publication-writers.json and scripts/check-publication-writer-coverage.py (--local is a required selftest, --remote reports adoption).
  • ADR-0097, the publishing skill paragraph (both mirrors, driver-manifest/skills-registry digest and changelog reconciled), and one stale sentence in telemetry-host-release.md.

Controls (tests/publication-writer-lock/run.py, credential-free; a virtual clock drives real threads through the real acquire over a fake job list). Six publishers across six repositories enter strictly one at a time in ticket order; three mutants (never blocks, blind to other repositories, every holder a ghost) each produce overlap or refusal. A holder that dies without releasing is replaced within one poll. A 7h+ ghost is ignored, a live young holder blocks and the waiter fails closed. Persistent read failure never acquires; transient failures are retried. Unregistered repository, missing gate step, missing or ambiguous job refuse. Equal tickets order by job id; non-participant jobs do not block. Nine registry defects, nine unsafe workflow placements and a 12-publisher pile-up are covered. Existing suites: new-sdd-workspace-successor 228, release-effect-sealing (new layer, 5 repinned workflow and 6 document mutants), release-successor-execution/candidate/live/recovery, telemetry-host-successor pass locally.

Independent review (round 1, fsgg-critic-normal) and response. Fixed: step-list-based participation (now name marker), single-shot HTTP reads (4-attempt backoff), lock wait eating the job clock (writer timeouts raised by exactly the declared wait, check-workflow-timeouts.py ceiling = 60 + declared wait), checker gaps (marker, gate first after checkout, credential/tag/API writes, release-saga-start now exempt with an honest reason), ticket from step clock, wait overshoot, ADR facts. Accepted and documented: the retired .104 continuation census refuses while a second dispatch waits; producers adopt in their own PRs.

Not in this PR. No publication, recovery or credential operation is run. The live-main equality guard and "hold main stable" for candidate/source equality remain (Done when clause 1). Producer adoption follows in each repository's own PR; FS.GG.Coordination is exempt in the registry because its Actions allow-list admits only GitHub-owned actions and one NuGet/login digest. The qualifying journey is the next ordinary publication by its own owner: observe it with GH_TOKEN=$(gh auth token) python3 scripts/publication_writer_lock.py observe and the waited-seconds output of its lock step.

Refs #4753 #4730.
architecture-map: unaffected

🤖 Generated with Claude Code

EHotwagner and others added 5 commits October 24, 2026 19:22
…4753 #4730)

GitHub concurrency groups are per repository, so they cannot serialize publishers in
different repositories. Add ADR-0097: every publishing job takes a bakery lock over the
public Actions job list. The oldest live publisher job proceeds, the job's end is the
release (a crashed holder cannot deadlock), a seven hour lease bounds ghosts and waiters
fail closed. Replace the three .github per-repository publisher groups with the lock step
(both Wizard writer jobs, the coherent publisher and its install-only suspension job, and
the Telemetry Host publisher), reseal the Wizard route by exact inversion, and add
credential-free serialization, crash, lease, fail-closed and coverage controls.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…iter lock guidance (Refs #4753)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…very workflow assertions (Refs #4753)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… and widen the coverage checker (Refs #4753)

Addresses the independent review: the Actions step list is often empty so participation is
decided by a job-name marker and the server-side started_at only; transient 5xx are retried;
the writer jobs keep their original working budget by raising the ceiling by exactly the
declared wait; the checker requires the marker, first-after-checkout placement and covers
credential, tag and API writes; ADR facts corrected.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…writers and close checker evasions (Refs #4753)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@EHotwagner
EHotwagner merged commit c799a9f into main Oct 24, 2026
80 of 81 checks passed
@EHotwagner
EHotwagner deleted the routine/4753-publication-writer-lock-20261024 branch October 24, 2026 18:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant