docs: add note on pipeline number priority vs. lock arrival order in serial groups - #10624
hidetaka-cci wants to merge 3 commits into
Conversation
|
@hidetaka-cci I think the commits from you server docs branch are in this branch too. Could you separate them? Then we can get a review from the pipelines team for the serial groups note |
…serial groups The existing documentation stated the priority rule but did not address a common source of confusion in dynamic config setups: when setup workflow duration differs between pipelines, lock arrival order and pipeline number order can diverge. This note makes the distinction explicit and warns users not to rely on merge order as a guarantee. Co-authored-by: Cursor <cursoragent@cursor.com>
8d29bfa to
14a7053
Compare
|
@rosieyohannan Two things still show in CI but look unrelated to this change: a pre-existing Vale error elsewhere in the file, and a flaky build job (Antora choking on an unrelated branch, v3-api-docs-link) — main's recent builds are all green. Want me to fix the pre-existing Vale error here too, or leave it for later? Otherwise this should be ready to review. |
| ==== | ||
| Serial group priority is based on pipeline number, not on when a job reaches the lock. When a job attempts to acquire a serial group lock, CircleCI checks for jobs from higher-numbered pipelines in the same serial group. If one is already queued or running, CircleCI skips the job. | ||
|
|
||
| With dynamic configuration, setup workflow duration varies between pipelines, so the order in which continuation jobs reach the lock is not guaranteed to match pipeline number (that is, merge) order. A job from an older pipeline may therefore be skipped rather than queued. Do not rely on merge order to guarantee that every pipeline's serially grouped job runs. |
There was a problem hiding this comment.
I don't think it's guaranteed that hooks don't reach us in order either.
There was a problem hiding this comment.
@stig Thanks for flagging this — I want to make sure I understand the concern correctly before revising.
Is the point that pipeline number order isn't a perfectly reliable proxy for merge order to begin with (e.g. webhook delivery to CircleCI isn't guaranteed to be strictly ordered), independent of the dynamic config timing issue I described? If so, I agree the note as written overstates it by attributing the mismatch solely to setup workflow duration.
If that's the concern, would it work to soften the note to say something like "pipeline number order may not always align with merge order, for reasons including but not limited to variable setup workflow duration under dynamic config"? Let me know if I'm still missing something.
Thanks
There was a problem hiding this comment.
web hooks arriving (or being processed) out of order either from the provider, or because of retries on our end, before they arrive at builds-service (which allocates the pipeline number) is my concern. Dynamic config is not relevant: continuations use the same pipeline number as the setup workflow.
3 participants
Summary
Motivation
The existing documentation stated the priority rule but did not address a common source of confusion in dynamic config setups: when setup workflow duration differs between pipelines, lock arrival order and pipeline number order can diverge. This note makes the distinction explicit and warns users not to rely on merge order as a guarantee.
Changes
docs/guides/modules/orchestrate/pages/controlling-serial-execution-across-your-organization.adoc: added NOTE block under "Pipeline number priority" sectionMade with Cursor