Skip to content

fix(ci): shard s390x spread tests - #406

Open
lczyk wants to merge 4 commits into
mainfrom
feat/shard-s390x-spread-tests
Open

lczyk wants to merge 4 commits into
mainfrom
feat/shard-s390x-spread-tests

Conversation

@lczyk

@lczyk lczyk commented Sep 29, 2026 •

Copy link
Copy Markdown
Collaborator

attempt at addressing #404

emulated s390x lanes run past their timeout: daemon_config restarts dockerd ~10 times at ~4-6m apiece, so it alone takes 42-45m+ on s390x, on top of ~15-20m allocate + prepare and ~15-22m for the remaining tasks.

in this PR we split each s390x system into 2 shards: daemon_config on its own, everything else on the other. we've also managed to reduce the number of unsets slightly from 10 to 8.

rough s390x timings are:

system allocate + prepare other 14 tasks daemon_config
24.04 10-14m ~14.5m 42m
26.10 12-16m ~16m 43m
26.04 18-19m ~22.5m killed at 45m

https://github.com/canonical/docker-snap/actions/runs/36532233527

Each snapcfg unset restarts dockerd, taking ~5m per restart on emulated s390x.
Combine unsets into single commands where they test independent options, roughly
halving the restart count and overall task time. Increase kill-timeout to 60m
to accommodate the remaining restarts within the test's margins.
daemon_config alone dominates an s390x lane; the other 14 tasks fit
one runner in ~42m worst case (26.04), so split only daemon_config
out and drop the hand-kept task list. A new task no longer needs a
shard assignment.

Shards are numbered after filtering, so a filter that leaves one lane
reads as 1/1, and only multi-shard lanes carry a shard suffix in the
job name.
@lczyk lczyk added Bug ci continuous integration labels Sep 29, 2026
@lczyk
lczyk requested a review from zhijie-yang September 29, 2026 06:49

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Bug ci continuous integration

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant