fix(worker-image): resolve the nodetool-huggingface version after the PyPI wait - #1014
Conversation
… PyPI wait Both repos publish on a tag push, so releasing nodetool-core and nodetool-huggingface together starts two wheel builds in parallel. The `meta` step read "latest nodetool-huggingface" at job start — before either wheel existed — so the image pinned the *previous* release and shipped without the fixes cut alongside it. Silently: nothing in the build or the image recorded which version went in. Move the lookup into its own step after "Wait for nodetool-core on PyPI", which already blocks long enough for the sibling wheel to land, and record both versions as OCI labels so a pulled image can be checked without a rebuild.
The wait polled /pypi/<pkg>/<version>/json while uv installs from /simple. They propagate independently: on the 0.7.10 release the JSON endpoint answered 200 and the wait passed, then the build died with "no version of nodetool-core==0.7.10" because /simple did not list it yet. Poll the surface the installer actually reads, and confirm the resolved nodetool-huggingface version is listed there too.
|
Both failure modes are now observed, not predicted. The 1. 2. So the wait now polls the simple index, and the resolved Checked against the live index, including that the check can fail and that the trailing dash anchors the version: Net effect: the 0.7.10 release produced no worker image at all. I have re-dispatched the build manually with |
… index (#1017) The worker image build resolved "latest" from /pypi/<pkg>/json. That is a different surface from /simple, with independent propagation, and it lags: minutes after 0.7.9 was listed on /simple, info.version still answered 0.7.8. The confirmation loop added in #1014 could not catch this. It verifies that the chosen version is on /simple — and 0.7.8 is. So the step passed, and the image built right after the 0.7.9 release carried the previous wheel while looking correct. Found by reading the image labels rather than trusting the green build. Discovery now reads /simple too, so both halves use the surface uv installs from. Sorting is numeric rather than lexical, which the JSON API never needed: info.version was authoritative, but max() over parsed filenames would put 0.7.9 ahead of 0.7.10. Stdlib only — no packaging import to fail in the release path. Verified against the live index: picks 0.7.9, exits non-zero on an index with no wheels, and orders 0.7.10 above 0.7.9.
The race
docker.ymlresolvesHF_VERSIONin themetastep — the first step of the job — as "latestnodetool-huggingfaceon PyPI". The "Wait for nodetool-core on PyPI" step runs after it.Both repos publish on a tag push. Releasing them together (
nodetool-core v0.7.10andnodetool-huggingface v0.7.8, pushed minutes apart today) starts two wheel builds in parallel, and the image job reads "latest" before either has finished. The result is an image tagged:0.7.10/:latestthat bundles the previousnodetool-huggingface.That is not hypothetical for this release: v0.7.8 carries the nunchaku Qwen rope shim and the FLUX variant routing fix, and an image built from
0.7.7has neither.Worse, it was invisible — the version only appeared in a build log line, nothing in the image recorded it.
The fix
so a pulled image can be checked with
docker inspectinstead of a rebuild.workflow_dispatchwith an explicithf_versionis unchanged and still wins.Found while releasing core 0.7.10 and huggingface 0.7.8 together.