Skip to content

Keep builds and uploads working across compiler and metadata updates - #410

Open
Str-Gen wants to merge 2 commits into
modular:mainfrom
Str-Gen:fix/compatibility-commit-after-pull
Open

Str-Gen wants to merge 2 commits into
modular:mainfrom
Str-Gen:fix/compatibility-commit-after-pull

Conversation

@Str-Gen

@Str-Gen Str-Gen commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Keep package builds and uploads moving when recipes use different Mojo compiler versions or another runner has already committed the same compatibility metadata.

 build each recipe:
-    share output/ as the highest-priority dependency channel
+    use output/<recipe>/ so other recipes cannot shadow published dependencies

 git pull origin main
+if nothing remains staged:
+    return success
 git commit
 git push

The Linux failure in run 39267609540 occurs when Pyroquet resolves its dependencies: the locally rebuilt mojo-snappy 0.2.0 requires Mojo 1.1.0, while Pyroquet requires Mojo 1.0.0. Strict channel priority hides the compatible published mojo-snappy build. Isolate each recipe's output channel; the upload helper already discovers packages recursively. Keep Pyroquet's existing pins while its dependency stack awaits published Mojo 1.1-compatible packages.

The earlier macOS failure in run 39098506151 was an empty metadata commit after pulling an identical upstream update. Recheck the staged diff after pulling, preserving errors from the diff command.

Evidence

  • Before: A local-channel regression runs the real rattler-build solver through scripts.build-all and fails with the same strict-channel-priority / Mojo 1.1 dependency error.
    After: The same regression passes with per-recipe output directories. A second test verifies recursive upload discovery.
  • The three Git integration tests cover identical concurrent updates, new updates committed and pushed, and unchanged data.
  • All five tests pass with pixi run python -m unittest discover -s tests -v; repository pre-commit checks pass using Pixi 0.81.0 and the locked environment.
  • Full package compilation remains subject to the updated PR's Linux/macOS/ARM CI checks.

Merge Danger

Door: Two-way; both changes can be reverted.

Blast Radius: CI. Build artifacts now live under output/<recipe>/<platform>/. Recipes resolve dependencies from published or explicitly supplied channels instead of implicitly consuming another recipe's output from the same invocation. This prevents build order from selecting incompatible compiler builds, but dependencies that are not published yet still require an explicit channel. The existing recursive upload helper supports the new layout. No recipe or dependency version changes. The metadata fix handles empty commits, not every concurrent push race.

After merge, verify that the build/upload workflow publishes the mojo-snappy 0.2.0 build 1 rebuild from #408, or trigger it on main if needed.

@Str-Gen Str-Gen changed the title Handle compatibility updates already present after pull Keep builds and uploads working across compiler and metadata updates Oct 20, 2026
@Str-Gen

Str-Gen commented Oct 20, 2026

Copy link
Copy Markdown
Contributor Author

Mojo-snappy was built against Mojo 1.1.0 with changes to conform to the changelog. However, it is used in pyroquet because parquet uses snappy compression (among others). However, my pyroquet package also imports NuMojo and they don't have a release against Mojo 1.1.0 yet so for now this is blocked but PR 412 should solve this and then both mojo-snappy and pyroquet can be built and distributed through prefix.dev.
I'll go through a similar process then for Balsa and Pyre (Treelite v4 reader / writer) & accelerated forest inference library
No action is needed for now. I'm still figuring out the workflow.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants