Repository navigation
[CI] Avoid duplicate Python snapshot builds - #40856
Conversation
5e57483 to
1760453
Compare
|
Concrete duplication evidence from the October 28 scheduled snapshot run, on
Both logs show the base-image task executing, without The dependency is explicit in distroless/common.gradle: dockerPrepare.dependsOn ":sdks:python:container:py${pythonVersionSuffix}:docker"The remaining distroless matrix jobs therefore still run the matching base-image tasks. The workflow arguments retain the registry, tag list, In that run, the standalone jobs took 77m50s for Python 3.11, 98m44s for 3.12, 78m43s for 3.13, and 78m06s for 3.14. That is 5h33m23s of aggregate runner-job time. This is the observed cost of those jobs in one run. A completed publication run with this change is still needed to measure the actual reduction. |
There was a problem hiding this comment.
📋 Review Summary
This PR removes the standalone python:container:py31x:docker matrix entries from the Publish Beam SDK Snapshots GitHub Actions workflow. Because each python:container:distroless:py31x:docker task already depends on :sdks:python:container:py31x:docker (via dockerPrepare.dependsOn) and inherits the same root project properties (-Ppush-containers, -Pdocker-tag-list, -Pcontainer-architecture-list, -Pdocker-pull-licenses), both the base and distroless Python container images are still built and pushed with identical configurations while saving significant CI runner time and avoiding concurrent pushes to the same image tags.
🔍 General Feedback
- The Gradle task dependency graph in
sdks/python/container/distroless/common.gradleandBeamDockerPlugin.groovyconfirms that:sdks:python:container:py31x:dockerbuilds and pushes the multi-arch base images with all specified tags prior to building and pushing the distroless images. - No issues found; the change is clean, accurate, and well-scoped.
💡 Tip: Comment /gemini review on this pull request to trigger a new Gemini code review.
|
Assigning reviewers: R: @Amar3tto for label build. Note: If you would like to opt out of this review, comment Available commands:
The PR bot will only process comments in the main thread (not review comments). |
The distroless Python image task already builds and pushes its matching base image. The snapshot matrix also runs that base task separately, repeating the SDK image build (example run). In that sample, the standalone Python jobs occupied about 5.6 hours of total runner time.
Remove the standalone Python base image jobs. The remaining distroless jobs still publish both image families with the same Python versions, tags, and architectures.
Validated the Gradle task graphs and Docker task configuration, along with workflow syntax. Full SDK image publication was not run locally.