Add --push flag to install_tpchgen_cli.sh to publish builder image - #368
Add --push flag to install_tpchgen_cli.sh to publish builder image#368misiugodfrey wants to merge 2 commits into
Conversation
Allows tagging and pushing the tpchgen-cli-builder image to GHCR instead of always discarding it after extracting the binary.
TomAugspurger
left a comment
TomAugspurger
left a comment
There was a problem hiding this comment.
Makes sense to me.
One thing to maybe think about is whether you'll need some kind of cross-compilation for different CPU architectures. But if not, then there's no need to complicate things.
| docker rm "$CONTAINER_ID" | ||
| docker rmi "$IMAGE_NAME" | ||
|
|
||
| if [[ "$PUSH" == true ]]; then |
There was a problem hiding this comment.
The final output of this script is the tpchgen-cli binary. Why push the entire docker image in this case?
There was a problem hiding this comment.
This is because the simplest workflow on a slurm cluster is to point at an external docker registry and pyxis/enroot auto-convert the container to a local enroot image.
There was a problem hiding this comment.
Unless I am missing something, copying over the binary to local_installs/bin and then running the datagen script seems much simpler than the above process?
3 participants
Summary
--pushflag toinstall_tpchgen_cli.shthat tags thetpchgen-cli-builderimage asghcr.io/rapidsai/velox-testing-images:tpchgen-cliand pushes it to GHCR, instead of always deleting it after extracting the binary.The motivation for this change is to enable the use of this binary in environments that don't support docker for a local build. Our Slurm environments can pull the docker image from the registry as an Enroot image and get access to the binary that way; but can't build it directly.
Test plan
install_tpchgen_cli.shwithout--pushand confirm the builder image is still cleaned up as beforeinstall_tpchgen_cli.sh --pushand confirm the image is tagged and pushed toghcr.io/rapidsai/velox-testing-images:tpchgen-cli