Skip to content

Cleanup Github workflows and Dockerfiles#90

Merged
sandreim merged 4 commits into
mainfrom
cyon_fix_cd_docker
Sep 25, 2025
Merged

Cleanup Github workflows and Dockerfiles#90
sandreim merged 4 commits into
mainfrom
cyon_fix_cd_docker

Conversation

@Sajjon

@Sajjon Sajjon commented Sep 19, 2025

Copy link
Copy Markdown
Contributor

Tip

I've locally built the sender binary using docker, and ran the container locally using docker and it spams transactions
successfully, see e.g. this block with lots of transactions

Fix issues with CI, see failing recent jobs, issues:

  • We were trying to compile our crates with non existing features, neither tick, versi-tick, versi-relay nor rococo
  • Error: building at STEP "COPY --from=builder /build/utils/funder/target/release/funder /usr/local/bin": checking on sources under "/home/runner/.local/share/containers/storage/overlay/6d09ccf8d236e9aefb6e662b7b49233d9e3fd68d5e3ef3e7fcc47cb7379452ba/merged": copier: stat: "/build/utils/funder/target/release/funder": no such file or directory

Several improvements, bug fixes and clean up regarding CI/CD (Github workflow) and Dockerfile contents:

  • Removed .github/workflows/build-images-pr.yml since it was being triggered by pushes to a branch which does not even exist (s0me0ne/overhaul)
  • Removed jobs to build and push docker releases named: tps-tick, sender-versi-tick, sender-versi-relay, tps-versi-relay, tps-versi-tick, sender-rococo, tps-rococo`, these all references cargo feature flags which does not exist.
  • leaving a two CI jobs: stps-funder and stps-sender, using the Dockerfiles utils/dockerfiles/Dockerfile.funder and utils/dockerfiles/Dockerfile.sender respectively, which has been updated too:
  • Updated content of utils/dockerfiles/Dockerfile.sender-any (renamed to utils/dockerfiles/Dockerfile.sender) to use latest Ubuntu and fix compilation warnings
  • Updated content of utils/dockerfiles/Dockerfile.funder to use latest Ubuntu and fix compilation warnings

@Sajjon Sajjon changed the title cleanup github workflows and dockerfiles Cleanup Github workflows and Dockerfiles Sep 19, 2025
@Sajjon
Sajjon requested a review from Copilot September 19, 2025 10:05

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR cleans up CI workflows and Dockerfiles by removing broken configurations and standardizing the build process. It addresses compilation failures caused by non-existent cargo features and missing build artifacts.

  • Removed broken Docker build jobs for non-existent features (tick, versi-tick, versi-relay, rococo)
  • Consolidated CI to two working jobs: stps-funder and stps-sender
  • Updated Dockerfiles to use latest Ubuntu and fix build paths

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
.github/workflows/build-images-pr.yml Removed entire workflow file for non-existent branch
.github/workflows/build-images.yml Removed broken jobs and updated tags for remaining jobs
utils/dockerfiles/Dockerfile.tps Removed entire Dockerfile
utils/dockerfiles/Dockerfile.sender-any Removed entire Dockerfile
utils/dockerfiles/Dockerfile.sender Updated to use workspace build and latest Ubuntu
utils/dockerfiles/Dockerfile.funder Updated to use workspace build and latest Ubuntu

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment thread utils/dockerfiles/Dockerfile.funder Outdated
@Sajjon
Sajjon requested a review from Copilot September 19, 2025 10:29

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.


Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment thread utils/dockerfiles/Dockerfile.sender
Comment thread utils/dockerfiles/Dockerfile.funder
Comment thread utils/dockerfiles/Dockerfile.sender
Comment thread utils/dockerfiles/Dockerfile.funder

@sandreim sandreim left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

run: |
echo "New image has been pushed to ${{ steps.push-to-dockerhub.outputs.registry-path }}"

tps-tick:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So now, we can basically use sender/funder and the previous images are not needed ?

@Sajjon Sajjon Sep 22, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, so in 700b4d8 the repo was significantly changed, and all the feature-flags got removed, however, the Github workflow pipeline were not updated, nor the Dockerfile.

So this PR (tries to) sync the Dockerfiles and Github workflow pipeline with the code base.

@sandreim
sandreim requested a review from a team September 22, 2025 09:13
Comment thread .github/workflows/build-images.yml Outdated
@@ -15,7 +15,7 @@ jobs:
uses: redhat-actions/buildah-build@v2

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should use commits instead of tags here, for security reasons.

@Sajjon Sajjon Sep 23, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed! Can you provide me with some appropriate tags commit hashes? :)

Comment thread .github/workflows/build-images.yml Outdated
Co-authored-by: Javier Viola <363911+pepoviola@users.noreply.github.com>
@Sajjon
Sajjon requested a review from pepoviola September 24, 2025 10:12
@sandreim
sandreim merged commit f3abd20 into main Sep 25, 2025
3 checks passed
@Sajjon

Sajjon commented Sep 25, 2025

Copy link
Copy Markdown
Contributor Author

@pepoviola CD failed when we merged this, see https://github.com/paritytech/polkadot-stps/actions/runs/18002958206/job/51216412992

we seem to have permission issues?

time="2025-09-25T09:26:02Z" level=error msg="Failed to parse the authentication scope from the given challenge" challenge="{bearer map[error:insufficient_scope realm:https://auth.docker.io/token scope:repository:paritytech/stps-sender:pull,push repository:library/ubuntu:pull service:registry.docker.io]}" error="error parsing auth scope: 'repository:paritytech/stps-sender:pull,push repository:library/ubuntu:pull'" scope="repository:paritytech/stps-sender:pull,push repository:library/ubuntu:pull"
Error: writing blob: initiating layer upload to /v2/paritytech/stps-sender/blobs/uploads/ in registry-1.docker.io: requested access to the resource is denied

can you help us with that? :)

@alvicsam

Copy link
Copy Markdown
Contributor

@Sajjon I'm checking the dockerhub settings

@alvicsam

alvicsam commented Sep 25, 2025

Copy link
Copy Markdown
Contributor

@Sajjon I fixed the permissions. The docker hub repo seemed abandoned to me (latest image push was 2 years ago) so I removed credentials. Now I put them back and the pipeline works: https://github.com/paritytech/polkadot-stps/actions/runs/18002958206
And a late review (sorry, missed the PR): idk the plans for the project but usually ci with docker consists of 2 parts: just building the image on PR (to check that it can actually be built) and build+push on master. Also using the latest tag is not a good practise because you don't actually know which version is deployed so it's better to use either a tag or a commit hash.

@alvicsam
alvicsam deleted the cyon_fix_cd_docker branch September 25, 2025 15:09
@pepoviola

Copy link
Copy Markdown
Contributor

Thanks for fix the permissions @alvicsam 🚀 .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants