Skip to content

infra: avoid excessive objdump calls and adjust all.sh - #16083

Merged
DavidKorczynski merged 2 commits into
masterfrom
speed-up-base-check
Sep 1, 2026
Merged

infra: avoid excessive objdump calls and adjust all.sh#16083
DavidKorczynski merged 2 commits into
masterfrom
speed-up-base-check

Conversation

@DavidKorczynski

Copy link
Copy Markdown
Collaborator

We're currently doing excessive objdump calls in the bad build check. We only need it once to calculate the sanitizer keywords.

Also adjust all.sh so it's no longer needed to set up specific Python packages to build the base images. This also makes it very easy to quickly adjust which base images to build by modifying the file. This is how it was earlier which IMO works better when adjusting the base images.

We're currently doing excessive objdump calls in the bad build check. We
only need it once to calculate the sanitizer keywords.

Also adjust all.sh so it's no longer needed to set up specific Python
packages to build the base images. This also makes it very easy to
quickly adjust which base images to build by modifying the file. This is
how it was earlier which IMO works better when adjusting the base
images.

Signed-off-by: David Korczynski <david@adalogics.com>
@ViniciustCosta

Copy link
Copy Markdown
Contributor

/gcbrun

@ViniciustCosta ViniciustCosta left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

Comment thread infra/base-images/all.sh
@DavidKorczynski

Copy link
Copy Markdown
Collaborator Author

/gcbrun trial_build.py htslib json-c javacpp jansson janet --fuzzing-engines libfuzzer --sanitizers address

DavidKorczynski referenced this pull request Sep 1, 2026
Updates LLVM usage to match
[6d1ace547c9f](llvm/llvm-project@6d1ace547c9f)

Indexer-PiperOrigin-RevId: 956634975
@DavidKorczynski

DavidKorczynski commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator Author

The trial run failed because the indexer is pushed out of sync with our llvm version. See 0f4d4d3#commitcomment-198554014

Consequently, no projects were build.

Will look into a fix for this to avoid bailing out of trial runs in case the indexer fails, considering the indexer is out-of-sync at times. Will look at making the indexer compatible with our LLVM version.

Signed-off-by: David Korczynski <david@adalogics.com>
@DavidKorczynski

Copy link
Copy Markdown
Collaborator Author

/gcbrun trial_build.py htslib json-c javacpp jansson janet --fuzzing-engines libfuzzer --sanitizers address

@DavidKorczynski

Copy link
Copy Markdown
Collaborator Author

The above succeeded:

Step #1 - "Legacy": INFO:root:-----------------------
Step #1 - "Legacy": INFO:root:Waiting for 4 builds from 4 projects to complete...
Step #2 - "Ubuntu 20.04": INFO:root:================================================================
Step #2 - "Ubuntu 20.04": INFO:root:            PHASE 1: BASE IMAGE BUILD REPORT
Step #2 - "Ubuntu 20.04": INFO:root:----------------------------------------------------------------
Step #2 - "Ubuntu 20.04": INFO:root:GCB Build ID: 575940fb-a375-4827-b6a9-6b0a7bda7a3b
Step #2 - "Ubuntu 20.04": INFO:root:GCB Build URL: https://console.cloud.google.com/cloud-build/builds;region=us-central1/575940fb-a375-4827-b6a9-6b0a7bda7a3b?project=oss-fuzz-base
Step #2 - "Ubuntu 20.04": INFO:root:Log URL: https://oss-fuzz-gcb-logs.storage.googleapis.com/log-575940fb-a375-4827-b6a9-6b0a7bda7a3b.txt
Step #2 - "Ubuntu 20.04": INFO:root:================================================================
Step #2 - "Ubuntu 20.04": INFO:root:  - build-base-image-ubuntu-20-04: SUCCESS
Step #2 - "Ubuntu 20.04": INFO:root:  - build-base-clang-ubuntu-20-04: SUCCESS
Step #2 - "Ubuntu 20.04": INFO:root:  - build-base-clang-full-ubuntu-20-04: SUCCESS
Step #2 - "Ubuntu 20.04": INFO:root:  - build-indexer-ubuntu-20-04: SUCCESS
Step #2 - "Ubuntu 20.04": INFO:root:  - build-base-builder-ubuntu-20-04: SUCCESS
Step #2 - "Ubuntu 20.04": INFO:root:  - build-base-builder-go-ubuntu-20-04: SUCCESS
Step #2 - "Ubuntu 20.04": INFO:root:  - build-base-builder-javascript-ubuntu-20-04: SUCCESS
Step #2 - "Ubuntu 20.04": INFO:root:  - build-base-builder-jvm-ubuntu-20-04: SUCCESS
Step #2 - "Ubuntu 20.04": INFO:root:  - build-base-builder-python-ubuntu-20-04: SUCCESS
Step #2 - "Ubuntu 20.04": INFO:root:  - build-base-builder-ruby-ubuntu-20-04: SUCCESS
Step #2 - "Ubuntu 20.04": INFO:root:  - build-base-builder-rust-ubuntu-20-04: SUCCESS
Step #2 - "Ubuntu 20.04": INFO:root:  - build-base-builder-swift-ubuntu-20-04: SUCCESS
Step #2 - "Ubuntu 20.04": INFO:root:  - build-base-runner-ubuntu-20-04: SUCCESS
Step #2 - "Ubuntu 20.04": INFO:root:  - build-base-runner-debug-ubuntu-20-04: SUCCESS
Step #2 - "Ubuntu 20.04": INFO:root:----------------------------------------------------------------
Step #2 - "Ubuntu 20.04": INFO:root:Summary: 14 succeeded, 0 failed.
Step #2 - "Ubuntu 20.04": INFO:root:================================================================
Step #2 - "Ubuntu 20.04": INFO:root:All base images built successfully.
Step #2 - "Ubuntu 20.04": INFO:root:"Build and Push Images" phase completed.
Step #2 - "Ubuntu 20.04": INFO:root:================================================================
Step #2 - "Ubuntu 20.04": INFO:root:            PHASE 2: STARTING TEST BUILDS
Step #2 - "Ubuntu 20.04": INFO:root:================================================================
Step #2 - "Ubuntu 20.04": INFO:root:Build type: fuzzing
Step #2 - "Ubuntu 20.04": INFO:root:  - Selected projects: 4 / 5 (due to successful production builds)
Step #2 - "Ubuntu 20.04": INFO:root:  - To build all projects, use the --force-build flag.
Step #2 - "Ubuntu 20.04": INFO:root:Starting to create and trigger builds for build type: fuzzing
Step #2 - "Ubuntu 20.04": INFO:root:Triggered all builds.
Step #2 - "Ubuntu 20.04": INFO:root:================================================================
Step #2 - "Ubuntu 20.04": INFO:root:               PHASE 2: SKIPPED BUILDS
Step #2 - "Ubuntu 20.04": INFO:root:================================================================
Step #2 - "Ubuntu 20.04": INFO:root:Total skipped builds: 5
Step #2 - "Ubuntu 20.04": INFO:root:--- SKIPPED BUILDS ---
Step #2 - "Ubuntu 20.04": INFO:root:  - fuzzing:
Step #2 - "Ubuntu 20.04": INFO:root:    - htslib: Project requires legacy, but build version is ubuntu-20-04
Step #2 - "Ubuntu 20.04": INFO:root:    - janet: Project requires legacy, but build version is ubuntu-20-04
Step #2 - "Ubuntu 20.04": INFO:root:    - jansson: Production build failed
Step #2 - "Ubuntu 20.04": INFO:root:    - javacpp: Project requires legacy, but build version is ubuntu-20-04
Step #2 - "Ubuntu 20.04": INFO:root:    - json-c: Project requires legacy, but build version is ubuntu-20-04
Step #2 - "Ubuntu 20.04": INFO:root:-----------------------
Step #2 - "Ubuntu 20.04": INFO:root:================================================================
Step #2 - "Ubuntu 20.04": INFO:root:               PHASE 2: STARTED BUILDS
Step #2 - "Ubuntu 20.04": INFO:root:================================================================
Step #2 - "Ubuntu 20.04": INFO:root:Total projects with builds: 0
Step #2 - "Ubuntu 20.04": INFO:root:--- STARTED BUILDS ---
Step #2 - "Ubuntu 20.04": INFO:root:-----------------------
Step #2 - "Ubuntu 20.04": INFO:root:Waiting for 0 builds from 0 projects to complete...
Step #2 - "Ubuntu 20.04": INFO:root:
Step #2 - "Ubuntu 20.04": All builds passed successfully!
Step #2 - "Ubuntu 20.04": INFO:root:------------------------
Step #2 - "Ubuntu 20.04": +------------------------------------------------------------------------------+
Step #2 - "Ubuntu 20.04": |                          BUILD REPORT: UBUNTU-20-04                          |
Step #2 - "Ubuntu 20.04": +------------------------------------------------------------------------------+
Step #2 - "Ubuntu 20.04": |  Total projects analyzed: 5                                                  |
Step #2 - "Ubuntu 20.04": |  [PASSED]  Successful builds: 0                                              |
Step #2 - "Ubuntu 20.04": |  [FAILED]  Failed builds: 0                                                  |
Step #2 - "Ubuntu 20.04": |  [SKIPPED] Skipped builds: 5                                                 |
Step #2 - "Ubuntu 20.04": +------------------------------------------------------------------------------+
Finished Step #2 - "Ubuntu 20.04"
Step #1 - "Legacy": INFO:root:
Step #1 - "Legacy": All builds passed successfully!
Step #1 - "Legacy": INFO:root:------------------------
Step #1 - "Legacy": +------------------------------------------------------------------------------+
Step #1 - "Legacy": |                             BUILD REPORT: LEGACY                             |
Step #1 - "Legacy": +------------------------------------------------------------------------------+
Step #1 - "Legacy": |  Total projects analyzed: 5                                                  |
Step #1 - "Legacy": |  [PASSED]  Successful builds: 4                                              |
Step #1 - "Legacy": |  [FAILED]  Failed builds: 0                                                  |
Step #1 - "Legacy": |  [SKIPPED] Skipped builds: 1                                                 |
Step #1 - "Legacy": +------------------------------------------------------------------------------+
Finished Step #1 - "Legacy"
Starting Step #4 - "Generate Report"
Step #4 - "Generate Report": Already have image: gcr.io/oss-fuzz-base/trial-build

@DavidKorczynski

Copy link
Copy Markdown
Collaborator Author

/gcbrun trial_build.py binutils boringssl bzip2 c-blosc ghostscript openssl giflib haproxy gstreamer libpng ffmpeg libjxl libavc muduo muparser --fuzzing-engines libfuzzer --sanitizers address

@DavidKorczynski
DavidKorczynski merged commit 3c5d9f1 into master Sep 1, 2026
25 checks passed
@DavidKorczynski
DavidKorczynski deleted the speed-up-base-check branch September 1, 2026 16:26
DavidKorczynski referenced this pull request Sep 2, 2026
Updates LLVM usage to match
[cbc5a226cbf8](llvm/llvm-project@cbc5a226cbf8)

Indexer-PiperOrigin-RevId: 974715562
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants