Skip to content

fix(builder): add ftp.gnu.org fallback for GNU source downloads - #361

Open
vbelouso wants to merge 1 commit into
calungaproject:mainfrom
vbelouso:fix/gnu-mirror-fallback
Open

fix(builder): add ftp.gnu.org fallback for GNU source downloads#361
vbelouso wants to merge 1 commit into
calungaproject:mainfrom
vbelouso:fix/gnu-mirror-fallback

Conversation

@vbelouso

@vbelouso vbelouso commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

ftpmirror.gnu.org occasionally redirects to a backend with an expired TLS certificate. curl --retry does not retry SSL errors (exit 60), so the build fails even though the artifact is available elsewhere.

fetch_source now accepts an optional fallback URL. When provided, the preferred source is tried preferred_attempts=3 times (shell loop, retries any curl error). On exhaustion it falls back to the canonical URL with the same retry policy. Without a fallback the existing --retry 10 behaviour is unchanged.

Containerfile exports AUTOCONF/AUTOMAKE/LIBTOOL_FALLBACK_URL=https://ftp.gnu.org/gnu/... for the three affected packages.

@sourcery-ai

sourcery-ai Bot commented Jul 27, 2026

Copy link
Copy Markdown

Reviewer's Guide

Adds a fallback mechanism for GNU source downloads in the builder scripts to improve resilience against bad TLS on ftpmirror.gnu.org, wires the new fallback into GNU-related install scripts and the Containerfile, tweaks Arrow build flags for x86_64, and slightly increases the CI test job timeout.

File-Level Changes

Change Details Files
Introduce a fallback-aware fetch_source helper that retries a primary URL then an optional fallback URL with robust cleanup and logging, while preserving legacy behavior when no fallback is provided.
  • Extend fetch_source to accept an optional third argument for a fallback URL in addition to the primary URL.
  • Preserve the original single-URL behavior by keeping curl --retry 10 when no fallback URL is passed.
  • Implement explicit retry loops for both preferred and fallback URLs with up to three attempts each, retrying on any curl failure including TLS errors.
  • On each failed attempt, log the curl exit code and remove any partially downloaded file before retrying.
  • Return early when the target file already exists and propagate the last curl exit code when all attempts fail.
builder/build_scripts/build_utils.sh
Wire GNU package installers to use the new fetch_source fallback support, with Containerfile exporting ftp.gnu.org-based fallback URLs.
  • Update autoconf, automake, and libtool installer scripts to pass a package-specific fallback URL into fetch_source when defined.
  • Export AUTOCONF_FALLBACK_URL, AUTOMAKE_FALLBACK_URL, and LIBTOOL_FALLBACK_URL in the Containerfile pointing to ftp.gnu.org mirrors for the respective packages.
builder/build_scripts/install-autoconf.sh
builder/build_scripts/install-automake.sh
builder/build_scripts/install-libtool.sh
builder/Containerfile
Adjust Arrow build configuration for x86_64 to avoid linking Boost UUID with libatomic via extra CMake arguments.
  • Introduce a cmake_extra_args array and detect the target architecture from AUDITWHEEL_ARCH.
  • When building for x86_64, add -DBOOST_UUID_LINK_LIBATOMIC=OFF to the CMake configuration via cmake_extra_args.
  • Append the cmake_extra_args expansion to the existing cmake invocation without changing non-x86_64 builds.
builder/build_scripts/build-arrow.sh
Relax CI constraints slightly by allowing more time for tests to complete.
  • Increase the GitHub Actions test job timeout from 60 minutes to 90 minutes to reduce spurious failures on slower runs.
.github/workflows/test.yml

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • The new fetch_source fallback logic hardcodes preferred_attempts and fallback_attempts to 3; consider making these configurable (e.g., via environment variables) so you can tune retry behavior without changing the script.
  • In fetch_source, all status messages (including failures and the final 'Both preferred and fallback sources failed') currently go to stdout; routing error-related logs to stderr would make it easier to distinguish build noise from actual failures in CI logs.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The new fetch_source fallback logic hardcodes preferred_attempts and fallback_attempts to 3; consider making these configurable (e.g., via environment variables) so you can tune retry behavior without changing the script.
- In fetch_source, all status messages (including failures and the final 'Both preferred and fallback sources failed') currently go to stdout; routing error-related logs to stderr would make it easier to distinguish build noise from actual failures in CI logs.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@vbelouso
vbelouso force-pushed the fix/gnu-mirror-fallback branch from 09826c7 to f787d71 Compare July 27, 2026 17:27
@vbelouso
vbelouso force-pushed the fix/gnu-mirror-fallback branch from f787d71 to eb0e931 Compare July 27, 2026 17:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant