[tslmy]

Summary

This adds an early build step that configures libdnf5 for faster and more resilient package downloads during image builds:

• enables fastestmirror
• raises max_parallel_downloads to 10
• sets a minimum transfer rate and timeout so very slow mirrors fail over sooner
• adds an optional ARMADA_FEDORA_MIRROR build arg for builders who know a nearby Fedora mirror

By default this does not hard-code a regional mirror. The explicit mirror override is only applied when ARMADA_FEDORA_MIRROR is set, and it only rewrites Fedora repo files that use Fedora standard commented baseurl template.

The build now writes the libdnf5 speed-up config at the start of the container build, optionally rewrites Fedora repo files when ARMADA_FEDORA_MIRROR is set, and then runs a late restore step after the final DNF operations. That restore step removes /etc/dnf/libdnf5.conf.d/10-armada-build.conf and restores any Fedora repo files it temporarily changed before the image is finalized.

So the installed OS should keep Fedora's normal repository configuration, while local/CI image builds can still use faster parallel downloads and an operator-selected nearby mirror.

Speed-up observed

Tested from the western US while building in a Fedora Lima VM on macOS. With an explicit nearby mirror, DNF download phases that had previously selected slow mirrors or stalled in the hundreds of KiB/s range were observed around 6-8 MiB/s on representative transactions. One later DNF transaction in the successful image build downloaded 43 MiB at about 7.8 MiB/s.

The total build-time improvement depends heavily on cache state and non-DNF work, especially package/image assembly, but the DNF download portions were roughly an order of magnitude faster in this test environment.

Usage

The general build path gets the libdnf5 tuning automatically. Builders who want to pin a known-good mirror can run, for example:

ARMADA_FEDORA_MIRROR=https://mirrors.kernel.org/fedora just build localhost/armada test

Builders elsewhere can use a geographically appropriate Fedora mirror, or leave ARMADA_FEDORA_MIRROR unset and rely on Fedora mirror selection with fastestmirror enabled.

Validation

• bash -n build_files/05-configure-dnf.sh build_files/build.sh
• just --fmt --check
• completed a local image build in a Fedora Lima VM from the western US using ARMADA_FEDORA_MIRROR=https://mirrors.kernel.org/fedora

[JPyke3]

Hey Mate, appreciate the PR. You mention specifically that you're optimizing for buildtime. However, it seems this PR is going to ship these dnf changes to users?

I've always found dnf to have pretty sensible defaults and I live in Au where out internet is not renowned for speed.

So as is I'd be a bit reluctant to merge as I don't think built time needs to be sped up at the expense of user defaults.

Happy to hear your thoughts though!

[tslmy]

Hey @JPyke3 , thanks for responding this quickly.

Totally agree that we should avoid shipping user defaults. As you see, I made a point of making ARMADA_FEDORA_MIRROR a variable instead of hardcoding it. As for fastestmirror=True -- I had thought that it was harmless to ship as a default.

But now that you asked, I got curious why fastestmirror defaults to False. Turns out it picks by TCP socket latency (not necessarily the fastest) and can override mirror-manager’s own ordering, including bandwidth/preferred-mirror hints. Indeed, that's not I'd like to ship to end users.

pretty sensible defaults

Right, TIL about "Fedora MirrorManager" (bear with me please -- I'm on a Mac and have little experience with DNF 😅 ). Somehow, in my western-US build environment, it appears to have selected one (or more) mirrors with poor sustained throughput. (https://terrapkg.com/ 500'd on me once.) Pinning at mirrors.kernel.org avoided that path and gave ~6-8 MiB/s.

Back to this PR. I've added a script to remove that DNF conf file and trap it on EXIT in build.sh.

This makes the override ephemeral, which feels less worthy of a PR merge than I'd previously thought. So totally cool if we don't end up merging it (I can still keep this change local; I'm not missing out on anything). Thanks!

[JPyke3]

@tslmy appreciate the contribution on this one but will close as per the discussion above :)