Assorted bootstrap LLVM refactors (part 1/N) by Kobzol · Pull Request #160645 · rust-lang/rust

Kobzol · 2026-08-06T15:03:10Z

This PR continues my LLVM and download-ci-llvm bootstrap refactors (it took me almost a year to get back to them, lol), with the goals of:

Remove dependency on implicit paths with artifacts. Ideally, everything should only depend on output paths from executed steps.
Reduce eager and implicit downloads and other shenanigans happening in config parsing, particularly around download-ci-llvm.
Allow downloading LLVM from CI for non-host targets. There is not really any reason why bootstrap shouldn't be able to download LLVM from CI if it's there for any given target, but right now it can only do so for the host target. This would be useful for debugging bootstrap itself, and perhaps for some people also for development.
Centralize the handling of LLVM into the Llvm step, and reduce overall knowledge between locally built and downloaded LLVM. Ideally, most of bootstrap shouldn't know about whether it uses a LLVM that was built locally or downloaded. But that is of course not the case today, because there are many places in bootstrap that ad-hoc work with some LLVM paths, and do things that depend on some part of bootstrap randomly modifying a config here or there, or doing some side effect, which then makes the other activity "work". But of course, this is very fragile, as we know.
If I manage to reach the previous goal, this should massively help unblock further unborking of download-ci-rustc. Those two features are quite interrelated, and sadly both are scattered across the codebase. I actually first wanted to start with refactoring download-ci-rustc, but I couldn't find a way to do that without first improving download-ci-llvm.

This PR contains a bunch of commits that slowly move us towards these goals. The refactors were quite explanatory, they are mostly a preparation for larger refactors that I had in mind, so nothing major. But as always, step by step.

As usually, best reviewed commit-by-commit.

Some notes for review:

The last commit might case a behavior change (well, or an error) on MSVC without Ninja, when filecheck is used without download-ci-llvm. As per the experiment on Zulip, this should be fine. And I'm not sure if anyone actually uses MSVC without Ninja in bootstrap, it seems to be quite broken.
07b88f2 removes a sanity check for FileCheck being present. It was a bit hacky, because it was only checking it if it wasn't locally built (or downloaded from CI...). To avoid introducing more hacks, I just removed this sanity check. If the code will need filecheck and it won't be available, it will error out at the usage site (or I could add an assert that the FileCheck binary is present in the FileCheck step if you want).

r? jieyouxu

rustbot · 2026-08-06T15:03:13Z

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

This PR changes how LLVM is built. Consider updating src/bootstrap/download-ci-llvm-stamp.

rustbot · 2026-08-06T15:03:16Z

jieyouxu is currently at their maximum review capacity.
They may take a while to respond.

jieyouxu · 2026-08-08T07:21:52Z

(Will take a look later today)

jieyouxu

Thanks, r=me after a rebase.
@bors rollup=never note="bootstrap llvm refactors"

View changes since this review

jieyouxu · 2026-08-09T08:43:18Z

@rustbot author

At this point in the code, the LLVM config could not have been set by `download-ci-llvm` yet, so we don't have to check it.

To make it consistent with `GccOutput`

So that it can be used explicitly in bootstrap, rather than depending on `builder.llvm_out`.

To remove dependency on implicit paths.

And replace it with an explicit `FileCheck` step

To avoid someone depending on implicit paths. Instead, the step should be executed and the path should be taken from its output.

…-llvm` is enabled Note: this commit removed reading `FileCheck` from `<artifact-dir>/build/<profile>/bin`, and instead reads it from `<artifact-dir>/bin` directly. It should still work with MSVC and without Ninja.

rustbot · 2026-08-10T08:05:45Z

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

Kobzol · 2026-08-10T08:05:46Z

@bors r=jieyouxu

rust-bors · 2026-08-10T08:05:49Z

📌 Commit 7a1d067 has been approved by jieyouxu

It is now in the queue for this repository.

rust-bors · 2026-08-10T18:51:34Z

☀️ Test successful - CI
Approved by: jieyouxu
Duration: 3h 9m 24s
Pushing ef20314 to main...

github-actions · 2026-08-10T18:54:47Z

What is this?

This is an experimental post-merge analysis report that shows differences in test outcomes between the merged PR and its parent PR.

Comparing ea06042 (parent) -> ef20314 (this PR)

Test differences

No test diffs found

Test dashboard

Run

cargo run --manifest-path src/ci/citool/Cargo.toml -- \
    test-dashboard ef20314466010b8b9259ec5f86230c530ca08661 --output-dir test-dashboard

And then open test-dashboard/index.html in your browser to see an overview of all executed tests.

Job duration changes

x86_64-gnu-gcc-core-tests: 14m 53s -> 6m 27s (-56.6%)
aarch64-apple-macos-26-2: 1h 49m -> 2h 38m (+44.7%)
x86_64-gnu-stdlib-semver-check: 9m 53s -> 5m 36s (-43.4%)
dist-powerpc64le-linux-musl: 1h 7m -> 1h 36m (+43.3%)
dist-ohos-aarch64: 53m 37s -> 1h 15m (+40.3%)
x86_64-mingw-2: 2h 41m -> 1h 36m (-39.9%)
x86_64-mingw-1: 2h 49m -> 1h 46m (-37.1%)
aarch64-apple-1: 1h 31m -> 2h 3m (+35.3%)
x86_64-gnu-parallel-frontend: 2h 5m -> 1h 27m (-30.5%)
pr-check-2: 40m 31s -> 28m 26s (-29.8%)

How to interpret the job duration changes?

Job durations can vary a lot, based on the actual runner instance
that executed the job, system noise, invalidated caches, etc. The table above is provided
mostly for t-infra members, for simpler debugging of potential CI slow-downs.

rust-timer · 2026-08-10T19:55:33Z

Finished benchmarking commit (ef20314): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This perf run didn't have relevant results for this metric.

Max RSS (memory usage)

Results (primary -2.9%, secondary 0.4%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	2.0%	[0.4%, 6.1%]	4
Improvements ✅ (primary)	-2.9%	[-2.9%, -2.9%]	1
Improvements ✅ (secondary)	-0.6%	[-0.9%, -0.5%]	6
All ❌✅ (primary)	-2.9%	[-2.9%, -2.9%]	1

Cycles

Results (secondary 0.1%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	1.0%	[0.4%, 2.7%]	11
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-0.7%	[-1.3%, -0.4%]	12
All ❌✅ (primary)	-	-	0

Binary size

This perf run didn't have relevant results for this metric.

Bootstrap: 453.486s -> 455.478s (0.44%)
Artifact size: 398.53 MiB -> 398.50 MiB (-0.01%)

Assorted bootstrap LLVM refactors (part 2/N) Continuation of #160645. This PR removes dependence on implicit LLVM CI paths, and generally tries to make depending on various LLVM things more explicit wherever possible. One nice outcome of that is that we no longer override the LLVM config during config parsing when using `download-ci-llvm` 🎉 We also no longer dynamically download LLVM from CI when someone asks what is the LLVM linkage mode, lol (well, we still need to do it in a few places, but now at least it is explicit). I tried to ensure that `x check compiler` does not build or even checkout the LLVM submodule. That being said, there might be some cases where calling `get_llvm_build_status` might now checkout the LLVM submodule (but it shouldn't happen for `x test mir-opt --bless`, which was fixed in #130918 - that PR introduced a boolean flag, but it was difficult to understand where it should be set to `false`/`true`). The next thing after this is to try getting rid of the `llvm_out` function, though that will be probably quite painful. And also explicitly depending on the output of `LlvmFromCi`, rather than reading `llvm_ci_mode`. I printed some step traces using commands like this: ``` BOOTSTRAP_TRACING=STEP=trace BOOTSTRAP_TRACING_SKIP_TIME=1 ./x build compiler 2> post-cg_default-build-compiler.txt ``` with this PR and on `main`. The `BOOTSTRAP_TRACING_SKIP_TIME` thing was added in the first commit to make it easier to compare step traces. But later I realized that we should also normalize source code locations of the steps, otherwise if you modify something in a given file with the step, its location will change and it will perturb the diff. Anyway, I can drop the first commit if you want. Best reviewed commit-by-commit (in fact I don't think it can be really reviewed in another way, lol). r? jieyouxu

rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Aug 6, 2026

rustbot assigned jieyouxu Aug 6, 2026

This comment has been minimized.

Sign in to view

Kobzol force-pushed the bootstrap-llvm branch from 332b3c6 to 91487a5 Compare August 7, 2026 06:54

This comment has been minimized.

Sign in to view

Kobzol force-pushed the bootstrap-llvm branch from 91487a5 to d29b734 Compare August 7, 2026 07:26

This comment has been minimized.

Sign in to view

jieyouxu approved these changes Aug 9, 2026

View reviewed changes

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 9, 2026

Kobzol added 11 commits August 10, 2026 10:02

Avoid using is_system_llvm in Config parsing

8d16052

At this point in the code, the LLVM config could not have been set by `download-ci-llvm` yet, so we don't have to check it.

Rename LlvmResult to LlvmOutput

1e8b7cb

To make it consistent with `GccOutput`

Store LLVM root directory in LlvmOutput

87f5a10

So that it can be used explicitly in bootstrap, rather than depending on `builder.llvm_out`.

Remove some uses of llvm_out

871da93

To remove dependency on implicit paths.

Add helper function for computing LLVM CMake profile

d357897

Remove unused root field of the Meta LLVM struct

2d6631e

Remove unused out_dir field and rename Meta to LlvmBuildInfo

e408d2f

Remove the builder::llvm_filecheck function

d938939

And replace it with an explicit `FileCheck` step

Remove lld_out, enzyme_out and offload_out functions

6c0a863

To avoid someone depending on implicit paths. Instead, the step should be executed and the path should be taken from its output.

Stop overwriting llvm_filecheck in config parsing when `download-ci…

bfc02c4

…-llvm` is enabled Note: this commit removed reading `FileCheck` from `<artifact-dir>/build/<profile>/bin`, and instead reads it from `<artifact-dir>/bin` directly. It should still work with MSVC and without Ninja.

Use the new LLVM output in the RustOffload step

7a1d067

Kobzol force-pushed the bootstrap-llvm branch from d29b734 to 7a1d067 Compare August 10, 2026 08:05

rust-bors Bot added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Aug 10, 2026

This comment has been minimized.

Sign in to view

rust-bors Bot added merged-by-bors This PR was explicitly merged by bors. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Aug 10, 2026

rust-bors Bot merged commit ef20314 into rust-lang:main Aug 10, 2026
14 checks passed

rustbot added this to the 1.99.0 milestone Aug 10, 2026

This was referenced Aug 10, 2026

Allow building LLVM DLL with MSVC #159574

Draft

Update openmp/offload to new LLVM-22 build setup #152011

Open

Kobzol deleted the bootstrap-llvm branch August 10, 2026 19:15

Kobzol mentioned this pull request Aug 11, 2026

Assorted bootstrap LLVM refactors (part 2/N) #160916

Open

Uh oh!

Uh oh!

Conversation

Kobzol commented Aug 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented Aug 6, 2026

Uh oh!

rustbot commented Aug 6, 2026

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

jieyouxu commented Aug 8, 2026

Uh oh!

jieyouxu left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jieyouxu commented Aug 9, 2026

Uh oh!

rustbot commented Aug 10, 2026

Uh oh!

Kobzol commented Aug 10, 2026

Uh oh!

rust-bors Bot commented Aug 10, 2026

Uh oh!

This comment has been minimized.

rust-bors Bot commented Aug 10, 2026

Uh oh!

Uh oh!

github-actions Bot commented Aug 10, 2026

Test differences

Job duration changes

Uh oh!

rust-timer commented Aug 10, 2026

Overall result: no relevant changes - no action needed

Instruction count

Max RSS (memory usage)

Cycles

Binary size

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Kobzol commented Aug 6, 2026 •

edited

Loading

jieyouxu left a comment •

edited by rustbot

Loading