Skip to content

Add stdlib load-time regression CI check (macOS aarch64)#558

Draft
IanButterworth wants to merge 2 commits into
JuliaCI:mainfrom
IanButterworth:ib/precompile_load_regression_check
Draft

Add stdlib load-time regression CI check (macOS aarch64)#558
IanButterworth wants to merge 2 commits into
JuliaCI:mainfrom
IanButterworth:ib/precompile_load_regression_check

Conversation

@IanButterworth

Copy link
Copy Markdown
Member

Claude:


Compares the Julia built by the pipeline (candidate) against the nightly associated with the merge-base commit (baseline). For each binary it re-precompiles every stdlib into an isolated, cache-free depot and measures load times, flagging clear regressions (relative and absolute thresholds).

  • utilities/stdlib_load_time_regression.sh: driver that downloads/extracts both binaries, building the baseline from source if the merge-base nightly is missing, then runs the comparison. Only runs on macOS aarch64.
  • compare_stdlib_load_times.jl: orchestrator. Per binary, generates a resolved stdlib Project/Manifest with the standard depot, precompiles the whole environment at once into a fresh depot, then measures load times in A/B/B/A/B/A order taking the per-stdlib minimum across repeats.
  • measure_stdlib_load_times.jl: per-binary helper with generate/precompile/ measure modes; each measure runs one stdlib in its own process so previously-loaded deps don't skew timings.
  • pipelines/main/misc/stdlib_load_time_regression.yml + launcher upload.

Compares the Julia built by the pipeline (candidate) against the nightly
associated with the merge-base commit (baseline). For each binary it
re-precompiles every stdlib into an isolated, cache-free depot and measures
load times, flagging clear regressions (relative and absolute thresholds).

- utilities/stdlib_load_time_regression.sh: driver that downloads/extracts
  both binaries, building the baseline from source if the merge-base nightly
  is missing, then runs the comparison. Only runs on macOS aarch64.
- compare_stdlib_load_times.jl: orchestrator. Per binary, generates a
  resolved stdlib Project/Manifest with the standard depot, precompiles the
  whole environment at once into a fresh depot, then measures load times in
  A/B/B/A/B/A order taking the per-stdlib minimum across repeats.
- measure_stdlib_load_times.jl: per-binary helper with generate/precompile/
  measure modes; each measure runs one stdlib in its own process so
  previously-loaded deps don't skew timings.
- pipelines/main/misc/stdlib_load_time_regression.yml + launcher upload.
@IanButterworth
IanButterworth force-pushed the ib/precompile_load_regression_check branch from da68d6e to 82e6009 Compare June 19, 2026 15:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant