Skip to content

CI gives faster feedback without losing GHC or packaging coverage #264

Description

@alunduil

Summary

Cut CI wall-clock and compute for a package whose library is a single module.
Measure first, then trim the redundant work the matrix currently repeats ten
times over.

Motivation

Latest successful run on main (31624751509):

Job Duration
GHC 9.14 / macos-latest 14m
GHC 9.6 / macos-latest 11m
GHC 9.12 / macos-latest 10m
GHC 9.10 / macos-latest 9m
GHC 9.8 / macos-latest 9m
GHC 9.6 / ubuntu-latest 7m
GHC 9.8 / ubuntu-latest 7m
GHC 9.10 / ubuntu-latest 7m
GHC 9.12 / ubuntu-latest 7m
GHC 9.14 / ubuntu-latest 5m
Coverage 4m

Roughly 90 job-minutes and ~14m wall clock per push, gated on the slowest macOS
leg, to validate one exposed module. That's the feedback loop on every Renovate
PR, and it becomes the loop on every automated release PR once #161 lands, so
it's worth trimming before the volume goes up.

Scope

Candidate levers, in rough order of payoff — the point is to measure each, not to
assume:

  • Trim the OS × GHC cross-product. macOS legs run 9-14m against ubuntu's
    5-7m and are the wall-clock gate. Running the full GHC set on ubuntu and only
    the matrix ends on macOS drops three legs and probably several minutes of wall
    clock. The tradeoff is losing middle-GHC coverage on Darwin; judge whether this
    package's surface can plausibly break there and not on Linux.
  • Stop rebuilding documentation ten times. cabal haddock all
    (ci.yml:99-101) runs on every leg. Haddock failures are near-always
    GHC-version-independent for this package; once on the newest GHC would likely
    do, and release.yml builds docs for real anyway.
  • Stop re-running cabal check ten times. ci.yml:95-97 validates package
    metadata, which is identical across every leg. It belongs in one job.
  • Cache the Coverage job. It restores nothing — the matrix jobs cache the
    cabal store (ci.yml:66-85) but the coverage job at ci.yml:103-148 builds
    from cold every run. It's the cheapest job at 4m, so this is minor, but it's
    free.
  • Check what the cache is actually returning. The store is keyed on the
    plan.json hash with a looser restore-keys prefix fallback. Worth confirming
    hit rates are what we think, since a mostly-missing cache would explain more of
    these numbers than any of the above.

Acceptance criteria

  • Before/after job timings recorded in this issue, from comparable runs.
  • Wall-clock time for a push to main is meaningfully reduced.
  • Coverage of the supported GHC set is unchanged on at least one OS.
  • No loss of the sdist-based build (ci.yml:42-51) — that property is the
    reason CI catches packaging bugs and is not negotiable for speed.

Additional context

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions