Skip to content

Fix and simplify CI caching - #634

Merged
julianspeith merged 3 commits into
masterfrom
fix/ci-workflow-caching
Aug 11, 2026
Merged

Fix and simplify CI caching#634
julianspeith merged 3 commits into
masterfrom
fix/ci-workflow-caching

Conversation

@julianspeith

@julianspeith julianspeith commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

The reported failure

The Ubuntu 22.04 job on fix/documentation died at the Cache pip Linux step:

The template is not valid. .github/workflows/ubuntu22.04.yml (Line: 34, Col: 16):
hashFiles('**/requirements.txt') failed. Fail to hash files under directory '/home/runner/work/hal/hal'

hashFiles returns an error when the pattern resolves to zero files, and the job stops there — before Install Dependencies, Configure, Build or Test. requirements.txt is present on that branch at the same three paths as on master, the workflow files are byte-identical between the branches, and the trees differ only by six new .rst files and two .DS_Store files, so the branch content does not explain it. What does stand out is that the pattern makes the runner walk the entire ~4500 file workspace, deps/abc included, to find a file whose path is known.

This PR hashes the file directly, hashFiles('requirements.txt'), in all six live occurrences. Same cache key semantics, no workspace traversal.

A real caching bug found along the way

The three Linux workflows key their ccache on ${{ steps.ccache_cache_timestamp.outputs.timestamp }}, but only macOS.yml defines a Prepare ccache timestamp step. On Linux the key therefore evaluated to a constant, which the run logs confirm:

Build and Test on Ubuntu 22.04    ccache cache files
Cache not found for input keys: Linux-ccache-, Linux-ccache-

The primary key and the restore key are the same string. actions/cache skips saving when the primary key hits exactly, so once any successful run writes Linux-ccache-, every later run restores that same snapshot and never saves an updated one — the compiler cache freezes at its first contents and stops helping as the tree moves on.

Fixed by adding the missing timestamp step to all three Linux workflows, so the key is unique per run and the Linux-ccache- restore-key prefix picks up the most recent entry.

Other cleanups

  • macOS.yml generated its timestamp with ::set-output, which GitHub deprecated and warns about on every run. It now writes to $GITHUB_OUTPUT. Both platforms use the same snippet and key format.
  • Removed the Cache CCache step from the Linux workflows: it cached ~/.ccache while CCACHE_DIR points at ${{runner.workspace}}/.ccache, so it archived an empty directory, and its restore keys referenced an undefined env.cache-name.
  • Removed the Cache pip macOS step from the Linux workflows, where startsWith(runner.os, 'macOS') can never be true.
  • Added a concurrency group so a new push to a branch cancels the superseded run. Runs on master are exempt so the documentation deployment always completes.

All four workflows parse and keep their step order; only dead steps were removed.

Workflow modernization (second commit)

  • actions/checkout@v1 (macOS) and @v2 (the rest) replaced with @v4. Both were deprecated and forced a Node 24 shim on every run. Since @v4 accepts fetch-depth, the three hand-written git fetch steps in the Linux workflows are gone; fetch-depth: 0 supplies the full history and tags that cmake/hal_cmake_tools.cmake needs for git describe --tags. That also removes git fetch --prune --unshallow, which is what failed run 31498874580 on master with fatal: shallow file has changed since we read it. macOS previously used @v1, which cloned full history implicitly, so fetch-depth: 0 preserves its behavior.
  • push restricted to master, pull_request added. Previously every push to every branch started four workflows of roughly 50 minutes each, including a full documentation build on branches that can never deploy. Changes now get exactly one run per push via their pull request, and the Deploy Doc step stays gated on github.ref == 'refs/heads/master', which no pull_request event satisfies.
  • -DPYBIND11_PYTHON_VERSION=3.6 dropped from the three Linux configure steps. That Python is long end-of-life, is not installed on any supported runner, and the variable is ignored by the pybind11 version HAL requires. It appears nowhere else in the tree.
  • Single-entry build matrix replaced with a plain runs-on, and the resulting always-true startsWith(runner.os, ...) step conditions removed (2 in each Ubuntu workflow, 4 in releaseDoc.yml, 4 in macOS.yml).

Check names change

Dropping the matrix shortens the reported check names:

before after
Build and Test on Ubuntu 22.04 (ubuntu-22.04) Build and Test on Ubuntu 22.04
Build and Test on Ubuntu 24.04 (ubuntu-24.04) Build and Test on Ubuntu 24.04
Build and Test macOS (macOS-latest) Build and Test macOS
Build and release documentation (ubuntu-22.04) Build and release documentation

Worth knowing while reviewing this: master's branch protection currently lists exactly one required status check, Build (macOS-latest), which no workflow in the repository produces — the macOS job is named Build and Test macOS. That required check can never report, so it is already stale today and this rename does not make it worse. Someone with admin rights may want to point it at the real check names above.

🤖 Generated with Claude Code

The pip cache key hashes a glob, `hashFiles('**/requirements.txt')`, which walks
the whole ~4500 file workspace including deps/abc to locate a file whose path is
known. That traversal returned no matches on one branch and failed the job
before any build step ran. Hash the file directly instead.

The ccache key of the three Linux workflows referenced
`steps.ccache_cache_timestamp.outputs.timestamp`, but only macOS.yml defines
that step. The key therefore evaluated to a constant `Linux-ccache-`, identical
to its own restore key, which is visible in the run logs:

    Cache not found for input keys: Linux-ccache-, Linux-ccache-

With a constant key, the first run to succeed saves the cache and every later
run gets an exact hit and skips saving, so the compiler cache freezes at its
first snapshot and never picks up new objects. Add the missing timestamp step so
the key is unique per run and the restore key prefix matches the newest entry.

Also:

* macOS.yml produced that timestamp via `::set-output`, which GitHub deprecated
  and warns about on every run. Write to `$GITHUB_OUTPUT` instead. Both
  platforms now use the same shell snippet and the same key format.
* Drop the `Cache CCache` step of the Linux workflows. It cached `~/.ccache`
  while `CCACHE_DIR` points at `${{runner.workspace}}/.ccache`, so it archived an
  empty directory, and its restore keys referenced an undefined
  `env.cache-name`.
* Drop the `Cache pip macOS` step from the Linux workflows, where its
  `startsWith(runner.os, 'macOS')` condition can never be true.
* Add a concurrency group so that pushing again to a branch cancels the
  superseded run. Runs on master are exempt so the documentation deployment
  always completes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
julianspeith and others added 2 commits August 11, 2026 17:27
Follow-up cleanups to the workflow files:

* Replace the deprecated actions/checkout@v1 (macOS) and @v2 (the rest) with @v4.
  Both force a Node 24 shim on every run. Since @v4 takes fetch-depth, the three
  hand-written 'git fetch' steps of the Linux workflows are no longer needed;
  'fetch-depth: 0' provides the full history and tags that
  cmake/hal_cmake_tools.cmake needs for 'git describe --tags'. This also removes
  'git fetch --prune --unshallow', which failed run 31498874580 on master with
  "fatal: shallow file has changed since we read it".
* Restrict the push trigger to master and add a pull_request trigger. Every push
  to any branch previously started four workflows of roughly 50 minutes each,
  including a full documentation build on branches that never deploy. Changes now
  get exactly one run per push through their pull request.
* Drop '-DPYBIND11_PYTHON_VERSION=3.6' from the three Linux configure steps. That
  Python version is long end-of-life, is not present on any supported runner, and
  the variable is ignored by the pybind11 version HAL requires.
* Replace the single-entry build matrix with a plain 'runs-on' and drop the
  resulting always-true 'startsWith(runner.os, ...)' step conditions.

Note that dropping the matrix shortens the reported check names, for example
"Build and Test macOS (macOS-latest)" becomes "Build and Test macOS".

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@julianspeith
julianspeith merged commit c398a10 into master Aug 11, 2026
4 checks passed
@julianspeith
julianspeith deleted the fix/ci-workflow-caching branch August 11, 2026 16:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant