Skip to content

ci: skip the tutorial rebuild on unrelated PRs, and unfreeze its ccache - #3811

Open
nbbrooks wants to merge 1 commit into
mainfrom
nbbrooks/speed-up-tutorial-job
Open

ci: skip the tutorial rebuild on unrelated PRs, and unfreeze its ccache#3811
nbbrooks wants to merge 1 commit into
mainfrom
nbbrooks/speed-up-tutorial-job

Conversation

@nbbrooks

@nbbrooks nbbrooks commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

Description

tutorial-source is the long pole on every moveit2 PR. Measured across eight consecutive runs:

43m33s  53m32s  56m41s  54m22s  60m58s  51m10s  54m01s  57m40s

It ran the full ~55 min on #3809 — a PR touching four launch .test.py files.

Two independent causes, both small fixes.


1. It runs on every pull_request with no path filter

The job rebuilds moveit2 + moveit2_tutorials + all upstream deps from source. Changes to test fixtures, markdown, or other workflows can't affect the tutorial image, but still trigger a full rebuild.

Uses paths with ! rather than paths-ignorenegation is only supported in paths, and the two filters can't be combined for one event:

You cannot use paths and paths-ignore to filter the same event in a single workflow. If you want to both include and exclude path patterns for a single event, use the paths filter prefixed with the ! character.

The filter still runs the job when this workflow or .docker/tutorial-source/** changes.

Verified safe to skip. tutorial-source (jazzy) is not a required status check on main — the required set is Format, humble-ci, jazzy-ci, rolling-ci + ikfast + clang-tidy (delta), rolling-ci + ccov. And .github/mergify.yml contains only label-driven backport rules with no check-success conditions. So a skipped run can't block a merge, and no no-op fallback job is needed.

2. The ccache was frozen after its first save

key: docker-tutorial-ccache-${{ matrix.ROS_DISTRO }}-${{ hashFiles( '.docker/tutorial-source/Dockerfile' ) }}

A static key with no restore-keys. actions/cache skips its save step on an exact-key hit, so the entry is written once and never refreshed — every build after the first restores a stale ccache and recompiles whatever changed since, indefinitely. The key only rotates when the Dockerfile changes, which is rare.

Switched to the rolling-key pattern used elsewhere in this repo: unique per run, falling back to the stable prefixes, so each run restores the newest entry and saves an updated one.


Not addressed here

Worth follow-up, but larger and Dockerfile-structural:

  • COPY . src/moveit2 sits above the expensive RUN, with .dockerignore deliberately removed ("enforce full source context") — so any source change busts the layer and forces a full workspace rebuild.
  • Gazebo install, the tutorials clone, vcs import, rosdep install and colcon build share one monolithic RUN, so the slow-and-stable parts can never cache separately from the fast-changing build.
  • The moveit2_tutorials clone is not shallow (--depth 1).

Checklist

  • Required by CI: Code is auto formatted — CI config only
  • Extend the tutorials / documentation — not applicable
  • Document API changes in MIGRATION.md — not applicable
  • Create tests, which fail without this PR — not applicable
  • Include a screenshot if changing a GUI — not applicable

🤖 Generated with Claude Code

tutorial-source is the long pole on every moveit2 PR. Measured across eight
consecutive runs: 43m33s, 53m32s, 56m41s, 54m22s, 60m58s, 51m10s, 54m01s,
57m40s. It ran the full ~55 min on #3809, a PR touching four launch
.test.py files.

Two independent causes.

1. It runs on every pull_request with no path filter

The job rebuilds moveit2 + moveit2_tutorials + all upstream deps from
source. Changes to test fixtures, markdown, or other workflows cannot
affect the tutorial image, but still trigger a full rebuild. Add a path
filter so those PRs skip it.

Uses `paths` with `!` rather than `paths-ignore`: negation is only
supported in `paths`, and the two filters cannot be combined for a single
event. The filter still runs the job when this workflow or the
tutorial-source Dockerfile itself changes.

Safe to skip -- verified tutorial-source (jazzy) is not a required status
check on main (the required set is Format, humble-ci, jazzy-ci,
rolling-ci + ikfast + clang-tidy (delta), rolling-ci + ccov), and
.github/mergify.yml contains only label-driven backport rules with no
check-success conditions. So a skipped run cannot block a merge and no
no-op fallback job is needed.

2. The ccache was frozen after its first save

  key: docker-tutorial-ccache-${{ matrix.ROS_DISTRO }}-${{ hashFiles(...) }}

A static key with no restore-keys. actions/cache skips its save step on an
exact-key hit, so the entry is written once and never refreshed -- every
build after the first restores a stale ccache and recompiles whatever
changed since, indefinitely. The key only rotates when the Dockerfile
changes, which is rare.

Switch to the rolling-key pattern used elsewhere in this repo: make the key
unique per run and fall back to the stable prefixes, so each run restores
the newest entry and saves an updated one.

Not addressed here, but worth follow-up: the Dockerfile does COPY .
src/moveit2 above the expensive RUN (with .dockerignore deliberately
removed), so any source change busts the layer; gazebo install, the
tutorials clone, vcs import, rosdep and colcon build share one monolithic
RUN; and the moveit2_tutorials clone is not shallow.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@codecov

codecov Bot commented Aug 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 46.24%. Comparing base (40f3140) to head (eff671a).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3811      +/-   ##
==========================================
- Coverage   46.28%   46.24%   -0.04%     
==========================================
  Files         726      726              
  Lines       59504    59509       +5     
  Branches     7624     7623       -1     
==========================================
- Hits        27536    27512      -24     
- Misses      31801    31830      +29     
  Partials      167      167              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@mergify

mergify Bot commented Aug 14, 2026

Copy link
Copy Markdown

This pull request is in conflict. Could you fix it @nbbrooks?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant