tests: clean up out-of-repo path in deform_conv2d parity test #36

Workflow file for this run

	name: CI

	on:
	push:
	branches: [main]
	pull_request:
	workflow_dispatch:

	# A new push to the same ref supersedes the previous run.
	concurrency:
	group: ci-${{ github.ref }}
	cancel-in-progress: true

	# Build-tool notes that apply to every job below:
	#
	# * Ninja, not Make. CMake's default generator on Linux/macOS is Unix
	# Makefiles, and `cmake --build --parallel` with no job count passes a bare
	# `-j` to make — which GNU make reads as unlimited parallelism. Enabling a
	# GPU backend flips BROTENSOR_HAS_GPU in tests/CMakeLists.txt and pulls in
	# the full GPU test tier (158 executables, up from 35), so an unbounded make
	# forks ~160 compiles onto a 3-core / 7 GB hosted runner and the box spends
	# its time swapping instead of compiling. Ninja bounds concurrency by
	# default; the job counts below are pinned anyway.
	#
	# * ccache on the two expensive jobs (Metal: ~80 Objective-C++ TUs; CUDA: ~90
	# .cu TUs). Cold runs are unaffected; a push that doesn't touch a backend
	# gets those TUs back for free.
	jobs:
	# ── CPU tier ──────────────────────────────────────────────────────────────
	#
	# The CPU backend is always built and has no opt-out, so this is the tier
	# every consumer gets. It is also the configuration most likely to rot
	# unnoticed: day-to-day work happens in a CUDA-enabled build dir, where the
	# 35 CPU-only tests are drowned out by the GPU suite and a CPU-only compile
	# break never surfaces.
	#
	# macOS here is deliberately without Metal — it is the arm64 / AppleClang
	# build check for the portable core (the SIMD baseline is x86-gated).
	cpu:
	name: CPU · ${{ matrix.name }}
	runs-on: ${{ matrix.os }}
	strategy:
	fail-fast: false
	matrix:
	include:
	- { name: linux-gcc, os: ubuntu-24.04, cc: gcc, cxx: g++ }
	- { name: linux-clang, os: ubuntu-24.04, cc: clang, cxx: clang++ }
	- { name: windows-msvc, os: windows-2022 }
	- { name: macos-arm64, os: macos-14 }
	steps:
	- uses: actions/checkout@v7

	- name: Install Ninja (Linux)
	if: runner.os == 'Linux'
	run: sudo apt-get update -qq && sudo apt-get install -y --no-install-recommends ninja-build

	- name: Install Ninja (macOS)
	if: runner.os == 'macOS'
	run: brew install ninja

	# Windows keeps the Visual Studio generator: MSBuild's /m is already
	# bounded, and the CPU tier is only 35 test targets there.
	- name: Configure
	env:
	CC: ${{ matrix.cc }}
	CXX: ${{ matrix.cxx }}
	run: \|
	if [ "$RUNNER_OS" = "Windows" ]; then
	cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
	else
	cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release
	fi
	shell: bash

	- name: Build
	run: cmake --build build --config Release --parallel 4

	- name: Test
	run: ctest --test-dir build -C Release --output-on-failure

	# ── Metal ─────────────────────────────────────────────────────────────────
	#
	# Builds the ~80 .mm backend TUs. This is the only Apple toolchain in the
	# loop — no development machine here runs one, so without this job a Metal
	# change lands on inspection alone.
	#
	# ctest runs too: the GPU suite guards on is_available() and skips cleanly
	# when the runner exposes no Metal device, so this is a build check at
	# minimum and a real parity run when the runner has a GPU. Either way a
	# genuine Metal parity failure fails the job rather than being silently
	# skipped.
	#
	# Runner must be macos-15, not macos-14: src/metal/runtime.mm builds an
	# offset-backed MPSGraphTensorData via -[MPSNDArray initWithBuffer:offset:
	# descriptor:], which is a macOS 15 API. Against the macOS 14 SDK that
	# selector is not declared and the TU does not compile. The CPU job above
	# stays on macos-14 (it never touches MPS), so the older OS keeps its arm64
	# coverage for the portable core.
	#
	# This is the heaviest job: the Metal backend and the full 158-executable
	# test tier, on a small runner (3 vCPU / 7 GB). Hence Ninja + a pinned job
	# count + ccache.
	metal:
	name: Metal · macos-15
	runs-on: macos-15
	steps:
	- uses: actions/checkout@v7

	- name: Install Ninja
	run: brew install ninja

	- name: ccache
	uses: hendrikmuhs/ccache-action@v1.2
	with:
	key: metal-macos-15
	max-size: 500M

	- name: Configure
	run: \|
	cmake -S . -B build -G Ninja \
	-DCMAKE_BUILD_TYPE=Release \
	-DBROTENSOR_WITH_METAL=ON \
	-DCMAKE_CXX_COMPILER_LAUNCHER=ccache \
	-DCMAKE_OBJCXX_COMPILER_LAUNCHER=ccache

	- name: Build
	run: cmake --build build --parallel 3

	- name: Test
	run: ctest --test-dir build --output-on-failure

	# ── CUDA ──────────────────────────────────────────────────────────────────
	#
	# Compile-only: GitHub's hosted runners have no NVIDIA GPU, so nothing can be
	# executed here. That still buys the thing that actually breaks — nvcc
	# accepting all ~90 .cu TUs, and the device link (CUDA_RESOLVE_DEVICE_SYMBOLS)
	# resolving. The parity runs stay on the local box / a self-hosted runner.
	#
	# CMAKE_CUDA_ARCHITECTURES must be pinned: the project defaults to `native`,
	# which cannot be probed without a GPU present.
	cuda:
	name: CUDA build · ubuntu-24.04
	runs-on: ubuntu-24.04
	steps:
	- uses: actions/checkout@v7

	- name: Install Ninja
	run: sudo apt-get update -qq && sudo apt-get install -y --no-install-recommends ninja-build

	- name: Install CUDA toolkit (nvcc + cudart)
	run: \|
	wget -q https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb
	sudo dpkg -i cuda-keyring_1.1-1_all.deb
	sudo apt-get update -qq
	sudo apt-get install -y --no-install-recommends cuda-nvcc-12-6 cuda-cudart-dev-12-6
	echo "/usr/local/cuda-12.6/bin" >> "$GITHUB_PATH"

	- name: ccache
	uses: hendrikmuhs/ccache-action@v1.2
	with:
	key: cuda-ubuntu-24.04
	max-size: 500M

	- name: Configure
	run: \|
	cmake -S . -B build -G Ninja \
	-DCMAKE_BUILD_TYPE=Release \
	-DBROTENSOR_WITH_CUDA=ON \
	-DCMAKE_CUDA_ARCHITECTURES=89 \
	-DCMAKE_CXX_COMPILER_LAUNCHER=ccache \
	-DCMAKE_CUDA_COMPILER_LAUNCHER=ccache

	- name: Build
	run: cmake --build build --parallel 4

	# ── Coverage ──────────────────────────────────────────────────────────────
	#
	# Runs on the macOS runner with Metal on, not on a GPU-less Linux box, and
	# that choice is the whole point of this job.
	#
	# Roughly 110 of the 158 tests are the CPU<->GPU parity suite, and every one
	# of them computes its reference by calling the op on the CPU backend:
	#
	# brotensor::foo(X, cpu_Y, ...); // <- CPU backend, CPU path
	# brotensor::foo(X.to(gpu), gpu_Y, ...); // <- GPU backend
	# compare_tensors(cpu_Y, gpu_Y);
	#
	# So the parity suite is the single biggest exerciser of src/cpu/ — and it is
	# gated behind BROTENSOR_HAS_GPU, meaning it does not even build without a
	# GPU backend, let alone run. Measuring coverage on a GPU-less runner
	# therefore reports the CPU backend with two-thirds of the tests that exercise
	# it structurally excluded, understating it by tens of points. The hosted
	# macos-15 runner has a real Metal device, so here the full suite runs and the
	# number reflects what the tests actually reach.
	#
	# What is measured: the core (dispatcher, Tensor, safetensors/GGUF readers)
	# plus the FP32 CPU reference backend. BROTENSOR_COVERAGE scopes --coverage to
	# COMPILE_LANGUAGE:CXX, so src/metal's Objective-C++ TUs are not instrumented;
	# they are excluded from the report rather than counted as 0%, which would
	# make the headline number meaningless.
	#
	# No ccache: it interacts badly with --coverage (the .gcno sidecars are
	# compile-time artifacts, and a cache hit skips emitting them).
	#
	# This job reports; it does not gate. gcovr's --fail-under-line would turn a
	# coverage dip into a red build, which is a policy choice worth making only
	# against a threshold the suite has held for a while.
	coverage:
	name: Coverage · CPU (via Metal parity suite)
	runs-on: macos-15
	steps:
	- uses: actions/checkout@v7

	- name: Install gcovr + Ninja
	run: brew install gcovr ninja

	- name: Configure (instrumented)
	run: \|
	cmake -S . -B build -G Ninja \
	-DCMAKE_BUILD_TYPE=Debug \
	-DBROTENSOR_COVERAGE=ON \
	-DBROTENSOR_WITH_METAL=ON

	- name: Build
	run: cmake --build build --parallel 3

	- name: Test
	run: ctest --test-dir build --output-on-failure

	# pipefail matters here: piping gcovr into tee otherwise hands the step
	# tee's exit status, so a gcovr crash produces no report and the job still
	# goes green — a coverage step that silently reports nothing is worse than
	# no coverage step at all.
	#
	# --gcov-executable: AppleClang emits gcov-format .gcno/.gcda, but the
	# `gcov` on PATH is a shim that cannot read them. `llvm-cov gcov` is the
	# matching reader.
	#
	# --gcov-ignore-parse-errors: gcov (both GCC's and llvm-cov's) can emit
	# negative branch hit counts — a long-standing bug, gcc bugzilla 68080 —
	# which gcovr treats as fatal. Downgrade to a per-file warning: line counts
	# are unaffected, only a few branch counts in the offending files are.
	- name: Report
	shell: bash
	run: \|
	set -o pipefail
	mkdir -p coverage-html
	gcovr --root . \
	--gcov-executable 'xcrun llvm-cov gcov' \
	--filter 'src/' --filter 'include/brotensor/' \
	--exclude 'src/cuda/' --exclude 'src/metal/' \
	--exclude 'tests/' \
	--exclude-unreachable-branches \
	--gcov-ignore-parse-errors negative_hits.warn_once_per_file \
	--print-summary \
	--cobertura coverage.xml --cobertura-pretty \
	--html-details coverage-html/index.html \
	\| tee coverage-summary.txt
	# The report must actually exist — guard against a silent no-op.
	test -s coverage.xml

	- name: Job summary
	if: always()
	run: \|
	{
	echo '## Coverage — core + CPU backend'
	echo
	echo 'Measured on the macOS runner with Metal enabled, so the CPU↔GPU'
	echo 'parity suite runs — it is the biggest exerciser of `src/cpu/`, since'
	echo 'every parity test calls the CPU op as its reference, and it does not'
	echo 'build at all without a GPU backend.'
	echo
	echo 'Scope: `src/` + `include/brotensor/`. The CUDA and Metal backends are'
	echo 'compiled by nvcc / the Apple toolchain, are not gcov-instrumented, and'
	echo 'are excluded rather than counted as 0%.'
	echo
	echo '```'
	cat coverage-summary.txt
	echo '```'
	} >> "$GITHUB_STEP_SUMMARY"

	# Coverage lives entirely inside GitHub: the headline numbers land in the
	# run's job summary (above) and the full line-by-line drill-down is this
	# artifact. There is no third-party coverage service in the loop, by
	# choice — it would add an external account and an upload token to the
	# critical path of a build, in exchange for a badge.
	- name: Upload HTML report
	if: always()
	uses: actions/upload-artifact@v7
	with:
	name: coverage-html
	path: coverage-html/
	retention-days: 14

	# ── Pages ─────────────────────────────────────────────────────────────────
	#
	# Publishes the docs and the coverage report to
	# https://wlejon.github.io/brotensor/.
	#
	# The coverage report is the reason this job exists. gcovr's --html-details
	# output is the only view of which lines the suite misses, and as a build
	# artifact it expired after 14 days — so the one artifact worth keeping was
	# the one guaranteed to be gone by the time anyone wanted it. Here it gets a
	# stable URL, rebuilt on every push to main.
	#
	# It reuses the artifact the coverage job already produced rather than
	# re-running an instrumented build: that build is a 4-minute macOS job, and
	# doing it twice per push to publish the same bytes would be silly.
	#
	# The docs come along because a coverage report with no docs around it is a
	# dead end, and because op-coverage.md is the table the sibling projects
	# actually want to link into.
	pages:
	name: Publish docs + coverage
	needs: coverage
	# Deploy only from main. A PR's build is not the published site, and the
	# deploy needs write scopes a fork PR must never get.
	if: github.event_name == 'push' && github.ref == 'refs/heads/main'
	runs-on: ubuntu-24.04
	permissions:
	pages: write # deploy
	id-token: write # OIDC token deploy-pages exchanges for the upload
	contents: read
	environment:
	name: github-pages
	url: ${{ steps.deploy.outputs.page_url }}
	# Pages accepts one deployment at a time. Queue rather than cancel: a
	# cancelled deploy leaves the previous site up, which is fine, but a
	# cancelled upload has been known to wedge the environment.
	concurrency:
	group: pages
	cancel-in-progress: false
	steps:
	- uses: actions/checkout@v7

	- uses: actions/setup-python@v7
	with:
	python-version: "3.x"

	- name: Render markdown -> site/
	run: \|
	python -m pip install --quiet markdown
	python .github/scripts/build_site.py site

	# Into site/coverage/, which is where build_site.py's nav link points.
	- name: Add the coverage report
	uses: actions/download-artifact@v8
	with:
	name: coverage-html
	path: site/coverage

	- uses: actions/configure-pages@v6

	- uses: actions/upload-pages-artifact@v5
	with:
	path: site

	- name: Deploy
	id: deploy
	uses: actions/deploy-pages@v5

Navigation Menu

tests: clean up out-of-repo path in deform_conv2d parity test #36

Workflow file

tests: clean up out-of-repo path in deform_conv2d parity test #36

Uh oh!

Workflow file for this run