Repository navigation
Pin the MPS test leg to macos-15: torch 2.14's attention kernel cannot compile on the macos-latest virtual GPU - #1241
Merged
Merged
Conversation
No code change: this commit exists only so the pull_request workflow runs on
main's tree. The "Test Package Compatibility (macos-latest, 3.14, highest)"
job resolves the newest torch, and since torch 2.14.0 was released it fails
with Metal compiler errors ("unsupported deferred-static-alloca-size") while
building MPS kernels, on the same macos-26-arm64 image where torch 2.13.0
passed hours earlier. If this job fails here too, the breakage is torch 2.14.0
on that runner, independent of any open branch.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Runs a dozen small MPS operations with torch 2.14.0 on macos-latest and on macos-15, to learn whether a cheap probe detects the Metal 4 cooperative-tensor compile failure on the virtual GPU, and which operations trigger it. Not meant to merge. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
…a mapping Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…dims and lengths Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…t compile torch's attention kernel
GitHub's macos-latest image (macos-26-arm64) runs on a virtual GPU. Since torch
2.14.0, the fp16/bf16 scaled_dot_product_attention kernel torch uses on macOS 26
fails to compile there ("unsupported deferred-static-alloca-size" from the Metal
compiler), which killed the 50 tests that run the model on MPS. Physical Macs
on macOS 26 compile the same kernel, so nothing in TabPFN or torch is wrong for
users; only this runner cannot run the tests.
The macOS "highest" leg now tries that exact attention call once before the
tests. If it compiles, the leg runs the tests as before. If it fails with the
Metal compile error, the leg emits a warning, skips its tests and stays green,
and a new job runs the same tests on macos-15, whose macOS 15 torch does not
use the new kernel. Any other failure of the check fails the leg. The fallback
job is skipped whenever macos-latest can compile the kernel again, so this
reverts itself when GitHub or torch fix the combination. No torch cap anywhere.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
It answered its question: a synthetic fp16 attention call with head dim >= 64 reproduces the compile failure on macos-latest and passes on macos-15. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
alanprior
changed the title
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit ca17af2. Configure here.
torch 2.14 uses a Metal 4 fp16/bf16 attention kernel on macOS 26 that the macos-latest runner's virtual GPU cannot compile, so the only CI leg that runs TabPFN on MPS failed 50 tests. Physical Macs are unaffected. Run that leg on macos-15 for now, as done in June, and drop the probe-and-fallback job added earlier in this PR. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
alanprior
changed the title
alanprior
enabled auto-merge
September 4, 2026 23:58
alanprior
disabled auto-merge
September 5, 2026 16:39
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Since torch 2.14.0, the only CI job that runs TabPFN on MPS,
Test Package Compatibility (macos-latest, 3.14, highest), fails 50 tests with a Metal compiler error (unsupported deferred-static-alloca-size). GitHub'smacos-latestimage (macOS 26) runs on a virtual GPU that cannot compile the fp16/bf16scaled_dot_product_attentionkernel torch >= 2.14 uses on macOS 26. Physical Macs on macOS 26 compile it fine (verified on an M5 / macOS 26.3 with the same torch and tests; kornia reports the same for an M1), so nothing is wrong for users and torch is not capped.Change
3.14 / highestmacOS leg runs onmacos-15, where torch does not use that kernel. Same fix as Pin macos-15 runner, as tests fail on macos-26. #1073 in June, with a TODO to retrymacos-latestaround November.Needs an admin
The required check is registered by name. Please replace
Test Package Compatibility (macos-latest, 3.14, highest)withTest Package Compatibility (macos-15, 3.14, highest)in the "Restrict Main Branch" ruleset, otherwise this and every later PR waits forever on the old name.Evidence
macos-latest, and all 93 variants pass onmacos-15.