Skip to content

Build macOS with the Command Line Tools instead of full Xcode - #83

Merged
swhitt merged 1 commit into
ThatGuyMcd:mainfrom
actuallyitsnathaniel:macos-metal-toolchain
Nov 1, 2026
Merged

swhitt merged 1 commit into
ThatGuyMcd:mainfrom
actuallyitsnathaniel:macos-metal-toolchain

Conversation

@actuallyitsnathaniel

@actuallyitsnathaniel actuallyitsnathaniel commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

What

This removes the multi-GB full Xcode requirement for building DKR-R on macOS. As a developer I avoid
Xcode like the plague, but Apple's Command Line Tools are genuinely useful and save me a lot of local
storage. With this change, the Command Line Tools (xcode-select --install) are all a macOS build needs.

Benefits

  • No full Xcode install. Only the Command Line Tools are needed (3.2 GB on my machine).
    Homebrew already requires them, so most Mac developers have them and building DKR-R needs no
    extra Apple download.
  • No Xcode setup steps. No App Store or Apple ID download, no sudo xcode-select --switch,
    no sudo xcodebuild -license accept, no first-launch run.
  • No separate Metal Toolchain download. Setup no longer runs xcodebuild -downloadComponent MetalToolchain.
  • Fewer build steps. Each RT64 shader skips its xcrun metal and metallib steps at build time.
  • Builds on macOS 14. Works with Apple clang 16, the newest compiler available for that macOS version.
  • More reliable Setup. It installs only missing Homebrew tools, so it no longer tries to
    build tools from source that are already present (see Python below).
  • Still native. The build is arm64 on Apple Silicon, and macOS compiles the shaders for the
    machine's actual GPU.
  • iOS is unchanged. It still embeds precompiled metallibs.

How

  • RT64 (patches/rt64/0021): on macOS, RT64 embeds the MSL source that spirv-cross already
    produces, instead of compiling it with xcrun metal and metallib.
  • plume (patches/plume/0003): a shader blob starting with the MTLB magic loads as a
    precompiled library as before; anything else is compiled at runtime with MTLDevice newLibraryWithSource.
    plume already compiles its own clear and resolve shaders this way.
  • N64Recomp (patches/n64recomp/0003): instruction hooks are emitted after a null statement
    (; <hook>). Apple clang 16 rejects a declaration directly after a branch label under -std=c17;
    Clang 18+ accepts it. The hooks are deliberately not wrapped in braces: several hooks declare
    locals (dkr_free_max, dkr_model_count_before, ...) that later hooks in the same function read.
    The policy JSON is untouched, so DKR_PATCH_POLICY_SHA256 and the netplay identity don't change.
  • Setup (scripts/macos-host-tools.sh): requires only the macOS SDK from the Command Line Tools,
    and installs only missing Homebrew tools. Python counts as missing when it is older than 3.11
    (tomllib). Previously brew install python pulled Python 3.15, which has no bottle on macOS 14
    and fails to build from source with Apple clang 16.

Pattern checklist

  • First-party changes are in-place edits under runtime-recomp/, modding/ or scripts/ (no new patches/<name>/)
  • Third-party changes are *.patch files registered in patches/manifest.json (rt64/0021, plume/0003, n64recomp/0003)
  • Guest hooks only via guest-hooks/hooks.toml; symbols only via symbols/*.toml (no hooks or symbols added)
  • New settings, launcher pages and owned services are registered, and their contract docs are updated (none added)
  • netplay/legacy_identity.hpp updated if simulation, wire or owned-service behaviour changed (not needed: generated code is semantically identical)
  • Tests added or updated; python scripts/check.py --suite all passes locally (updated the host-tools stub in build_tooling_tests.py)
  • Platforms considered: built and ran on macOS 14.6.1, Apple M1, arm64, Command Line Tools 16.2 only, no Xcode. Not built on Windows, Linux, iOS or Android; the iOS shader path is unchanged.

Verification

  • python3 scripts/check.py --suite all: passes.
  • python3 scripts/apply_dependency_patches.py --check: all 15 dependencies validate.
  • ./Setup-macOS.sh then ./Build-macOS.sh in a fresh shell with no Xcode installed: the build
    completes and 166 of 168 CTest tests pass.
  • DKRFriendService and DKRProfileBridge fail with or without this change. The callbacks they need
    are installed only by patches/online-directory/native/online_directory.cpp, which no CMake target builds.
  • Booted the US 1.0 ROM: RT64 initialized on Metal, there were no shader compile errors, and the
    first frame was presented. The runtime stopped cleanly.

Risk

  • First launch: shaders compile at runtime on the first macOS launch (RT64 setup took 2.66 s on
    the M1). macOS caches them after that; setup took 58 ms on the second launch.
  • Shader errors at launch: errors now surface when the game starts instead of at build time.
    The MSL is generated by spirv-cross, and the boot test above showed no errors.
  • Not tested on Intel: the macOS x86_64 path hasn't been run locally; the macos-15-intel CI
    job will exercise it. The CI runners have full Xcode installed, so CI does not prove the no-Xcode
    path; the local M1 build above does.
  • Rollback: revert this commit. It restores the offline metallib path and the Xcode requirement.

🤖 Assited with Claude Code, reviewed by @actuallyitsnathaniel

RT64 now embeds its MSL shader source on macOS and plume compiles it at
runtime, so the build no longer needs Xcode's offline Metal toolchain.
iOS keeps precompiled metallibs. N64Recomp emits a null statement before
instruction hooks so Apple clang 16 (the newest for macOS 14) accepts a
declaring hook after a branch label under C17. Setup installs only
missing Homebrew tools, avoiding source builds of tools already present.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@swhitt swhitt left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for helping, this looks good! Will merge later today. Would benefit from a couple of tweaks but I can add these in a different PR since they are so minor. ❤️

Comment on lines +17 to +19
+ # macOS embeds the MSL source; plume compiles it at runtime, so the
+ # build needs no Xcode Metal toolchain.
+ set(_dkr_msl_blob ${OUTNAME}.metal)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RasterShader calls createShader with these blobs for every new combiner/other-mode combo (src/render/rt64_raster_shader.cpp:166-167), so I think every specialized raster shader now recompiles the whole ubershader MSL, not just once at setup. The 2.66 s number only covers setup. We should watch for pop-in during gameplay on a cold cache.

Maybe keep the metallib when xcrun -sdk macosx -f metal resolves at configure time and only embed source when it doesn't? The plume patch already handles both, and CI/release builds keep the old load path and build-time shader errors.

Comment on lines +17 to +18
# Install only what is missing: on macOS releases Homebrew no longer
# bottles, reinstalling a present tool (python) means a failing source build.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

brew install on an already-installed formula is a no-op with HOMEBREW_NO_INSTALL_UPGRADE=1, so the failing case is really brew python missing while some other 3.11+ is on PATH. A fresh macOS 14 box with only the CLT python (3.9) still hits brew install python here. Maybe reword the comment so it doesn't read like it fixes that?

@swhitt
swhitt merged commit d2b5f4f into ThatGuyMcd:main Nov 1, 2026
12 checks passed
@actuallyitsnathaniel

Copy link
Copy Markdown
Contributor Author

WOW! I'm stoked this made the cut! Just caught up on all this. The revisions are great, definitely cracks in the ground on my end. The following PR was insightful as well. Definitely will look deeper across the codebase and see what else I can do. No joke this is the first video game I ever played growing up, so I'm jazzed about this recomp project 🙌🏻 Thanks for letting me contribute!

@actuallyitsnathaniel
actuallyitsnathaniel deleted the macos-metal-toolchain branch November 2, 2026 19:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants