macOS (Apple Silicon) support on the WebGPU/Metal renderer (#715 rebased on main, with fixes)

.github/workflows/macos-platform.yml

@@ -0,0 +1,141 @@
+name: macOS platform
+
+on:
+  push:
+    tags: ['v*']
+  pull_request:
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+jobs:
+  macos-posix:
+    # Apple Silicon runner: verify the native arm64 game and runtime target.
+    runs-on: macos-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Build configuration test
+        run: |
+          python -I -B tests/test_macos_build_path.py
+          python -I third_party/a64-kit/lift/build_emit.py
+          python -I scripts/macos/check-arm64-translation.py
+
+      - name: Link native arm64 macOS executable
+        run: |
+          printf 'int main(void) { return 0; }\n' > "$RUNNER_TEMP/macos-probe.c"
+          clang -target arm64-apple-macos13.3 -mcpu=apple-m1 "$RUNNER_TEMP/macos-probe.c" -o "$RUNNER_TEMP/macos-probe"
+          lipo "$RUNNER_TEMP/macos-probe" -verify_arch arm64
+          printf 'int main(void) { unsigned __int128 v = 0, e = 0; return __atomic_compare_exchange_n(&v, &e, 1, 0, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST); }\n' \
+            > "$RUNNER_TEMP/arm64-atomic-probe.c"
+          clang -target arm64-apple-macos13.3 -mcpu=apple-m1 "$RUNNER_TEMP/arm64-atomic-probe.c" \
+            -o "$RUNNER_TEMP/arm64-atomic-probe"
+
+      - name: Compile macOS POSIX host shims
+        run: |
+          mkdir -p "$RUNNER_TEMP/a64-generated"
+          printf '/* Generated lift helpers are unused by these host translation units. */\n' \
+            > "$RUNNER_TEMP/a64-generated/a64_suyu.h"
+          printf '/* Generated per-game floating-point helpers are unused by these host translation units. */\n' \
+            > "$RUNNER_TEMP/a64-generated/a64_fpx.h"
+          test "$(uname -m)" = arm64
+          python -c "import platform; assert platform.machine() == 'arm64', platform.machine()"
+          clang -target arm64-apple-macos13.3 -mcpu=apple-m1 -fsyntax-only -D_GNU_SOURCE -Wno-everything \
+            -I"$RUNNER_TEMP/a64-generated" -Ihost/posix -Ihost -Ithird_party/a64-kit/rt \
+            host/posix/win32_posix.c host/hos_main.c host/rx_backend.c host/hos_setjmp.c
+          clang -target arm64-apple-macos13.3 -mcpu=apple-m1 -fsyntax-only -D_GNU_SOURCE -Wno-everything \
+            -I"$RUNNER_TEMP/a64-generated" -Ihost/posix -Ihost -Ithird_party/a64-kit/rt \
+            host/posix/display_sdl.c host/posix/macos_stubs.c
+
+  macos-builder:
+    # Builds a native arm64 downloadable CLI and dependencies; it never receives or builds game data.
+    runs-on: macos-latest
+    timeout-minutes: 150
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Verify Apple Silicon runner
+        run: |
+          test "$(uname -m)" = arm64
+          python -c "import platform; assert platform.machine() == 'arm64', platform.machine()"
+
+      - uses: actions/cache@v4
+        with:
+          path: build-macos/deps
+          key: macos-builder-deps-${{ hashFiles('scripts/macos/deps.sh') }}
+
+      - name: Install dependency build tools
+        run: brew install cmake nasm
+
+      - name: Build builder dependencies
+        run: bash scripts/macos/deps.sh
+
+      - name: Validate bundled builder toolchain
+        env:
+          WGPU_NATIVE: ${{ github.workspace }}/build-macos/deps/wgpu-native
+          HOOKSHOT_SDL2: ${{ github.workspace }}/build-macos/deps/sdl2/lib/libSDL2-2.0.0.dylib
+          HOOKSHOT_FFMPEG: ${{ github.workspace }}/build-macos/deps/ffmpeg
+        run: |
+          build-macos/deps/python/bin/python3 -I -B tests/test_macos_build_path.py
+          build-macos/deps/python/bin/python3 -I -c "import tkinter"
+          build-macos/deps/python/bin/python3 -I -c \
+            "import platform; assert platform.machine() == 'arm64', platform.machine()"
+          build-macos/deps/python/bin/python3 -I -c \
+            "import runpy; runpy.run_path('nx2/make_game.py')['check_macos_toolchain']()"
+          python -I scripts/macos/check-renderer.py
+          lipo build-macos/deps/python/bin/python3 -verify_arch arm64
+          lipo build-macos/deps/sdl2/lib/libSDL2-2.0.0.dylib -verify_arch arm64
+          lipo build-macos/deps/wgpu-native/lib/libwgpu_native.dylib -verify_arch arm64
+          for lib in build-macos/deps/ffmpeg/lib/*.a; do lipo "$lib" -verify_arch arm64; done
+
+      - name: Package CLI builder
+        run: bash scripts/macos/package-builder.sh
+
+      - name: Verify packaged builder starts
+        run: |
+          test -d build-macos/package/HookshotBuilder/HookshotBuilder.app
+          test -x build-macos/package/HookshotBuilder/HookshotBuilder.app/Contents/Resources/tools/python/bin/python3
+          test -f build-macos/package/HookshotBuilder/HookshotBuilder.app/Contents/Resources/tools/source/builder/linux/hookshot_builder.py
+          test ! -L build-macos/package/HookshotBuilder/HookshotBuilder.app/Contents/Resources/tools
+          lipo build-macos/package/HookshotBuilder/HookshotBuilder.app/Contents/MacOS/applet -verify_arch arm64
+          cd build-macos/package/HookshotBuilder
+          ./HookshotBuilder.command --help
+
+      - uses: actions/upload-artifact@v4
+        with:
+          name: HookshotBuilder-macos-arm64
+          path: |
+            build-macos/HookshotBuilder-macos-arm64.zip
+            build-macos/HookshotBuilder-macos-arm64.app.zip
+          if-no-files-found: error
+
+  release:
+    name: macOS release asset
+    if: startsWith(github.ref, 'refs/tags/v')
+    needs: macos-builder
+    runs-on: ubuntu-24.04
+    permissions:
+      contents: write
+    env:
+      GH_TOKEN: ${{ github.token }}
+      GH_REPO: ${{ github.repository }}
+    steps:
+      - name: Check out source
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with:
+          persist-credentials: false
+      - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
+        with:
+          name: HookshotBuilder-macos-arm64
+          path: dist
+      - name: macOS builder to the release
+        run: scripts/release_upload.sh "$GITHUB_REF_NAME" dist/HookshotBuilder-macos-arm64.app.zip

.gitignore

@@ -34,3 +34,10 @@ __pycache__/
 # ---- Linux builds ----
 build-linux/
 *.so
+
+# ---- macOS ----
+# (the game folder a local build writes, the dependencies scripts/macos/deps.sh builds, the helpers built at a build's start)
+game/
+build-macos/
+*.dylib
+.DS_Store

builder/linux/hookshot_builder.py

@@ -28,7 +28,8 @@
 import make_game  # noqa: E402  (LANGUAGES, BUILD_ID)
 import toolchain  # noqa: E402  (default_jobs: RX_JOBS)
 
-CONFIG = os.path.join(os.environ.get("XDG_CONFIG_HOME") or os.path.expanduser("~/.config"), "hookshot", "builder.ini")
+CONFIG = os.path.join(os.path.expanduser("~/Library/Application Support"), "Hookshot", "builder.ini") if sys.platform == "darwin" else \
+    os.path.join(os.environ.get("XDG_CONFIG_HOME") or os.path.expanduser("~/.config"), "hookshot", "builder.ini")
 STAGES = {"read": "Reading your game files", "lift": "Translating the game to PC code", "compile": "Compiling",
           "finish": "Putting the game folder together"}
 
@@ -88,6 +89,8 @@ def desktop_exec(path):
 
 def add_to_menu(folder):
     """an entry in the applications menu that starts the game"""
+    if sys.platform == "darwin":
+        return
     icon = os.path.join(folder, "hookshot.png")
     src = os.path.join(ROOT, "assets", "hookshot.png")
     if os.path.exists(src):
@@ -609,7 +612,8 @@ def on_exit(self, code):
             except OSError:
                 pass
             ttk.Button(self.buttons, text="Play", command=lambda: self.play_and_close(folder)).pack(side="right", padx=self.px(4))
-            ttk.Button(self.buttons, text="Open folder", command=lambda: subprocess.Popen(["xdg-open", folder])).pack(side="right", padx=self.px(4))
+            opener = "open" if sys.platform == "darwin" else "xdg-open"
+            ttk.Button(self.buttons, text="Open folder", command=lambda: subprocess.Popen([opener, folder])).pack(side="right", padx=self.px(4))
         else:
             self.stage_text.set("The build stopped: %s" % getattr(self, "error", "see the lines above"))
             ttk.Button(self.buttons, text="Back", command=self.show_form).pack(side="right", padx=self.px(4))

docs/development.md

@@ -48,6 +48,43 @@ background when the existing "Controller in background" option or `HOS_INPUT_BAC
 `HOS_INPUT_BACKGROUND=0` overrides that option. `HOS_INPUT_EXCLUSIVE=1` controls merging the player's devices while
 the script's existing exclusive interval is active; it is not a focus override.
 
+## macOS (Apple Silicon)
+
+The macOS builder and generated game are native arm64, with no Rosetta dependency. It uses SDL2 for the window and
+input, and the existing WebGPU renderer through `wgpu-native`'s Metal backend; the Vulkan backend is not built on
+macOS. The executable targets macOS 13.3 or newer because Apple's C++20 floating-point formatting support requires it.
+
+The `macOS platform` workflow uploads a `HookshotBuilder-macos-arm64` artifact containing a command-line builder,
+the Hookshot source, a native arm64 Python runtime and modules, SDL2, static FFmpeg, and Metal-enabled `wgpu-native`.
+It does not contain game data or a built game. Download and unpack it on the target Mac; Xcode Command Line Tools
+must be installed there. Double-click `HookshotBuilder.app` to choose your own `.nup` dump and output folder, or
+use the command-line launcher:
+
+```sh
+./HookshotBuilder/HookshotBuilder.command --nup /path/to/game.nup --out /path/to/game-folder --jobs 4
+```
+
+For development outside the artifact, use arm64 builds of SDL2 2.26 or newer and `wgpu-native`. `wgpu-native` must
+include its matching `include/webgpu/webgpu.h` and `lib/libwgpu_native.dylib`, built with Metal support. Set
+`HOOKSHOT_FFMPEG` to the static FFmpeg prefix (or provide FFmpeg through `pkg-config`) and build natively:
+
+```sh
+export WGPU_NATIVE=/path/to/arm64-wgpu-native
+export HOOKSHOT_SDL2=/path/to/arm64/libSDL2-2.0.0.dylib
+export HOOKSHOT_FFMPEG=/path/to/arm64/ffmpeg
+python3 -I nx2/make_game.py --nup /path/to/game.nup --out /path/to/game-folder --jobs 4
+```
+
+The normal builder command validates the arm64 compiler, SDL2, FFmpeg and `wgpu-native` before lifting the game,
+then places `hookshot`, `libSDL2-2.0.0.dylib` and `libwgpu_native.dylib` beside `image.bin` in the output folder;
+FFmpeg is statically linked when `HOOKSHOT_FFMPEG` points to the packaged static libraries. The macOS FFmpeg build
+uses its native AV1 decoder to avoid compiling an additional dav1d dependency in CI.
+For lower-level development, `nx2/lift.py --standalone --write` followed by `nx2/build.py --standalone --jobs 4`
+writes the executable and libraries under `work/sa/build/`. Set `RX_BACKEND=metal` to make the renderer choice
+explicit (Metal is the macOS default). As on other platforms, the local game dump is required to generate and build
+the game. The workflow packages and validates builder dependencies without running a game build; it is not a
+notarized app release.
+
 ## Layout
 
 | Path | What |
@@ -56,7 +93,7 @@ the script's existing exclusive interval is active; it is not a focus override.
 | `nx2/` | the lift pipeline and its tests (mkw-recomp's, made game-neutral; `kit.py` has the paths) |
 | `host/` | the standalone Horizon runtime (TOTKPort's `hos/` as adapted by mkw-recomp), with the native NVN2 layer `nvn_native.c` |
 | `gpu/vk/` | the Vulkan back end (`hookshot_vk.dll`), with Eden's shader recompiler and sirit |
-| `gpu/render/`, `gpu/third_party/` | the WebGPU renderer the exe falls back to, its op stream (`rx_nvn.h`), suyu's decoders |
+| `gpu/render/`, `gpu/third_party/` | the WebGPU renderer (Metal on macOS), its op stream (`rx_nvn.h`), suyu's decoders |
 | `scripts/` | test runs and the profile reader |
 | `third_party/` | a64-kit (unmodified), libopus, zbic (Atmosphere's zstd with ZBIC, for unpacking only) |
 

docs/macos-port-status.md

@@ -0,0 +1,88 @@
+# Porte para macOS (Apple Silicon): estado em 2026-11-09
+
+Branch local `macos-local`, por cima do PR #715 ("Add initial macOS build support"). O jogo roda nativo em arm64,
+com o renderizador WebGPU (`gpu/render`) sobre Metal via `wgpu-native`. Testado num MacBook com Apple M5, 16 GB,
+macOS 27.2, Xcode 27.2.
+
+## Como compilar e rodar
+
+```sh
+bash scripts/macos/deps.sh        # Python, SDL2, dav1d + FFmpeg, wgpu-native em build-macos/deps (sem Homebrew)
+scripts/macos/build-game.sh --nup /caminho/do/jogo.nup --out game --language pt-BR
+cd game && ./hookshot
+```
+
+A primeira compilação leva uns 10 minutos (lift ~1 min, 413 arquivos C em paralelo). A pasta `game/` (o jogo
+montado, ~25 GB) e `build-macos/` ficam fora do repositório (`.gitignore`). Qualidade: `game/hookshot_env.txt`
+(`HOS_QUALITY=alta|media|baixa|minima`); teclas fn+F8 (FSR), fn+F9 (nitidez), fn+F10 (ajustar/esticar), F11 (tela cheia).
+
+## O que funciona
+
+- Build completo nativo arm64; o jogo abre, toca áudio, reconhece controle (PS4 testado) e grava saves.
+- Tela de título com o vídeo de fundo e o menu em português.
+- Abertura (vídeo + legendas em português) a ~50–59 fps.
+- A gameplay carrega (HUD, corações, textos), mas a cena 3D sai branca: ver "Onde paramos".
+
+## O que foi ajustado
+
+### Build e dependências
+- `scripts/macos/deps.sh`: cmake, meson, ninja e pkg-config num venv quando o Mac não os tem (sem Homebrew);
+  dav1d compilado e FFmpeg com `libdav1d` (o decodificador AV1 nativo do FFmpeg só usa hardware: vídeos pretos).
+- `nx2/zbic.py`: o sysroot do SDK passado ao clang chamado pelo caminho completo (`string.h` não encontrado).
+- `nx2/build.py`: `-lc++` no link do macOS; `libwgpu_native.dylib` copiado para a pasta do jogo.
+- `nx2/make_game.py`: copia `scripts/macos/hookshot_env.txt` para a pasta do jogo quando ela não tem um.
+- `scripts/macos/build-game.sh`: roda o builder deste checkout com as dependências de `build-macos/deps`.
+
+### Runtime (`host/`)
+- `posix/win32_posix.c`:
+  - fibras: a página de guarda colocada depois do `makecontext` (o do macOS zera a pilha inteira) e do tamanho de página real (16 KB);
+  - `InitializeCriticalSection` e `SleepConditionVariable*` também no macOS (faltavam no link);
+  - **esperas por endereço com `os_sync_wait_on_address`** (macOS 14.4+). Os "baldes" anteriores eram 64 variáveis de condição compartilhadas, e `WakeConditionVariable` acordava a thread errada (acordadas perdidas). A abertura foi de 13–16 para 50–59 fps. `HOS_WAIT_BUCKETS=1` volta ao método antigo;
+  - QoS `USER_INTERACTIVE` nas threads do jogo (`HOS_QOS`).
+- `posix/display_sdl.c`: sem a dica `wayland,x11` no Mac (a janela não abria); teclas F7–F10 ligadas a `rx_present_hotkey`.
+- `nn_video.c`: o núcleo virtual é liberado enquanto o dav1d decodifica; quadros YUV420 guardados só como NV12 (sem
+  converter para RGBA a cada quadro); `HOS_VIDEO_TIMES=1` mede o decodificador.
+- `hos_main.c`: presets `HOS_QUALITY` (alta, media, baixa, minima) aplicados depois de ler `hookshot_env.txt`.
+- `rx_backend.c`: sem o menu Insert (só existe no Vulkan), `rx_option` lê `HOS_DYNRES`, `HOS_RES_SCALE` e
+  `HOS_REALTIME_STEP` do ambiente, e a resolução fixa passa a funcionar no Mac.
+
+### Renderizador WebGPU/Metal (`gpu/render/`)
+- Buffers de vértice com strides inteiros: o Metal conta `tamanho / stride` elementos e lia o último vértice como
+  zero (o logo do título cortado na diagonal).
+- **Push constants do compute NVN2 no c0** (como `gpu/vk/vk_bank.cpp`): sem elas, as tabelas ficavam 64 bytes
+  adiantadas e o passo YUV→RGB dos vídeos nunca rodava (vídeos pretos, só som).
+- **Base vertex, base instance e draw ID no c0** do vertex (0x640/0x644/0x648, como `GetReplaceConstBuffer` do Vulkan):
+  o texto da interface acha os glifos pelo base instance (todo o texto sumia).
+- **Substituto das redes neurais com HMMA** (`nvn_upscale_stub`), só para programas com HMMA (`nvn_cs_matrix`):
+  - vídeo: a entrada RGB ampliada linearmente para a saída;
+  - gameplay (TAAU): a cor da cena em RGB9E5 empacotada numa imagem R32UI é decodificada no shader e elevada ao
+    quadrado, como `taau_upscale.comp` do Vulkan.
+- **Samplers compartilhados a partir do 16º** (`RX_SHARE_SAMPLERS`, também no emissor WGSL): o Metal permite 16 por
+  estágio, e com 17–18 o wgpu descartava o lote inteiro de comandos do frame.
+- Diagnóstico nos frame dumps: índices do desenho, fatores de blend, imagens e texturas de cada dispatch, chave do
+  programa. Chaves: `RX_NO_UPSCALE_STUB`, `RX_NO_CS_PUSH`.
+
+### Tradutor de shaders
+- `gpu/third_party/suyu/.../frontend/ampere/lower_to_maxwell.h` agora é a versão do eden (`gpu/vk/eden`, a do backend
+  Vulkan), muito mais completa nas instruções do Switch 2. A versão intermediária, com as instruções portadas à mão
+  (0x8a5, 0x4a4, 0x899, 0x825, TXQ), ficou só localmente em `build-macos/lower_to_maxwell.suyu-ported.h`.
+- Emissor WGSL: atômicos de imagem emulados como leitura, operação e escrita (o passe de exposição usa).
+
+## Onde paramos
+
+1. **Gameplay 3D branca.** A cena HDR é desenhada (luminância média ~22), mas a imagem final satura em 1,0.
+   Com `RX_SYNC_DISPATCH_WRITES=1` (copiar de volta para a memória do jogo o que os shaders de compute escrevem, a
+   cada frame) aparecem formas e degradê, a ~4 fps. O Vulkan teve a mesma "parede branca" e só saiu dela com correções
+   específicas do Zelda que o renderizador WebGPU ainda não tem:
+   - shaders que leem dados por ponteiros de 64 bits: ainda falham `3720e3ab9f56fbfa`, `ba00b8c5a7b43504`,
+     `c7f6c62cc98485ac` e `e7d75810de840595` ("WGSL global memory (64-bit addresses) is not implemented"; no Vulkan
+     é a tabela de ponteiros em 0x880 de `vk_bank.cpp`);
+   - grades de dispatch indireto da folhagem (`08b413e`);
+   - a sincronização dos resultados da GPU que a CPU lê (exposição automática), sem custo por frame.
+2. As redes com HMMA (vídeo, TAAU e outras) não são traduzidas; os substitutos dão uma imagem um pouco mais suave.
+3. Escritas de CPU em texturas `Depth32Float` são proibidas no WebGPU: cerca de 2 lotes descartados por execução.
+4. Uma textura "não filtrável" pode receber um sampler compartilhado filtrável (1–2 pipelines inválidas).
+5. Cosmético: caminhos no estilo Windows no log de saves e no nome do cache (`rxcache\pipelines.bin`).
+6. Fps da gameplay: 20–29 com a cena ainda errada; não dá para avaliar antes de corrigir a imagem.
+
+Alternativa: o NX2Apple (baseado no nxnative) lista o Ocarina of Time de Switch 2 rodando no Mac via NVN sobre Metal.