Skip to content

Latest commit

 

History

History
303 lines (248 loc) · 14.1 KB

File metadata and controls

303 lines (248 loc) · 14.1 KB

Installation Guide (macOS Apple Silicon)

MetalOpticalPhoton builds an Apple-Metal GPU optical-photon engine into OpenTOPAS (Geant4). Everything that is specific to this work is in this repository; OpenTOPAS, Geant4 and the toolchain are public software that you install yourself (§1). This document is the zero-to-run sequence on a fresh Mac.

0. Summary

# (install the prerequisites of section 1 first)
./build_topas_gpu.sh full         # from the repository root: patch OpenTOPAS 4.2.3 + build + install
export TOPAS=<TOPAS_INSTALL>/bin/topas-gpu    # printed at the end of the build (§3)
"$TOPAS" your_sim.txt

build_topas_gpu.sh does everything in one pass: path auto-detection → OpenTOPAS version check → source patch (verified) → shader + engine build → OpenTOPAS + extension build → install → run-wrapper generation. If your paths are non-standard, point the script at them with environment variables (§2).

1. Prerequisites (install yourself)

The build script only detects and validates the following — it does not install them.

1-1. Toolchain

  • macOS on Apple Silicon (M series)
  • Xcode + the Metal toolchain — needed for shader compilation (xcrun metal).

    ⚠ The Command Line Tools (xcode-select --install) do NOT include the Metal compiler. From Xcode 26 on, the Metal toolchain is a separate download from Xcode itself. If the build script stops with ✗ Metal compiler not found - Xcode and the Metal toolchain are required …, follow these steps:

    # 1. Install Xcode from the App Store, then make it the active developer directory
    sudo xcode-select -s /Applications/Xcode.app/Contents/Developer
    sudo xcodebuild -license accept
    
    # 2. Install first-launch components (needed if you see the IDESimulatorFoundation plugin error)
    xcodebuild -runFirstLaunch
    
    # 3. Download the Metal toolchain (~700 MB)
    xcodebuild -downloadComponent MetalToolchain
    
    # 4. Verify — success if it prints "Apple metal version ..."
    xcrun metal --version

    Note: the Metal toolchain is mounted as a cryptex and can unmount after a reboot. If xcrun metal --version fails again after rebooting, remount it with the same command (no re-download): xcodebuild -downloadComponent MetalToolchain

  • Homebrew → cmake, gdcm, qt (Qt6)
    brew install cmake gdcm qt

1-2. Geant4

  • Geant4 11.3.2 or later, built and installed with its data sets (G4DATA) — for example as in the OpenTOPAS macOS quick-start guide (OpenTOPAS_quickStart_MacOS.md in the OpenTOPAS source), which installs it to /Applications/GEANT4/geant4-install.
  • The build script finds the install through geant4-config on PATH or the common locations, and stops if the version is below 11.3.2. Use the data sets that belong to your Geant4 release: data sets from another release silently change the physics.

1-3. OpenTOPAS source

  • upstream: https://github.com/OpenTOPAS/OpenTOPAS
  • Required version: exactly v4.2.3. The source patch targets this release; the build script reads the version from the source's CMakeLists.txt and stops on any other version (or if the version cannot be read). ALLOW_UNTESTED_TOPAS=1 lets you try another version at your own risk — the patch must still apply cleanly, otherwise the build stops anyway.
  • You only need the (unmodified) source checked out — the build script patches and builds it.
    git clone https://github.com/OpenTOPAS/OpenTOPAS.git
    cd OpenTOPAS && git checkout v4.2.3
  • For OpenTOPAS's own build prerequisites (Qt6 etc.), see the official OpenTOPAS documentation.

2. Build & install

./build_topas_gpu.sh full          # from the repository root; or run it without arguments and choose f (full) at the prompt

What the script does:

  1. Auto-detects the Geant4 install, the OpenTOPAS source, GDCM and G4DATA (and tells you which environment variable to set for anything it cannot find; on an interactive terminal it asks for the path instead).
  2. Checks the versions: OpenTOPAS must be 4.2.3, Geant4 11.3.2 or later.
  3. Applies the OpenTOPAS source patch topas_patches/opentopas_local.patch (4 source files, 2 logical fixes). It first checks with a no-fuzz dry run that the patch applies exactly, applies it, and verifies the result (patch markers in 4/4 files + a reverse dry run). If the source is already patched it verifies that instead and skips. If the patch cannot be applied or verified, the build stops — an unpatched or partially patched OpenTOPAS is never built.
  4. Builds the Metal shaders → build/default.metallib, and the GPU engine → build/libMetalOpticalPhoton.dylib (in the repository root).
  5. Builds OpenTOPAS together with the GPU extension (topas_extension/).
  6. Installs it, copies the engine dylib + metallib into the install's lib/, re-signs them, and writes the topas-gpu run wrapper (with the Geant4 library and data paths baked in).

Environment-variable overrides

Geant4_DIR=/my/geant4-install TOPAS_SRC=/my/OpenTOPAS GDCM_DIR=/my/gdcm/lib/gdcm-3.x \
  ./build_topas_gpu.sh full
Variable Meaning Default
Geant4_DIR Geant4 install prefix geant4-config / common locations
TOPAS_SRC OpenTOPAS source tree (v4.2.3) common locations (e.g. /Applications/TOPAS/OpenTOPAS)
TOPAS_INSTALL / TOPAS_BUILD install / build output folders ${TOPAS_SRC}-install-gpu / ${TOPAS_SRC}-build-gpu
GDCM_DIR GDCM versioned cmake folder brew --prefix gdcm/lib/gdcm-* / common locations
TOPAS_G4_DATA_DIR Geant4 data sets (G4DATA) the selected Geant4's bundled share/Geant4*/data, then common locations, geant4-config --datasets as last resort
ALLOW_UNTESTED_TOPAS=1 accept an OpenTOPAS version other than 4.2.3 (untested) unset (other versions stop the build)
JOBS parallel compile jobs (make -j) all cores
START start stage 2/3/4/5 (resume after a failure) 2
SKIP_CMAKE=1 in stage 4, skip the cmake re-configure and run make only unset
TOPAS_QT Qt major version for OpenTOPAS (6 or 5) 6
NO_COLOR=1 disable terminal colors color on if TTY

When a compile fails — resume from that stage

You don't have to start over — resume from the failed stage; on failure the script prints the exact retry command. Stages 3–5 reuse the outputs that an earlier run left in build/ and in TOPAS_BUILD, so they only work after a run that got past the earlier stages.

# stages: 2=shaders  3=engine dylib  4=TOPAS build  5=install
START=4 ./build_topas_gpu.sh full              # from stage 4 (TOPAS build), reusing stages 2-3
START=4 SKIP_CMAKE=1 ./build_topas_gpu.sh full # continue with make only (no cmake re-configure)
JOBS=4 ./build_topas_gpu.sh full               # compile with 4 jobs (low memory / debugging)

The full cmake/make logs of stage 4 are kept in ${TOPAS_BUILD}/cmake.log and make.log.

3. Where the binary ends up — and how the benchmarks find it

The build installs into TOPAS_INSTALL (default: <OpenTOPAS source>-install-gpu, next to the source tree) and creates the run wrapper

<TOPAS_INSTALL>/bin/topas-gpu       e.g. /Applications/TOPAS/OpenTOPAS-install-gpu/bin/topas-gpu

The wrapper sets TOPAS_G4_DATA_DIR, the Geant4 / engine library paths and the Qt plugin path, then runs <TOPAS_INSTALL>/bin/topas. The script prints the exact path when it finishes.

The benchmark runners (benchmark/*/run_*.sh, via benchmark/common/topas_env.sh) use

  1. the TOPAS environment variable, if set; otherwise
  2. topas-gpu found on PATH; otherwise
  3. they stop with an error — no other location is searched.

So set one of:

export TOPAS=<TOPAS_INSTALL>/bin/topas-gpu
# or
export PATH="<TOPAS_INSTALL>/bin:$PATH"

Check which binary the runners will use: bash benchmark/common/topas_env.sh.

4. Run

topas-gpu your_sim.txt          # or "$TOPAS" your_sim.txt

Add the GPU optical module to your TOPAS parameter file:

s:Ph/ListName = "Optical"
sv:Ph/Optical/Modules = 2 "g4em-standard_opt4" "gpuoptical"

Working parameter files are in benchmarks 01–11 under benchmark/ (each folder has a README); manual/ documents the GPU parameters.

5. Rebuild the engine only (no TOPAS rebuild)

After changing only the engine or shader code, use wrapper mode:

./build_topas_gpu.sh wrapper

It rebuilds the dylib and metallib, copies them into an existing install, and refreshes the wrapper (no TOPAS rebuild, no patching). The install must come from an earlier full build (its bin/topas must link libMetalOpticalPhoton.dylib; the script checks this and stops otherwise); select it with TOPAS_INSTALL if it is not found automatically. If you changed the TOPAS extension C++ (topas_extension/) or a struct shared between host and shaders (include/MOPTypes.h / shaders/Common.h), rebuild with full.

6. OpenTOPAS patches

What the patch fixes and why it is needed: topas_patches/README.md, topas_patches_summary.md.

7. Manual build (if the build script does not work on your machine)

These are the exact commands the script runs. First fill in the paths for your machine:

export MOP="$(pwd)"                                  # the repository root (run from inside it)
export TOPAS_SRC=/path/to/OpenTOPAS                  # OpenTOPAS 4.2.3 source
export TOPAS_BUILD="${TOPAS_SRC}-build-gpu"
export TOPAS_INSTALL="${TOPAS_SRC}-install-gpu"
export GEANT4_DIR=/path/to/geant4-install            # Geant4 install prefix
export GDCM_DIR="$(brew --prefix gdcm)/lib/gdcm-3.2" # folder with GDCMConfig.cmake (version subfolder varies)
export G4DATA=/path/to/G4DATA                        # Geant4 run-time data sets
export SDK="$(xcrun --show-sdk-path)"

(A) Patch the OpenTOPAS source

# must print no errors; do not continue if it does (wrong version or modified source)
patch -p1 --dry-run -F 0 -d "$TOPAS_SRC" < "$MOP/topas_patches/opentopas_local.patch"
patch -p1 -F 0 -d "$TOPAS_SRC" < "$MOP/topas_patches/opentopas_local.patch"
# to revert: the same command with -R

(B) Metal shaders → metallib

mkdir -p "$MOP/build" && cd "$MOP/build"
for s in PhotonGeneration OpticalPhotonKernel DDAScoring CompactPhotons; do
  xcrun metal -c "$MOP/shaders/$s.metal" -I "$MOP/include" -I "$MOP/shaders" -o "$s.air" -std=metal3.1
done
xcrun metallib PhotonGeneration.air OpticalPhotonKernel.air DDAScoring.air CompactPhotons.air \
  -o default.metallib

(C) GPU engine dylib

cd "$MOP/build"
clang++ -std=c++17 -ObjC++ -fobjc-arc -O2 -c "$MOP/src/MetalOpticalEngine.mm" \
  -I "$MOP/include" -I "$MOP/topas_extension" -o MetalOpticalEngine.o
clang++ -std=c++17 -O2 -c "$MOP/topas_extension/TopasParameterParser.cc" \
  -I "$MOP/topas_extension" -I "$MOP/include" -o TopasParameterParser.o
clang++ -dynamiclib -O2 MetalOpticalEngine.o TopasParameterParser.o \
  -o libMetalOpticalPhoton.dylib \
  -framework Metal -framework MetalPerformanceShaders -framework Foundation \
  -install_name "@rpath/libMetalOpticalPhoton.dylib"

(D) Build TOPAS + the GPU extension

mkdir -p "$TOPAS_BUILD" && cd "$TOPAS_BUILD"
export Geant4_DIR="$GEANT4_DIR"
cmake "$TOPAS_SRC" \
  -DCMAKE_INSTALL_PREFIX="$TOPAS_INSTALL" -DCMAKE_BUILD_TYPE=Release \
  -DTOPAS_EXTENSIONS_DIR="$MOP/topas_extension" \
  -DGeant4_DIR="$GEANT4_DIR/lib/cmake/Geant4" -DGDCM_DIR="$GDCM_DIR" \
  -DEXPAT_INCLUDE_DIR="$SDK/usr/include" -DZLIB_INCLUDE_DIR="$SDK/usr/include" \
  -DEXPAT_LIBRARY="$SDK/usr/lib/libexpat.tbd" -DZLIB_LIBRARY_RELEASE="$SDK/usr/lib/libz.tbd" \
  -DCMAKE_CXX_FLAGS="-include cmath -iquote $MOP/include -iquote $MOP/topas_extension" \
  -DCMAKE_EXE_LINKER_FLAGS="-L$MOP/build -lMetalOpticalPhoton -Wl,-rpath,$MOP/build -framework Metal -framework Foundation -framework MetalPerformanceShaders" \
  -DCMAKE_OSX_ARCHITECTURES=arm64 \
  -DTOPAS_USE_QT=ON -DTOPAS_USE_QT6=ON
make -j"$(sysctl -n hw.ncpu)"     # use -j4 etc. to lower the job count
make install

Both flags in -DCMAKE_CXX_FLAGS are required:

  • -include cmath : prevents the macOS SDK <math.h> isinf macro from clashing with libc++ <complex>'s std::isinf. Without it, compilation fails.
  • -iquote $MOP/include -iquote $MOP/topas_extension : TOPAS copies the extension .cc/.hh files flat into build/extensions/, so the wrapper headers' (MOPTypes.hh, MetalOpticalEngine.hh) #include "../include/MOPTypes.h" would resolve to build/include/ (which does not exist) and fail with '../include/MOPTypes.h' file not found. -iquote puts the original include/ and topas_extension/ folders on the quoted-include search path so the canonical headers are found. Without it, the extensions target fails to build.

If make fails, fix the cause and continue incrementally with cd "$TOPAS_BUILD" && make.

(E) Install the engine + generate the run wrapper

mkdir -p "$TOPAS_INSTALL/lib"
cp "$MOP/build/libMetalOpticalPhoton.dylib" "$TOPAS_INSTALL/lib/"
cp "$MOP/build/default.metallib"            "$TOPAS_INSTALL/lib/"

# Re-sign after copying. Relinking invalidates the ad-hoc signature and macOS then kills the
# process with SIGKILL: 0-byte output, exit 137, and no dyld message.
codesign --force --sign - "$TOPAS_INSTALL/lib/libMetalOpticalPhoton.dylib"
codesign --force --sign - "$TOPAS_INSTALL/bin/topas"

cat > "$TOPAS_INSTALL/bin/topas-gpu" <<EOF
#!/bin/bash
SCRIPT_DIR="\$(cd "\$(dirname "\$0")/.." && pwd)"
export QT_QPA_PLATFORM_PLUGIN_PATH="\$SCRIPT_DIR/Frameworks"
export TOPAS_G4_DATA_DIR="$G4DATA"
export DYLD_LIBRARY_PATH="\$SCRIPT_DIR/lib:\${DYLD_LIBRARY_PATH}"
export DYLD_LIBRARY_PATH="$GEANT4_DIR/lib:\${DYLD_LIBRARY_PATH}"
export METAL_DEVICE_WRAPPER_TYPE=1
export G4TRACE_DIR=OFF
exec "\$SCRIPT_DIR/bin/topas" "\$@"
EOF
chmod +x "$TOPAS_INSTALL/bin/topas-gpu"
export TOPAS="$TOPAS_INSTALL/bin/topas-gpu"

Then run with "$TOPAS" your_sim.txt. (To rebuild only the engine or shaders, repeat (B), (C) and (E) — no TOPAS rebuild needed.)