diff --git a/.agent/HANDOFF.md b/.agent/HANDOFF.md index 74a2268..f1b31e0 100644 --- a/.agent/HANDOFF.md +++ b/.agent/HANDOFF.md @@ -4,12 +4,26 @@ Last updated: 2026-10-01 ## Current task -Move the repository from GocciaScript 0.10.0 to 0.14.0 and replace the -workarounds that 0.10.0 forced with what 0.11.0 to 0.14.0 provide natively. +Nothing is in flight. The 0.14.0 update is merged (PR #11, merge commit +`3eeab1f`) and tagged `0.1.0`. PR #12 adds `benchmarks/fleet-bunnymark.sh` and +records the fleet results in `benchmarks/import-binding-aliases.md`. -State: implemented and validated locally on the branch `t3code/4b7db725`, and -published as a pull request. Merging is left to Johannes. After the merge the -repository gets its first tag; the tag name is not decided yet. +The next piece of work is Bunnymark performance; see "Next steps". + +Fleet results, 10,000 sprites, 100 frames, three interleaved runs: + +| Machine | Renderer | Instanced 0.10.0 -> 0.14.0 | Direct 0.10.0 -> 0.14.0 | +|---|---|---|---| +| boiler (Ryzen 9 6900HX, Linux) | Radeon RX 6600M | 31.3 -> 37.4 FPS | 10.2 -> 10.0 FPS | +| boiler | Radeon 680M | 31.4 -> 37.8 FPS | 10.2 -> 10.6 FPS | +| boiler | llvmpipe under Xvfb | 20.7 -> 23.0 FPS | 8.8 -> 9.1 FPS | +| firepit (Apple M5 Max, macOS 27.0.1) | Apple M5 Max | 51.5 -> 62.0 FPS | 17.8 -> 18.2 FPS | +| burnside (Apple M1 Max, macOS 26.5.2) | Apple M1 Max | 26.9 -> 30.9 FPS | 10.2 -> 10.5 FPS | + +The whole fleet is measured and recorded in +`benchmarks/import-binding-aliases.md`. The Macs do not accept SSH; they were +driven through lantaarn (`lantaarn-ctl` from boiler over Tailscale), which was +capturing the screen at 30 FPS during the runs. ## What changed @@ -76,13 +90,32 @@ frame in its 10,000 native calls. - Callback-taking raylib functions accept `FFI.callback(...).create(fn)` handles at run time, but `FFIPointerInput` does not type them. This predates the update. -- Which name the first tag gets. `github:` provider imports and npm - publication both depend on it. ## Next steps -1. Merge the pull request once CI is green on both lanes. -2. Tag the merge commit. -3. Collect Bunnymark numbers across machines, then update - `benchmarks/import-binding-aliases.md`. -4. Optimise the Bunnymark update loop, which now bounds the instanced path. +The instanced Bunnymark path is bound by the JavaScript update loop +(`updateBunnies` in `examples/lib/bunny-state.ts`), not by rendering. Measured +on boiler with GocciaScript 0.14.0, 10,000 sprites, headless and without +raylib: + +| Variant of the update loop | ms per frame | Same positions | +|---|---:|---| +| Current | about 31 | yes | +| Module constants hoisted into locals, bounds precomputed | about 23 | yes | +| The same with a counted `for` loop (needs a compatibility flag) | about 20.5 | yes | +| Positions kept only in the Float32 instance buffer | about 20 | no | + +The opcode profile of the current loop shows about 88 opcodes per sprite at +roughly 36 ns each. `OP_GET_LOCAL` is 38% of them. Each sprite also costs three +`OP_GET_GLOBAL` for module-level constants, six `OP_LOAD_HOLE`, and a +finally-handler push and pop for `for...of`. + +1. Apply the hoisting variant here: about 25% less update time with identical + output and default syntax. Re-run `benchmarks/fleet-bunnymark.sh` with + `CURRENT_REF` set to the new revision. +2. Take the opcode profile to GocciaScript. The release binaries are stripped + and `perf` is restricted on boiler, so native hot spots need a symbolized + build. +3. The `DrawTextureV` path spends about 72 ms per frame in 10,000 native + calls, about 7 microseconds each including six struct field writes. That + is an FFI call-overhead question for GocciaScript. diff --git a/README.md b/README.md index f468927..a069822 100644 --- a/README.md +++ b/README.md @@ -181,7 +181,8 @@ directly. [the preceding application baseline](benchmarks/application-performance.md), [the instancing validation](benchmarks/bunnymark-instancing.md), and [the import-alias removal](benchmarks/import-binding-aliases.md) are recorded -in the repository. +in the repository. `benchmarks/fleet-bunnymark.sh` repeats the 0.10.0 versus +0.14.0 comparison on any macOS or Linux machine and prints its table rows. ## Project scripts and tests diff --git a/benchmarks/fleet-bunnymark.sh b/benchmarks/fleet-bunnymark.sh new file mode 100755 index 0000000..1fedcb9 --- /dev/null +++ b/benchmarks/fleet-bunnymark.sh @@ -0,0 +1,192 @@ +#!/usr/bin/env bash +# Bunnymark across GocciaScript versions on one machine. +# +# benchmarks/fleet-bunnymark.sh +# +# Compares the last 0.10.0 revision of this repository on GocciaScript 0.10.0 +# with a current revision on GocciaScript 0.14.0, and prints one Markdown table +# row per draw path. It opens thirteen short raylib windows on the current +# display. It needs raylib 6.0 as a dynamic library (`brew install raylib` on +# macOS) plus git, curl and unzip or tar. A copy outside a clone of this +# repository fetches its own clone. +# +# Optional environment: +# CURRENT_REF revision measured on 0.14.0 (default: 0.1.0) +# FRAMES frames per run after startup (default: 100) +# ROUNDS interleaved runs per variant (default: 3) +# SPRITES sprites per run (default: 10000) +# RAYLIB_LIB raylib 6.0 dynamic library to use when it is not in a path +# the bindings already search +# LABEL note for the table, such as the selected GPU +# WORK_DIR scratch directory, reused between runs + +set -euo pipefail + +PREVIOUS_REF=70a04532029e17156d4e206dd55f19cd449106e1 +CURRENT_REF=${CURRENT_REF:-0.1.0} +FRAMES=${FRAMES:-100} +ROUNDS=${ROUNDS:-3} +SPRITES=${SPRITES:-10000} +LABEL=${LABEL:-} +WORK_DIR=${WORK_DIR:-${TMPDIR:-/tmp}/gocciascript-raylib-fleet} + +# Outside a clone, such as a copy sent to another machine, fetch one. +if ! repository=$(git -C "$(dirname "$0")" rev-parse --show-toplevel 2>/dev/null) || + ! git -C "$repository" cat-file -e "$PREVIOUS_REF^{commit}" 2>/dev/null; then + repository="$WORK_DIR/repository" + if [ ! -d "$repository/.git" ]; then + mkdir -p "$WORK_DIR" + git clone --quiet https://github.com/frostney/GocciaScript-Raylib.git "$repository" + fi + git -C "$repository" fetch --quiet --tags origin +fi + +case "$(uname -s)-$(uname -m)" in + Darwin-arm64) platform=macos-arm64; archive=zip; library=libraylib.dylib ;; + Darwin-x86_64) platform=macos-x64; archive=zip; library=libraylib.dylib ;; + Linux-aarch64) platform=linux-arm64; archive=tar.gz; library=libraylib.so ;; + Linux-x86_64) platform=linux-x64; archive=tar.gz; library=libraylib.so ;; + *) echo "Unsupported platform: $(uname -s)-$(uname -m)" >&2; exit 1 ;; +esac + +# Prints the path of the named binary from a pinned GocciaScript release. +goccia_binary() { + local version=$1 name=$2 + local directory="$WORK_DIR/gocciascript-$version" + local asset="gocciascript-$version-$platform.$archive" + # The callers run this in a command substitution, where bash does not exit + # on a failed command, so each step reports its own failure. The release is + # extracted beside its final path so an interrupted download is retried. + if [ ! -d "$directory" ]; then + rm -rf "$directory.partial" + mkdir -p "$directory.partial" + curl --fail --location --silent --show-error \ + "https://github.com/frostney/GocciaScript/releases/download/$version/$asset" \ + --output "$WORK_DIR/$asset" || return 1 + if [ "$archive" = zip ]; then + unzip -q "$WORK_DIR/$asset" -d "$directory.partial" || return 1 + else + tar -xzf "$WORK_DIR/$asset" -C "$directory.partial" || return 1 + fi + mv "$directory.partial" "$directory" + fi + local binary + binary=$(find "$directory" -type f -name "$name" -print -quit) + if [ -z "$binary" ]; then + echo "$name is not in $asset" >&2 + return 1 + fi + chmod +x "$binary" + printf '%s\n' "$binary" +} + +# Extracts one revision and adds a fixture per draw path. +prepare_tree() { + local ref=$1 tree=$2 + rm -rf "$tree" + mkdir -p "$tree/.cache/fleet" + git -C "$repository" archive "$ref" | tar -x -C "$tree" + if [ -n "${RAYLIB_LIB:-}" ]; then + ln -s "$RAYLIB_LIB" "$tree/$library" + fi + local path + for path in instanced direct; do + cat > "$tree/.cache/fleet/$path.ts" < "$tree/.cache/fleet/renderer.ts" < "$log" + +# raylib names the OpenGL renderer in its start-up log. +renderer=$(cd "$WORK_DIR/current" && "$runner" -P .cache/fleet/renderer.ts 2>&1 | + sed -n 's/.*> Renderer: *//p' | head -1) +if [ -z "$renderer" ]; then + echo "raylib did not report an OpenGL renderer; is a display available?" >&2 + exit 1 +fi + +# Runs one variant and appends " ". +measure() { + local version=$1 path=$2 output summary + if [ "$version" = 0.10.0 ]; then + output=$(cd "$WORK_DIR/previous" && "$loader" ".cache/fleet/$path.ts" 2>&1) + else + # -P accepts the ffi request in goccia.json for this run only. + output=$(cd "$WORK_DIR/current" && "$runner" -P ".cache/fleet/$path.ts" 2>&1) + fi + summary=$(printf '%s\n' "$output" | grep "Bunnymark summary" || true) + if [ -z "$summary" ] || ! printf '%s\n' "$summary" | grep -q "draw_path=$path "; then + printf '%s\n' "$output" | tail -20 >&2 + echo "Bunnymark $version $path did not report the $path draw path" >&2 + exit 1 + fi + printf '%s %s %s %s\n' "$version" "$path" \ + "$(printf '%s\n' "$summary" | sed -n 's/.*frames=\([0-9]*\).*/\1/p')" \ + "$(printf '%s\n' "$summary" | sed -n 's/.*frame_ms=\([0-9]*\).*/\1/p')" >> "$log" +} + +round=1 +while [ "$round" -le "$ROUNDS" ]; do + for path in instanced direct; do + measure 0.10.0 "$path" + measure 0.14.0 "$path" + done + round=$((round + 1)) +done + +if [ "$(uname -s)" = Darwin ]; then + processor=$(sysctl -n machdep.cpu.brand_string) + system="macOS $(sw_vers -productVersion)" +else + processor=$(sed -n 's/^model name[^:]*: *//p' /proc/cpuinfo | head -1) + system=$(. /etc/os-release && printf '%s' "$PRETTY_NAME") +fi + +echo +echo "Bunnymark, $SPRITES sprites, uncapped, $FRAMES frames per run, $ROUNDS interleaved runs." +echo "Previous: ${PREVIOUS_REF:0:7} on GocciaScript 0.10.0. Current: $CURRENT_REF on GocciaScript 0.14.0." +echo +echo "| Machine | Renderer | Draw path | 0.10.0 | 0.14.0 | Change |" +echo "|---|---|---|---:|---:|---:|" +for path in instanced direct; do + awk -v path="$path" -v machine="$processor, $system${LABEL:+, $LABEL}" \ + -v renderer="$renderer" ' + $2 == path { frames[$1] += $3; milliseconds[$1] += $4 } + END { + previous = frames["0.10.0"] * 1000 / milliseconds["0.10.0"] + current = frames["0.14.0"] * 1000 / milliseconds["0.14.0"] + printf "| %s | %s | `%s` | %.1f FPS | %.1f FPS | %+.1f%% |\n", + machine, renderer, path, previous, current, + (current / previous - 1) * 100 + }' "$log" +done +echo +echo "Per-run frame times (version, draw path, frames, milliseconds):" +cat "$log" diff --git a/benchmarks/import-binding-aliases.md b/benchmarks/import-binding-aliases.md index 6b30477..bd4f830 100644 --- a/benchmarks/import-binding-aliases.md +++ b/benchmarks/import-binding-aliases.md @@ -9,8 +9,8 @@ into local constants. - AMD Ryzen 9 6900HX, Linux Mint 22.3 (x86-64) - GocciaScript 0.10.0 and 0.14.0 release binaries, bytecode mode - dynamically loaded raylib 6.0 at the pinned commit -- Xvfb with Mesa llvmpipe 25.2.8 (software OpenGL 4.5), so absolute frame rates - are not comparable with the Apple M1 Max results in the other reports +- Xvfb with Mesa llvmpipe 25.2.8 (software OpenGL 4.5) for the two sections + below; [Across machines](#across-machines) lists its own renderers ## Imported binding versus local alias @@ -26,8 +26,8 @@ one process; the three runs agreed within 0.4%. Each iteration reads two imported bindings. Compared with a local read, one imported read cost about 9.3 microseconds more on 0.10.0 and costs about -0.12 microseconds more on 0.14.0. GocciaScript 0.11.0 lists the change as "retain resolved import -bindings". +0.12 microseconds more on 0.14.0. GocciaScript 0.11.0 lists the change as +"retain resolved import bindings". ## Bunnymark with and without aliases @@ -45,14 +45,54 @@ The means differ by 0.9% on the per-sprite `DrawTextureV` path and by 0.6% in the other direction on the instanced path. Both differences are smaller than the spread between runs of one variant. -For the whole update, the previous revision on 0.10.0 (aliases, heightmap -quad) was compared with this revision on 0.14.0 (direct imports, plane quad) -under the same workload, three interleaved runs each: - -| Draw path | Previous revision, 0.10.0 | This revision, 0.14.0 | -|---|---|---| -| `DrawTextureV` | 2946, 2916, 2940 ms (mean 2934.0) | 2875, 2877, 2877 ms (mean 2876.3) | -| `DrawMeshInstanced` | 1260, 1290, 1268 ms (mean 1272.7) | 1182, 1181, 1176 ms (mean 1179.7) | +## Across machines + +`benchmarks/fleet-bunnymark.sh` compared the last 0.10.0 revision (`70a0453`: +aliases, heightmap quad) on GocciaScript 0.10.0 with tag `0.1.0` (direct +imports, plane quad) on GocciaScript 0.14.0. Each row is 10,000 fixed-seed +sprites, uncapped, 100 frames per run after startup, three interleaved runs +per variant, reported as total frames over total frame time. The comparison +covers the engine and the example together. + +| Machine | Renderer | Draw path | 0.10.0 | 0.14.0 | Change | +|---|---|---|---:|---:|---:| +| Apple M5 Max, macOS 27.0.1 | Apple M5 Max | `DrawMeshInstanced` | 51.5 FPS | 62.0 FPS | +20.3% | +| Apple M5 Max, macOS 27.0.1 | Apple M5 Max | `DrawTextureV` | 17.8 FPS | 18.2 FPS | +2.2% | +| Apple M1 Max, macOS 26.5.2 | Apple M1 Max | `DrawMeshInstanced` | 26.9 FPS | 30.9 FPS | +14.7% | +| Apple M1 Max, macOS 26.5.2 | Apple M1 Max | `DrawTextureV` | 10.2 FPS | 10.5 FPS | +2.6% | +| Ryzen 9 6900HX, Linux Mint 22.3 | Radeon RX 6600M | `DrawMeshInstanced` | 31.3 FPS | 37.4 FPS | +19.7% | +| Ryzen 9 6900HX, Linux Mint 22.3 | Radeon RX 6600M | `DrawTextureV` | 10.2 FPS | 10.0 FPS | -2.3% | +| Ryzen 9 6900HX, Linux Mint 22.3 | Radeon 680M | `DrawMeshInstanced` | 31.4 FPS | 37.8 FPS | +20.4% | +| Ryzen 9 6900HX, Linux Mint 22.3 | Radeon 680M | `DrawTextureV` | 10.2 FPS | 10.6 FPS | +4.6% | +| Ryzen 9 6900HX, Linux Mint 22.3 | llvmpipe under Xvfb | `DrawMeshInstanced` | 20.7 FPS | 23.0 FPS | +11.0% | +| Ryzen 9 6900HX, Linux Mint 22.3 | llvmpipe under Xvfb | `DrawTextureV` | 8.8 FPS | 9.1 FPS | +3.1% | + +Per-run 100-frame times in milliseconds, in run order: + +| Machine and renderer | Draw path | 0.10.0 | 0.14.0 | +|---|---|---|---| +| Apple M5 Max | `DrawMeshInstanced` | 1948, 1934, 1945 | 1629, 1595, 1618 | +| Apple M5 Max | `DrawTextureV` | 5504, 5603, 5769 | 5563, 5469, 5480 | +| Apple M1 Max | `DrawMeshInstanced` | 3913, 3644, 3583 | 3229, 3246, 3236 | +| Apple M1 Max | `DrawTextureV` | 9857, 9747, 9717 | 9566, 9510, 9508 | +| Radeon RX 6600M | `DrawMeshInstanced` | 3197, 3187, 3212 | 2658, 2676, 2682 | +| Radeon RX 6600M | `DrawTextureV` | 9760, 9840, 9804 | 10062, 10018, 10011 | +| Radeon 680M | `DrawMeshInstanced` | 3186, 3185, 3190 | 2645, 2651, 2644 | +| Radeon 680M | `DrawTextureV` | 9852, 9825, 9796 | 9394, 9383, 9403 | +| llvmpipe under Xvfb | `DrawMeshInstanced` | 4797, 4851, 4857 | 4322, 4363, 4382 | +| llvmpipe under Xvfb | `DrawTextureV` | 11285, 11326, 11320 | 10941, 11027, 10928 | + +The instanced path gains 15 to 20% on every hardware renderer, and the +per-sprite `DrawTextureV` path stays within 5% either way. On the Ryzen +machine the two GPUs give the same instanced frame rate: at 10,000 sprites the +frame is bound by the JavaScript update loop, which measured about 27.6 ms of +a 28.2 ms frame on the Radeon RX 6600M, with drawing and presenting under +1 ms. + +Both Macs were driven through a remote-desktop session that captured the +screen at 30 FPS during the runs, so their absolute numbers may be slightly +low. The first 0.10.0 instanced run on the M1 Max is about 8% slower than the +other two. ## Decision