feat: stream and accelerate large build logs

Benchmarks/README.md

@@ -0,0 +1,72 @@
+# Large-log benchmark
+
+`large-log.sh` measures the release CLI against deterministic synthetic Xcode build logs. It
+records elapsed time, throughput, and peak resident memory while also verifying that the TOON
+summary reports a successful build.
+
+Run the 10 MiB pull-request smoke benchmark:
+
+```bash
+Benchmarks/large-log.sh 10
+```
+
+Run the full size progression used for large-log regression analysis:
+
+```bash
+Benchmarks/large-log.sh 10 100 500
+```
+
+To benchmark an existing binary without rebuilding, set `XCSIFT_BENCHMARK_BINARY`:
+
+```bash
+XCSIFT_BENCHMARK_BINARY=/path/to/xcsift Benchmarks/large-log.sh 10
+```
+
+Select a workload with `XCSIFT_BENCHMARK_PROFILE` (the default is `phase`):
+
+```bash
+XCSIFT_BENCHMARK_PROFILE=video-go-shaped Benchmarks/large-log.sh 10 100 500
+```
+
+| Profile | Purpose |
+| --- | --- |
+| `phase` | Phase-heavy parser stress case |
+| `fast-reject` | ASCII build-command noise with no reportable event |
+| `fast-reject-unicode` | The same rejection path with Unicode input |
+| `warning-duplicate` | Repeated compiler warning and deduplication path |
+| `warning-unique` | Distinct warning identities and retained-state growth |
+| `fixture-mixed` | Repeated sections of the checked-in real build fixture |
+| `video-go-shaped` | 0.13% phases, 9.43% warnings, and otherwise build-command noise |
+
+The CSV output is intended for comparison on the same machine. Hosted CI timing and RSS vary too
+much for a strict wall-clock gate, so normal XCTest coverage asserts bounded framing and output
+correctness instead.
+
+Every profile appends a successful terminal marker. The synthetic profiles isolate specific parser
+paths; they are not substitutes for checking output equivalence on real build logs. Reader
+buffering is bounded, but retained errors and unique-warning deduplication state still scale with
+the number of distinct diagnostics.
+
+## Reference results
+
+Recorded on 2026-08-07 using the release build on an arm64 Mac with macOS 26.6 and Swift 6.3.3.
+The primary results are medians of three runs:
+
+| Profile | Nominal input | Elapsed | Throughput | Peak RSS |
+| --- | ---: | ---: | ---: | ---: |
+| `phase` | 10 MiB | 0.02 s | 500.00 MiB/s | 8.70 MiB |
+| `phase` | 100 MiB | 0.18 s | 555.56 MiB/s | 8.75 MiB |
+| `phase` | 500 MiB | 0.87 s | 574.71 MiB/s | 8.75 MiB |
+| `video-go-shaped` | 10 MiB | 0.02 s | 500.00 MiB/s | 10.28 MiB |
+| `video-go-shaped` | 100 MiB | 0.23 s | 434.78 MiB/s | 20.45 MiB |
+| `video-go-shaped` | 500 MiB | 1.10 s | 454.55 MiB/s | 20.47 MiB |
+
+Additional single-run 500 MiB checks measured 0.72 s for `fast-reject`, 0.94 s for
+`fast-reject-unicode`, 0.63 s for `fixture-mixed`, and 3.41 s for `warning-duplicate`. The
+`warning-unique` profile measured 1.36 s and 265.92 MiB RSS at 100 MiB, illustrating that exact
+deduplication state grows with distinct diagnostics even though input framing remains bounded.
+
+For comparison, the initial streaming implementation processed the same 500 MiB `phase` workload
+in 226.97 s at 2.20 MiB/s with 8.92 MiB peak RSS. The optimized parser completes it in 0.87 s at
+574.71 MiB/s with 8.75 MiB peak RSS: about 261 times faster with the same bounded-memory behavior.
+Treat these numbers as a reference snapshot rather than a portable performance threshold.

Benchmarks/large-log.sh

@@ -0,0 +1,133 @@
+#!/usr/bin/env bash
+
+set -eu
+
+script_directory=$(CDPATH= cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)
+repository_root=$(CDPATH= cd -- "$script_directory/.." && pwd)
+benchmark_root=$(mktemp -d "${TMPDIR:-/tmp}/xcsift-large-log.XXXXXX")
+trap 'rm -rf -- "$benchmark_root"' EXIT
+
+if [ -n "${XCSIFT_BENCHMARK_BINARY:-}" ]; then
+  xcsift_binary=$XCSIFT_BENCHMARK_BINARY
+else
+  (cd "$repository_root" && swift build -c release >/dev/null)
+  xcsift_binary="$repository_root/.build/release/xcsift"
+fi
+
+if [ ! -x "$xcsift_binary" ]; then
+  echo "error: xcsift binary is not executable: $xcsift_binary" >&2
+  exit 1
+fi
+
+if [ "$#" -gt 0 ]; then
+  sizes_mib="$*"
+else
+  sizes_mib="10 100 500"
+fi
+
+profile=${XCSIFT_BENCHMARK_PROFILE:-phase}
+
+generate_input() {
+  local requested_bytes=$1
+  local generated_path=$2
+
+  case "$profile" in
+    phase)
+      yes "CompileSwiftSources normal arm64 /tmp/Foo.swift (in target 'VideoGo' from project 'VideoGo')" \
+        | head -c "$requested_bytes" >"$generated_path"
+      ;;
+    fast-reject)
+      yes "    cd /Users/runner/work/VideoGo/VideoGo && /usr/bin/touch /private/tmp/DerivedData/VideoGo/Build/Intermediates.noindex/stamp" \
+        | head -c "$requested_bytes" >"$generated_path"
+      ;;
+    fast-reject-unicode)
+      yes "    cd /Users/runner/work/VideoGo/构建路径 && /usr/bin/touch /private/tmp/DerivedData/VideoGo/Build/Intermediates.noindex/stamp" \
+        | head -c "$requested_bytes" >"$generated_path"
+      ;;
+    warning-duplicate)
+      yes "/tmp/VideoGo/Sources/Foo.swift:42:7: warning: immutable value 'value' was never used" \
+        | head -c "$requested_bytes" >"$generated_path"
+      ;;
+    warning-unique)
+      awk -v target="$requested_bytes" '
+        BEGIN {
+          bytes = 0
+          for (i = 1; bytes < target; i++) {
+            line = sprintf("/tmp/VideoGo/Sources/File%d.swift:%d:7: warning: unique diagnostic number %d", i % 10000, i % 500 + 1, i)
+            print line
+            bytes += length(line) + 1
+          }
+        }
+      ' >"$generated_path"
+      ;;
+    fixture-mixed)
+      while :; do
+        sed '$d' "$repository_root/Tests/XCSiftCoreTests/Fixtures/build.txt"
+      done | head -c "$requested_bytes" >"$generated_path"
+      ;;
+    video-go-shaped)
+      awk -v target="$requested_bytes" '
+        BEGIN {
+          bytes = 0
+          quote = sprintf("%c", 39)
+          for (i = 0; bytes < target; i++) {
+            slot = i % 10000
+            if (slot < 13) {
+              line = "CompileSwiftSources normal arm64 /tmp/Foo.swift (in target " \
+                quote "VideoGo" quote " from project " quote "VideoGo" quote ")"
+            } else if (slot < 956) {
+              line = sprintf("/tmp/VideoGo/Sources/File%d.swift:%d:7: warning: repeated diagnostic group %d", i % 1024, i % 500 + 1, i % 1024)
+            } else {
+              line = "    cd /Users/runner/work/VideoGo/VideoGo && /usr/bin/touch " \
+                "/private/tmp/DerivedData/VideoGo/Build/Intermediates.noindex/generated-stamp"
+            }
+            print line
+            bytes += length(line) + 1
+          }
+        }
+      ' >"$generated_path"
+      ;;
+    *)
+      echo "error: unknown benchmark profile: $profile" >&2
+      exit 1
+      ;;
+  esac
+
+  printf '\n** BUILD SUCCEEDED **\n' >>"$generated_path"
+}
+
+printf 'profile,size_mib,bytes,elapsed_seconds,throughput_mib_per_second,peak_rss_mib\n'
+
+for size_mib in $sizes_mib; do
+  input_path="$benchmark_root/input-${size_mib}m.log"
+  output_path="$benchmark_root/output-${size_mib}m.toon"
+  metrics_path="$benchmark_root/metrics-${size_mib}m.txt"
+  requested_bytes=$((size_mib * 1024 * 1024))
+
+  generate_input "$requested_bytes" "$input_path"
+
+  if [ "$(uname -s)" = "Darwin" ]; then
+    /usr/bin/time -lp "$xcsift_binary" -f toon <"$input_path" >"$output_path" 2>"$metrics_path"
+    elapsed_seconds=$(awk '$1 == "real" { print $2 }' "$metrics_path")
+    peak_rss_bytes=$(awk '/maximum resident set size/ { print $1 }' "$metrics_path")
+    peak_rss_mib=$(awk -v bytes="$peak_rss_bytes" 'BEGIN { printf "%.2f", bytes / 1048576 }')
+  else
+    /usr/bin/time -f 'elapsed_seconds=%e\npeak_rss_kib=%M' \
+      "$xcsift_binary" -f toon <"$input_path" >"$output_path" 2>"$metrics_path"
+    elapsed_seconds=$(awk -F= '$1 == "elapsed_seconds" { print $2 }' "$metrics_path")
+    peak_rss_kib=$(awk -F= '$1 == "peak_rss_kib" { print $2 }' "$metrics_path")
+    peak_rss_mib=$(awk -v kib="$peak_rss_kib" 'BEGIN { printf "%.2f", kib / 1024 }')
+  fi
+
+  if ! grep -q '^status: success$' "$output_path"; then
+    echo "error: benchmark output did not report success for ${size_mib} MiB" >&2
+    exit 1
+  fi
+
+  actual_bytes=$(wc -c <"$input_path" | tr -d ' ')
+  throughput=$(awk -v bytes="$actual_bytes" -v seconds="$elapsed_seconds" \
+    'BEGIN { if (seconds > 0) printf "%.2f", (bytes / 1048576) / seconds; else printf "n/a" }')
+
+  printf '%s,%s,%s,%s,%s,%s\n' \
+    "$profile" "$size_mib" "$actual_bytes" "$elapsed_seconds" "$throughput" "$peak_rss_mib"
+done

README.md

@@ -27,6 +27,7 @@ A Swift command-line tool to parse and format xcodebuild/SPM output for coding a
 - **Configuration files** — `.xcsift.toml` for project or user-wide defaults
 - **Quiet/Werror/exit-on-failure modes** — for CI pipelines
 - **xcbeautify/Tuist input** — parse pre-formatted output with `--xcbeautify`
+- **Streaming large-log parsing** — consumes stdin incrementally with bounded line buffering
 
 See the [full documentation](https://ldomaradzki.github.io/xcsift/documentation/xcsift) for details.
 
@@ -114,6 +115,13 @@ swift test                             # Run tests
 swift format --recursive --in-place .  # Format (required before committing)
 ```
 
+Large-log release benchmarks are available in [`Benchmarks/`](Benchmarks/README.md):
+
+```bash
+Benchmarks/large-log.sh 10          # Pull-request smoke benchmark
+Benchmarks/large-log.sh 10 100 500  # Full size progression
+```
+
 Documentation source is in `Sources/xcsift.docc/`. Preview locally:
 
 ```bash