feat: stream and accelerate large build logs

[SunsetWan]

What

• add a public single-use StreamingOutputParser while preserving OutputParser as the complete-input compatibility API
• replace CLI readToEnd() with a POSIX-backed 64 KiB streaming reader and bounded 5,000-byte line framing
• classify parser candidates in one UTF-8 scan and skip build-info, JSON, xcbeautify, linker, and diagnostic work when the line cannot match
• use memchr framing and decode complete in-chunk lines directly while preserving split UTF-8, CRLF, EOF, empty-line, and oversized-line behavior
• keep exact warning counts without retaining full warning models when details are not requested, using a collision-free compact identity
• parse compiler-warning locations with a fixed UTF-8 marker search and backward field lookup instead of repeated Foundation search plus split/join
• add deterministic phase, rejection, Unicode, warning, real-fixture, and VideoGo-shaped benchmark profiles

Why

The CLI previously waited for EOF, retained the whole build log as Data/String, then created a second full collection of line strings before parsing. The first streaming implementation bounded memory, but profiling a 500 MiB phase-heavy stress log still took 226.97 seconds because every line ran many independent Foundation string searches and parsers whose output was not requested.

The final implementation parses while stdin arrives, scans each line once for relevant categories, and avoids unnecessary work without changing output semantics.

Performance

Reference run on arm64 macOS 26.6 with Swift 6.3.3. Primary figures are medians of three release runs:

Workload	Input	Before	After	Throughput	Peak RSS
phase-heavy	10 MiB	4.80 s	0.02 s	500.00 MiB/s	8.70 MiB
phase-heavy	100 MiB	44.58 s	0.18 s	555.56 MiB/s	8.75 MiB
phase-heavy	500 MiB	226.97 s	0.87 s	574.71 MiB/s	8.75 MiB
VideoGo-shaped	100 MiB	-	0.23 s	434.78 MiB/s	20.45 MiB
VideoGo-shaped	500 MiB	-	1.10 s	454.55 MiB/s	20.47 MiB

The 500 MiB phase-heavy stress case is about 261 times faster. The VideoGo-shaped profile uses the measured log mix (about 0.13% phase lines and 9.43% warning lines), so it is more representative than the intentionally pathological all-phase fixture.

Additional 500 MiB checks completed in 0.72 s (fast-reject), 0.94 s (fast-reject-unicode), 0.63 s (fixture-mixed), and 3.41 s (warning-duplicate). Exact distinct-warning state still scales with the number of unique diagnostics; warning-unique used 265.92 MiB RSS at 100 MiB.

TDD and correctness

• added characterization tests before each refactor for streaming/chunk boundaries, warning identity and locations, JSON-like input, xcbeautify hints, linker state, Swift Testing lifecycle lines, and Unicode character boundaries
• compared normalized JSON and byte-identical TOON output against the pre-optimization streaming binary on real build, linker, Swift Testing, and warning fixtures
• independently differential-tested line framing across 20,013 fixed/random byte streams and chunk partitions
• stopped optimizing after profiling showed the remaining large hotspot was the global candidate scan with no small exact-semantics replacement; all other individually removable work was below the 5% threshold

Verification

• swift format lint --strict --recursive .
• swift test - 423 tests, 0 failures
• swift build -c release
• bash -n Benchmarks/large-log.sh
• all seven benchmark profiles smoke-tested
• Benchmarks/large-log.sh 10 100 500 for phase-heavy and VideoGo-shaped inputs, three runs each

Scope

The referenced analysis also recommends VideoGo gate/process changes. Those belong to the consuming project rather than this repository and are intentionally not part of this PR.

[alexey1312]

Thanks for this, really solid work 🚀.