Skip to content

fix: follow-up cleanups for the streaming parser - #81

Merged
alexey1312 merged 1 commit into
masterfrom
fix/streaming-parser-followups
Aug 18, 2026
Merged

fix: follow-up cleanups for the streaming parser#81
alexey1312 merged 1 commit into
masterfrom
fix/streaming-parser-followups

Conversation

@alexey1312

Copy link
Copy Markdown
Collaborator

Follow-up to #79. Same measured speed, minus the sharp edges I found while reviewing it.

Fixes

The line limit dropped non-ASCII diagnostics. #79 changed the 5,000 limit from characters to UTF-8 bytes and moved it into the reader, so a Cyrillic or CJK message spends two to three bytes per character and a diagnostic that parsed before is now ignored:

main.swift:1:1: error: <3000 x "é">     before #79: errors: 1     after #79: errors: 0

The limit is now 64 KiB. Memory stays bounded, and a real diagnostic fits.

Oversized lines were dropped silently. InputScan.oversizedLinesDropped was counted but never read. It now goes to stderr:

hint: Ignored 1 input line(s) longer than 65536 bytes.

CRLF logs still broke the in-memory API. The new reader frames lines on the newline byte, so the CLI is fine, but OutputParser.parse(input:) still called split(separator: "\n"). Swift reads "\r\n" as one Character, so that split never matches and the whole log stays one line. On a CRLF log with 300 errors the library API reports incomplete with 0 errors. It now splits on the newline byte, same as the CLI.

--coverage scanned every line. extractTestedTarget(fromLine:) ran two Foundation searches per line until it found a test suite, which never happens in a build-only log. On a 100 MiB log that cost 6 seconds on top of a 1.8 second parse. A .xctest byte check in front of it brings parsing back to 1.7 seconds.

Cleanups

  • One location parser shared by parseError and parseWarning. feat: stream and accelerate large build logs #79 rewrote the warning path by hand and left the error path on split/joined, so the same job had two implementations.
  • CompactWarningKey is gone. WarningKey was already there and already Hashable, and it holds the existing strings instead of building a new one per warning.
  • hasPotentialLinkerPrefix uses UInt8(ascii:) instead of raw hex. The byte for f was unreachable, every framework and library pattern starts with ld: .
  • CLAUDE.md and DocC describe the streaming reader and the line limit.

Verification

  • swift test: 427 tests, 0 failures
  • swift format lint --strict --recursive .
  • Output is byte-identical to master on all three fixtures across six flag combinations and on a 100 MiB synthetic log
  • 100 MiB run: 1.78 s and 95 MiB peak RSS, against 1.80 s and 95 MiB for feat: stream and accelerate large build logs #79 alone
  • New tests: CRLF input through OutputParser, invalid UTF-8 bytes in the reader, exact-byte error markers, a long non-ASCII diagnostic

- raise the input line limit to 64 KiB, because the 5,000 limit now counts
  bytes and dropped non-ASCII diagnostics that parsed before
- split complete input on the newline byte, because Swift reads "\r\n" as one
  Character and CRLF logs parsed as a single line
- report ignored oversized lines on stderr instead of dropping them silently
- scan for the tested target only when the line holds ".xctest"
- share one location parser between error and warning parsing
- keep one warning identity type instead of two
@alexey1312
alexey1312 merged commit 4b578ab into master Aug 18, 2026
2 checks passed
@alexey1312
alexey1312 deleted the fix/streaming-parser-followups branch August 18, 2026 05:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant