fix: follow-up cleanups for the streaming parser - #81
Merged
Conversation
- raise the input line limit to 64 KiB, because the 5,000 limit now counts bytes and dropped non-ASCII diagnostics that parsed before - split complete input on the newline byte, because Swift reads "\r\n" as one Character and CRLF logs parsed as a single line - report ignored oversized lines on stderr instead of dropping them silently - scan for the tested target only when the line holds ".xctest" - share one location parser between error and warning parsing - keep one warning identity type instead of two
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #79. Same measured speed, minus the sharp edges I found while reviewing it.
Fixes
The line limit dropped non-ASCII diagnostics. #79 changed the 5,000 limit from characters to UTF-8 bytes and moved it into the reader, so a Cyrillic or CJK message spends two to three bytes per character and a diagnostic that parsed before is now ignored:
The limit is now 64 KiB. Memory stays bounded, and a real diagnostic fits.
Oversized lines were dropped silently.
InputScan.oversizedLinesDroppedwas counted but never read. It now goes to stderr:CRLF logs still broke the in-memory API. The new reader frames lines on the newline byte, so the CLI is fine, but
OutputParser.parse(input:)still calledsplit(separator: "\n"). Swift reads"\r\n"as oneCharacter, so that split never matches and the whole log stays one line. On a CRLF log with 300 errors the library API reportsincompletewith 0 errors. It now splits on the newline byte, same as the CLI.--coveragescanned every line.extractTestedTarget(fromLine:)ran two Foundation searches per line until it found a test suite, which never happens in a build-only log. On a 100 MiB log that cost 6 seconds on top of a 1.8 second parse. A.xctestbyte check in front of it brings parsing back to 1.7 seconds.Cleanups
parseErrorandparseWarning. feat: stream and accelerate large build logs #79 rewrote the warning path by hand and left the error path onsplit/joined, so the same job had two implementations.CompactWarningKeyis gone.WarningKeywas already there and alreadyHashable, and it holds the existing strings instead of building a new one per warning.hasPotentialLinkerPrefixusesUInt8(ascii:)instead of raw hex. The byte forfwas unreachable, every framework and library pattern starts withld:.Verification
swift test: 427 tests, 0 failuresswift format lint --strict --recursive .masteron all three fixtures across six flag combinations and on a 100 MiB synthetic logOutputParser, invalid UTF-8 bytes in the reader, exact-byte error markers, a long non-ASCII diagnostic