fix: count XCTest suite totals once per run - #84
Merged
Conversation
XCTest prints the same "Executed N tests" line at each suite level: the nested suite, the .xctest bundle, and the "Selected tests"/"All tests" wrapper. The parser added the wrapper count to the bundle count. A filtered run thus reported twice the tests and twice the failures. The parser also added every duration line. It reported three times the real test time. Count the .xctest bundles only. Read the run duration from the same suite level as the counts. Fixes #83
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
XCTest prints the same
Executed N testsline at each suite level: the nested suite, the.xctestbundle, and theSelected tests/All testswrapper.StreamingOutputParseradded the wrapper count to the bundle count.+=is correct for two.xctestbundles, because their tests are disjoint. The wrapper is not a sibling bundle — it contains the bundles, so the same tests were counted twice:A full run prints
All tests, which failed the same condition and fell into the ignored fallback. This hid the defect outside of--filter.The parser also added the duration of every summary line, so
test_timecounted each level again.Fix
.xctestbundles only. The wrapper falls into the fallback slot, which the parser reads only when no bundle line exists — and there the wrapper is the last summary line, so the count stays correct.resolvedXCTestDuration()).testTimeAccumulatortoswiftTestingTimeAccumulator, because it now holds the Swift Testing time only.selectedTestsSuiteconstant, which lost its last reference.Result
swift test --filter …(2 tests)passed_tests: 4,test_time: 0.006s2,0.003sswift test(full)435,~45s435,15.157sA filtered run with failures reported twice the failures. It now reports the true count.
Tests
Four regression cases in
Tests/XCSiftCoreTests/ParsingTests.swift:test_timeacross suite levels.xctestbundles still add up (guards the disjoint case)Full suite: 435 tests, 0 failures.
Fixes #83