Skip to content

test(branch): fix two flaky tests (snapshot race, witness port) - #231

Merged
SauravYadav12 merged 4 commits into
mainfrom
fix/snapshot-test-race
Nov 11, 2026
Merged

SauravYadav12 merged 4 commits into
mainfrom
fix/snapshot-test-race

Conversation

@SauravYadav12

@SauravYadav12 SauravYadav12 commented Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes two flaky unit tests that fail CI on PRs that never touched them. The first just failed #230, whose change is in enterprise/masking only; the second failed this PR's own first CI run.

1. TestSnapshotServesThePreviousAnswerRatherThanHang (race detector)

Cause: TestSnapshotServesThePreviousAnswerRatherThanHang (added in #222) ended while its deliberately hung read was still running. The deferred close(gate) let that read finish as the test ended. As it finished, Snapshot.fetch read stateNow while withFakeDocker's cleanup was writing the real clock back into it, which the race detector reports. Reproduced on main: 2 races in 300 runs.

Fix: the test only. A cleanup registered after withFakeDocker's, so it runs before it, lets the read go and waits for it to finish. Nothing outside the test changed.

2. TestWitnessAnswersOneQuestion

Cause: it closed a listener and took the freed port as a primary that is down. The httptest servers it started next were sometimes handed that very port, so the "down" primary answered. CI failed with view = the witness reaches the primary.

Fix: the down primary is 127.0.0.1:0. Nothing can listen on port 0, so connecting fails at once on Linux, macOS and Windows, and the witness treats any failed connection as down. The one other test using the freed-port pattern (internal/controlplane/stale_test.go) already skips when something answers there, so it is left alone.

Tests

  • The test 1000 times under -race: clean.
  • internal/branch 20 times under -race: clean.
  • The console's and the Gateway's snapshot and stale tests (internal/controlplane, internal/proxy) 10 times under -race: clean.
  • The witness test 500 times on macOS and 500 times on Linux (the test VM): clean. Vet for Windows.

…ut back

TestSnapshotServesThePreviousAnswerRatherThanHang ended with its hung read
still running; let go as the test ended, it read the test's clock while
withFakeDocker's cleanup restored it, which go test -race caught now and then
(2 in 300 runs) and failed CI on unrelated PRs.
@vercel

vercel Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
foxbyte-get Ready Ready Preview Nov 11, 2026 5:13pm UTC

…reed

TestWitnessAnswersOneQuestion closed a listener and took its port for a
primary that is down, but the test servers it started next were sometimes
handed that very port, so the down primary answered (seen in CI).
@SauravYadav12 SauravYadav12 changed the title test(branch): let the hung snapshot read finish before the clock is put back test(branch): fix two flaky tests (snapshot race, witness port) Nov 11, 2026
@SauravYadav12
SauravYadav12 merged commit e3ea9a7 into main Nov 11, 2026
8 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators Nov 11, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant