Skip to content

e2e: fix the Go watch-set loader reconnect flake - #919

Merged
bkeroack merged 1 commit into
release/0.6from
fix/go-e2e-watchset-reconnect-flake
Oct 6, 2026
Merged

bkeroack merged 1 commit into
release/0.6from
fix/go-e2e-watchset-reconnect-flake

Conversation

@bkeroack

@bkeroack bkeroack commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

Fixes the Go e2e flake TestE2EWatchSetLoaderRebuildsFromTruthOnReconnect, which fails with resilient watch Next (saw map[Heartbeat:180]): context deadline exceeded at resilientwatch_test.go:249. It was in 11 of the 18 failed ci.yml runs from 2026-09-24 to today, on master pushes as well as PRs, and failed 7 of 10 runs locally. With a release-branch CI trigger coming (#913), it would turn most release/0.6 runs red for no reason.

Cause

The test's bug, not the SDK's. The priming loop waits for the first ScriptMatched for its payment, which is usually the mempool match. An unconfirmed match carries no cursor: the server sends none for a height-less match (watch_match_to_proto_sp_unconfirmed_has_no_cursor pins this). The test then cuts the transport. If the confirmed match from the block that payTo mines has not reached the pump by then, ResilientWatch holds no resume cursor. connectOnce sends SetCursor only when it has one, so no CursorAccepted ever comes, and the wait for it sees only heartbeats until the deadline.

Logging w.ResumeCursor() and the proxy's dial count at the cut, over six runs:

Result Resume cursor at the cut Reconnects Loader runs
3 failed nil 1 2
3 passed {Height:141 MempoolSeq:144 …} 1 2

The reconnect and the loader worked in every run; only the anchor was missing.

primeRW, used by the sibling test, already waits for the confirmed match, with a comment explaining this exact failure. This test kept the older loop.

Fix

The priming loop also requires m.Confirmed. The reload test has the same loop but never cuts the transport, so it needs no cursor and is unchanged.

Verified

  • The test alone: 30 of 30 runs pass (go test -tags e2e -count=10, then -count=20), against 3 of 10 before.
  • The full Go e2e suite passes once. go vet -tags e2e and clients/go/lint.sh are clean.

Test-only, so no changelog entry. A forward-port to master follows once this merges.

🤖 Generated with Claude Code

…ream

TestE2EWatchSetLoaderRebuildsFromTruthOnReconnect failed most runs (7 of
10 locally, and in 11 of the 18 failed CI runs since 2026-09-24). Its
priming loop stopped at the first ScriptMatched for its payment, which is
usually the mempool match, and an unconfirmed match carries no cursor.
The test then cut the transport, often before the confirmed match from
the block payTo mines had reached the pump. The watch was left with no
resume cursor, so the reconnect sent no SetCursor and the wait for
CursorAccepted saw only heartbeats until its deadline. Every failing run
had a nil ResumeCursor at the cut; every passing run had one.

Wait for the confirmed match, as primeRW already does for the same
reason. 30 of 30 local runs pass.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@bkeroack
bkeroack merged commit 9c7eb02 into release/0.6 Oct 6, 2026
34 checks passed
@bkeroack
bkeroack deleted the fix/go-e2e-watchset-reconnect-flake branch October 6, 2026 21:20
bkeroack added a commit that referenced this pull request Oct 6, 2026
…ream (#919)

TestE2EWatchSetLoaderRebuildsFromTruthOnReconnect failed most runs (7 of
10 locally, and in 11 of the 18 failed CI runs since 2026-09-24). Its
priming loop stopped at the first ScriptMatched for its payment, which is
usually the mempool match, and an unconfirmed match carries no cursor.
The test then cut the transport, often before the confirmed match from
the block payTo mines had reached the pump. The watch was left with no
resume cursor, so the reconnect sent no SetCursor and the wait for
CursorAccepted saw only heartbeats until its deadline. Every failing run
had a nil ResumeCursor at the cut; every passing run had one.

Wait for the confirmed match, as primeRW already does for the same
reason. 30 of 30 local runs pass.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
(cherry picked from commit 9c7eb02)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant