Summary
tests/chaos/property2_test.go gates its SIGKILL on waitForReadCount — a lagging, parent-side
observation of a free-running child — and then asserts kill-point preconditions that only hold if
the kill actually landed where the parent last looked. It doesn't reliably.
This is the same defect fixed for TestSIGKILL_RecoveryLoop_CrashDuringRecoveredRun in #2835, and
for sigkill_test.go's mid-snapshot case in #2534. property2_test.go never received either
treatment — it has no persistDelayMS, and its comments still carry the superseded arithmetic
("~30ms in, far short of the persister's FIRST automatic flush (~1000ms)").
Affects all three cases across both prune classes — 6 subtests.
Evidence
With a 1.5s stall injected between the read-count wait and the kill:
| case |
result |
mid-handoff |
expected RESUME_POSITION to be empty/fresh …, got 100 |
mid-position-write |
expected RESUME_POSITION (128) to be STALE - strictly behind the kill point (read #95) |
mid-snapshot |
hangs — timed out waiting for child to exit, 41.6s |
mid-snapshot is the worst of the three: it doesn't fail fast, it starves. That is the hang
sigkillCase's own doc comment describes — a full resume leaves produceLoop nothing to read. On a
required status check (tests/chaos (race, x3)) that is a 40s stall per affected subtest before
anything goes red.
Proposed fix
Mostly mechanical, and the machinery already exists:
Whatever lands should keep the vacuity guards intact. They are the good part: they are what turns
"this test silently stopped testing anything" into a visible failure.
Not affected
fanout_sigkill_test.go:65 and nsource_sigkill_test.go:102 also gate kills on waitForReadCount,
but their post-kill assertions don't depend on the child being mid-run — they restart and assert
gapless delivery through total. They degrade to silently weaker coverage (the crash may not be
mid-flight), not to red. fanout's own comment already labels its pre-kill check "Diagnostic only
(best-effort)". Worth tightening eventually; not a flake source.
Acceptance criteria
Related: #2835, #2534, #2832, #2834
Summary
tests/chaos/property2_test.gogates its SIGKILL onwaitForReadCount— a lagging, parent-sideobservation of a free-running child — and then asserts kill-point preconditions that only hold if
the kill actually landed where the parent last looked. It doesn't reliably.
This is the same defect fixed for
TestSIGKILL_RecoveryLoop_CrashDuringRecoveredRunin #2835, andfor
sigkill_test.go'smid-snapshotcase in #2534.property2_test.gonever received eithertreatment — it has no
persistDelayMS, and its comments still carry the superseded arithmetic("~30ms in, far short of the persister's FIRST automatic flush (~1000ms)").
Affects all three cases across both prune classes — 6 subtests.
Evidence
With a 1.5s stall injected between the read-count wait and the kill:
mid-handoffexpected RESUME_POSITION to be empty/fresh …, got 100mid-position-writeexpected RESUME_POSITION (128) to be STALE - strictly behind the kill point (read #95)mid-snapshottimed out waiting for child to exit, 41.6smid-snapshotis the worst of the three: it doesn't fail fast, it starves. That is the hangsigkillCase's own doc comment describes — a full resume leavesproduceLoopnothing to read. On arequired status check (
tests/chaos (race, x3)) that is a 40s stall per affected subtest beforeanything goes red.
Proposed fix
Mostly mechanical, and the machinery already exists:
childConfig.persistDelayMSis already wired, so the tworesumeEmptycases (mid-handoff,mid-snapshot) are the Flaky test suite: multiple order/timing-dependent failures make CI green unreliable #2534 one-liner — make the precondition structurally true with a longpersister debounce instead of a racing margin.
mid-position-writeneeds the ceiling treatment from fix(chaos): bound the recovery kill point by construction, not by a race #2835, because its guard isresumePos < killAfterReads— an upper bound that a stalled parent can blow through.Whatever lands should keep the vacuity guards intact. They are the good part: they are what turns
"this test silently stopped testing anything" into a visible failure.
Not affected
fanout_sigkill_test.go:65andnsource_sigkill_test.go:102also gate kills onwaitForReadCount,but their post-kill assertions don't depend on the child being mid-run — they restart and assert
gapless delivery through
total. They degrade to silently weaker coverage (the crash may not bemid-flight), not to red.
fanout's own comment already labels its pre-kill check "Diagnostic only(best-effort)". Worth tightening eventually; not a flake source.
Acceptance criteria
mid-snapshotfails fast rather than starving, if its precondition is ever violatedRelated: #2835, #2534, #2832, #2834