Skip to content

Commit 565d7af

Browse files
rubenvdlindeclaude
andcommitted
test(flow): cover the heartbeat recovery paths that no test executed
The changed-files coverage ratchet on #3358 was right, and it was pointing at something real rather than at a percentage. Of the 32 statements that change added, 30 were the body of `FlowTaskBridge::recordHeartbeatRecovery()` -- and every test that exercised the recovery MOCKED FlowTaskBridge, because in those tests the nodes are the unit. So the method that writes the recovery's audit trail had no execution coverage at all: the audit entry, its attribution, and the catch that makes it best-effort were asserted nowhere. That matters more than the percentage does. The guarded signal seam records a refusal; this entry is the other half of that trail, and a silent regression in it would make a recovered answer read as one that vanished. Three tests through the REAL bridge, in the suite that already builds one: - the entry is recorded as `heartbeat-recovered`, attributed to the task's completedBy, with a reason naming the run whose signal never arrived; - an audit write that THROWS is swallowed, because the recovery is the node applying the outcome and letting the failure out would abort the very walk that was un-wedging the run; - an ending nobody answered (terminated, expired -- `completedBy` is null on exactly those) records no actor rather than a guessed one. And one test for the symmetric case the change documented but left unpinned: a completion that RACED the suspension. `signal()` refuses a run that is not suspended, so that wake is lost with nothing to retry it; the test asserts the refusal, asserts the run parks on a non-null heartbeat, and asserts the next wake recovers it. That is the whole basis for deciding the race needs no new mechanism, and it is now falsifiable. Every one of the four was checked by mutation -- breaking the action name, the attribution, the catch, or the recovery call itself turns each red. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent 925c204 commit 565d7af

2 files changed

Lines changed: 144 additions & 0 deletions

File tree

tests/Unit/Service/Flow/FlowHeartbeatRecoveryTest.php

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -511,4 +511,49 @@ public function testOnlyTheNodeWhoseTaskEndedRecovers(): void {
511511
$this->assertArrayNotHasKey('askA', $kept, 'a node that answered has nothing left to remember');
512512
$this->assertSame($taskB, (string)($kept['askB']['taskUuid'] ?? ''), 'the waiting sibling keeps its own task');
513513
}//end testOnlyTheNodeWhoseTaskEndedRecovers()
514+
515+
/**
516+
* 🔴 THE SYMMETRIC CASE, PINNED: a task completed while the run was NOT yet
517+
* suspended. `signal()` refuses any run that is not `suspended`, so that
518+
* completion's wake is simply LOST — there is no queue for it, and nothing
519+
* retries it. The design decided this needs no new mechanism, and this test
520+
* is what makes that decision falsifiable: the lost wake must cost latency
521+
* only, because the node parks on a NON-NULL heartbeat and the next wake
522+
* re-reads the task.
523+
*
524+
* Were the heartbeat ever allowed to be null here, this run would be
525+
* unreachable forever — `findDue()` never returns a run with a null
526+
* `resume_at` — which is exactly the trap `UserTaskNode` documents.
527+
*
528+
* @return void
529+
*
530+
* @spec openspec/changes/flow-heartbeat-recovery/specs/flow-heartbeat-recovery/spec.md#requirement-a-heartbeat-wake-re-reads-the-awaited-task-and-applies-a-terminal-outcome
531+
*/
532+
public function testACompletionThatRacedTheSuspensionIsRecoveredByTheHeartbeat(): void {
533+
$run = $this->suspendedOnBothTasks();
534+
$slots = ($run->getContext()['resumeState'] ?? []);
535+
$taskA = (string)$slots['askA']['taskUuid'];
536+
537+
// THE RACE: the task completes while the run is still mid-walk. The
538+
// completion listener calls signal(), which refuses a non-suspended
539+
// run — so the wake is lost and nothing queues a retry of it.
540+
$this->complete(uuid: $taskA, completedBy: 'bob');
541+
$run->setStatus(FlowRun::STATUS_RUNNING);
542+
$this->assertNull(
543+
$this->service->signal($run, []),
544+
'a run that is not suspended refuses the signal, so the completion wake is lost'
545+
);
546+
547+
// The walk finishes and the run parks — on a heartbeat that is NEVER
548+
// null, which is the only reason the lost wake is recoverable at all.
549+
$run->setStatus(FlowRun::STATUS_SUSPENDED);
550+
$this->assertNotNull($run->getResumeAt(), 'a task-waiting run must park on a clock, never on a signal alone');
551+
552+
// The next heartbeat re-reads the task and applies the outcome.
553+
$run = $this->service->execute($run, $this->flow(), new HeartbeatSubject());
554+
555+
$this->assertSame([$taskA], $this->recovered, 'the raced completion is recovered, and audited as a recovery');
556+
$this->assertSame(['askA', 'askB'], $this->created, 'recovery never creates a task');
557+
}//end testACompletionThatRacedTheSuspensionIsRecoveredByTheHeartbeat()
558+
514559
}//end class

tests/Unit/Service/Flow/FlowTaskBridgeTest.php

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -332,4 +332,103 @@ public function testTheBagSeparatesADecisionFromAnEnding(): void {
332332
$this->assertFalse($bag['rejected']);
333333
$this->assertSame(Task::STATE_TERMINATED, $bag['outcome'], 'an ending with no outcome reports its state');
334334
}//end testTheBagSeparatesADecisionFromAnEnding()
335+
336+
// ---- Heartbeat recovery -------------------------------------------------------
337+
338+
/**
339+
* 🔴 THE OTHER HALF OF THE REFUSAL TRAIL. The guarded signal seam records
340+
* that a completion was refused; without this entry the trail ends there
341+
* and a recovered answer reads as one that vanished. Attributed to the
342+
* task's COMPLETER, because the fact being recorded is that person's
343+
* answer arriving late by poll — not the cron job acting.
344+
*
345+
* Driven through the REAL bridge. Every other test of this behaviour mocks
346+
* FlowTaskBridge (the nodes are the unit there), so this method's body had
347+
* no execution coverage at all until this test.
348+
*
349+
* @return void
350+
*
351+
* @spec openspec/changes/flow-heartbeat-recovery/specs/flow-heartbeat-recovery/spec.md#requirement-a-heartbeat-recovered-delivery-is-recorded-on-the-tasks-audit
352+
*/
353+
public function testAHeartbeatRecoveryIsAuditedToTheTasksCompleter(): void {
354+
$task = $this->terminalTask();
355+
$task->setCompletedBy('bob');
356+
357+
$seen = [];
358+
$this->tasks->expects($this->once())
359+
->method('record')
360+
->willReturnCallback(
361+
function (string $uuid, string $action, ?string $actor, string $reason) use (&$seen, $task): Task {
362+
$seen = ['uuid' => $uuid, 'action' => $action, 'actor' => $actor, 'reason' => $reason];
363+
364+
return $task;
365+
}
366+
);
367+
368+
$this->bridge->recordHeartbeatRecovery(task: $task);
369+
370+
$this->assertSame('t-1', $seen['uuid']);
371+
$this->assertSame('heartbeat-recovered', $seen['action']);
372+
$this->assertSame('bob', $seen['actor'], 'the recovery is the completer\'s answer arriving, not the worker\'s');
373+
$this->assertStringContainsString('run-1', $seen['reason'], 'the reason names the run whose signal never arrived');
374+
}//end testAHeartbeatRecoveryIsAuditedToTheTasksCompleter()
375+
376+
/**
377+
* 🔴 BEST-EFFORT, AND THAT IS THE POINT. The recovery itself is the node
378+
* applying the outcome; this entry only describes it. An audit write that
379+
* fails must therefore NOT propagate — letting it out would abort the walk
380+
* that was recovering the run and put the run straight back into the wedge
381+
* this whole change exists to remove.
382+
*
383+
* @return void
384+
*
385+
* @spec openspec/changes/flow-heartbeat-recovery/specs/flow-heartbeat-recovery/spec.md#requirement-a-heartbeat-recovered-delivery-is-recorded-on-the-tasks-audit
386+
*/
387+
public function testAFailedRecoveryAuditIsSwallowedSoTheRecoveredRunStands(): void {
388+
$task = $this->terminalTask();
389+
$task->setCompletedBy('bob');
390+
391+
$this->tasks->expects($this->once())
392+
->method('record')
393+
->willThrowException(new RuntimeException('the audit table is unavailable'));
394+
395+
$this->bridge->recordHeartbeatRecovery(task: $task);
396+
397+
// Reached only because nothing propagated: the recovery outlives its
398+
// own audit failure.
399+
$this->addToAssertionCount(1);
400+
}//end testAFailedRecoveryAuditIsSwallowedSoTheRecoveredRunStands()
401+
402+
/**
403+
* A task that ended WITHOUT a completer — terminated or expired rather than
404+
* answered — still records its recovery, with no actor rather than an
405+
* invented one. `completedBy` is null on exactly those endings, and an
406+
* audit that guessed a name there would be worse than one that admits it
407+
* has none.
408+
*
409+
* @return void
410+
*
411+
* @spec openspec/changes/flow-heartbeat-recovery/specs/flow-heartbeat-recovery/spec.md#requirement-a-heartbeat-recovered-delivery-is-recorded-on-the-tasks-audit
412+
*/
413+
public function testARecoveredEndingWithNoCompleterRecordsNoActor(): void {
414+
$task = $this->terminalTask();
415+
$task->setState(Task::STATE_TERMINATED);
416+
$task->setCompletedBy(null);
417+
418+
$actor = 'unset';
419+
$this->tasks->expects($this->once())
420+
->method('record')
421+
->willReturnCallback(
422+
function (string $uuid, string $action, ?string $seenActor, string $reason) use (&$actor, $task): Task {
423+
$actor = $seenActor;
424+
425+
return $task;
426+
}
427+
);
428+
429+
$this->bridge->recordHeartbeatRecovery(task: $task);
430+
431+
$this->assertNull($actor, 'an ending nobody answered is recorded with no actor, never a guessed one');
432+
}//end testARecoveredEndingWithNoCompleterRecordsNoActor()
433+
335434
}//end class

0 commit comments

Comments
 (0)