Skip to content

fix(engine): resume replays completed sessions persisted at DEV_VERIFY/REVIEW_VERIFY instead of demanding a rollback#116

Merged
pbean merged 1 commit into
mainfrom
fix/100-resume-replay-dev-verify
Jul 10, 2026
Merged

fix(engine): resume replays completed sessions persisted at DEV_VERIFY/REVIEW_VERIFY instead of demanding a rollback#116
pbean merged 1 commit into
mainfrom
fix/100-resume-replay-dev-verify

Conversation

@pbean

@pbean pbean commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

Fixes #100. Refs #99 (closed duplicate), #115 (follow-up).

The bug

A host death (crash/OOM/kill) in the post-verify decision window persists the task at DEV_VERIFY — the _save() right after the verify/decide pass — with the completed dev session's record (status: completed, result_json.status: done) already durably saved, but the decision's action unapplied and commit_sha still null. task.spec_file is only set on full verify success, so when verify had not passed at death, resume found:

  • arm A (DEV_VERIFY and task.spec_file_resume_after_dev_verify): skipped, no spec_file
  • arm B (_resumable_session replay): skipped — the matcher only knew DEV_RUNNING/REVIEW_RUNNING
  • arm C: resume-restart → _rollback_or_pause(cause="stopped") → with rollback OFF (the default) a pause whose notice instructs a bare git reset --hard <baseline> — past the attempt's finished, possibly already-pushed commits, without ever consulting the recorded evidence or git.

Note the routing never reads commit_sha (the issue's suggested trigger) — it is write-only bookkeeping; the actual keys are phase + spec_file.

The fix

Routing: _resumable_session now also matches DEV_VERIFY → (dev, task.attempt) and REVIEW_VERIFY → (review, task.review_cycle). The existing replay arms re-enter the normal reconcile → verify → decide pipeline: the persisted baseline is kept, no attempt/cycle budget is burned, and completion authority is unchanged (only a completed record with a recorded result replays — anything less still takes resume-restart). When verify passes on replay the story proceeds to review/commit and commit_sha is stamped; when the original failure reproduces, the normal retry/defer/escalate semantics apply — behavior parity with a live run, minus the destructive instruction.

Notice hardening: the rollback-OFF manual-recovery notice now probes verify.commits_above(baseline) (advisory, GitError-safe). When the attempt committed work, the notice leads with saving the commits (git branch my-rescue HEAD, names <baseline>..HEAD) and checking whether they are already integrated or pushed — a reset is only ever the last, optional step. This also covers the notice-wording ask from #99 and de-fangs the COMMITTING-window sibling tracked as #115.

Scope

  • REVIEW_VERIFY is the same crash window one phase later (persisted by the save right after the review session, before the decision acts); it shares the two-line matcher fix and is covered by its own test.
  • The COMMITTING window is real but needs different machinery (no session record exists to replay on the skip-review path) — split out as resume-restart discards verified work when the host dies in the COMMITTING window (no session record to replay) #115.
  • SweepEngine hand-rolls bundle recovery outside _finish_inflight (documented there as a resume-fidelity change of its own); it does inherit the hardened notice.

Tests

  • test_resume_dev_verify_replays_recorded_dev_result_to_done — the [BUG] bmad-loop resume requests rollback of already-committed work when orchestrator dies after successful dev session #100 regression: hand-built crash state in the issue's exact shape (phase dev-verify, empty spec_file, completed/done record, commits above baseline, rollback OFF) resumes to DONE with nothing re-run, no rollback journal, commit_sha stamped, work preserved.
  • test_resume_dev_verify_replay_verify_still_failing_retries_normally — a reproducing verify failure re-enters the budgeted retry path with the commits parked on a recovery ref.
  • test_resume_dev_verify_record_incomplete_still_restarts — stalled / result-less records at DEV_VERIFY keep resume-restart.
  • test_resume_review_verify_replays_recorded_review — crash injected after the REVIEW_VERIFY save; resume replays to DONE, review_cycle unburned.
  • test_resume_dev_verify_replay_stories_mode — stories-mode parity (replay after operator repair, no plan-halt leg triggered).
  • test_manual_recovery_notice_names_committed_work / test_manual_recovery_notice_probe_failure_falls_back — the new notice branch and its advisory-probe fallback.

All new tests fail without the fix. Full suite: 2058 passed, 1 skipped.

E2E validation

Drove the real bmad-loop resume CLI against a sandbox repo whose state.json was hand-written to the reporter's diagnostic dump shape (real git repo, attempt commit above baseline, finalized spec on disk, fake-CLI profile): the run resumed straight to story-done — journal resume-verifystory-done, commit_sha stamped, the attempt's work squash-committed, and no reset --hard instruction anywhere in the output.

Summary by CodeRabbit

  • Bug Fixes
    • Resuming after a crash during verification now correctly replays recorded results without rerunning completed work.
    • Completed sessions no longer trigger unnecessary rollback prompts.
    • Resume behavior now supports both development and review verification stages.
    • Manual recovery notices preserve and report committed work when automatic rollback is disabled.
    • Improved fallback messaging when commit information cannot be checked.
  • Documentation
    • Updated the changelog with the verification and recovery fixes.

…VERIFY instead of demanding rollback

A host death in the post-verify decision window persists the task at
DEV_VERIFY (or REVIEW_VERIFY one phase later) with the completed session's
record durably saved but the decision's action unapplied. The resume replay
matcher only knew the *_RUNNING phases, so those tasks fell through to
resume-restart -> _rollback_or_pause, pausing with a bare
'git reset --hard <baseline>' instruction that would discard the attempt's
finished -- possibly already-pushed -- commits.

_resumable_session now matches DEV_VERIFY (dev, attempt) and REVIEW_VERIFY
(review, review_cycle); the existing replay arms re-enter the normal
reconcile -> verify -> decide pipeline with no budget burned and the
persisted baseline kept. The spec-approval-gate arm stays first, so a
DEV_VERIFY task with a verified spec keeps _resume_after_dev_verify.

The rollback-OFF manual-recovery notice now probes commits_above(baseline)
and, when the attempt committed work, leads with saving/checking those
commits (they may already be pushed) instead of instructing a blind reset;
the probe is advisory and falls back to the classic notice on a git fault.

Fixes #100. Refs #99.
@coderabbitai

coderabbitai Bot commented Jul 10, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: eeab0745-3801-4a4f-84a9-5a8b27a8b336

📥 Commits

Reviewing files that changed from the base of the PR and between 120999c and f362d0e.

📒 Files selected for processing (4)
  • CHANGELOG.md
  • src/bmad_loop/engine.py
  • tests/test_engine.py
  • tests/test_stories_engine.py

Walkthrough

Resume now replays durably recorded dev or review results from verify phases, while rollback-off manual recovery identifies committed work above baseline and reports it without resetting the tree.

Changes

Resume and recovery behavior

Layer / File(s) Summary
Verify-phase resume replay
src/bmad_loop/engine.py, tests/test_engine.py, tests/test_stories_engine.py, CHANGELOG.md
Completed dev and review session records are replayed from DEV_VERIFY and REVIEW_VERIFY; tests cover terminal replay, retries, incomplete records, stories mode, and changelog documentation.
Committed-work manual recovery
src/bmad_loop/engine.py, tests/test_engine.py
Rollback-off recovery detects commits above baseline, includes preservation guidance and commit counts, and falls back to the existing wording when Git probing fails.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related issues

Possibly related PRs

Sequence Diagram(s)

sequenceDiagram
  participant Operator
  participant Engine
  participant SessionRecord
  participant VerifyDecide
  Operator->>Engine: resume persisted task
  Engine->>SessionRecord: load completed result_json
  Engine->>VerifyDecide: replay recorded result
  VerifyDecide-->>Engine: return verify/decide outcome
  Engine-->>Operator: finish or retry task
Loading

Poem

I’m a rabbit with commits in a row,
No hard reset makes my garden low.
Results replay, the verify bells ring,
Saved work stays safe through everything.
Hop, hop—resume completes the spring! 🐇

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main fix: resume now replays completed DEV_VERIFY/REVIEW_VERIFY sessions instead of forcing rollback.
Linked Issues check ✅ Passed The PR addresses #100 by recognizing completed post-verify sessions and advancing through normal replay instead of issuing rollback.
Out of Scope Changes check ✅ Passed All changes support the resume/rollback fix and its regression tests; no unrelated code paths or features were added.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/100-resume-replay-dev-verify

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@pbean pbean merged commit 658f67d into main Jul 10, 2026
8 checks passed
@pbean pbean deleted the fix/100-resume-replay-dev-verify branch July 10, 2026 20:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] bmad-loop resume requests rollback of already-committed work when orchestrator dies after successful dev session

1 participant