Skip to content

[mcp] fix: retain cleanup debt after failed start - #292

Merged
Wangmerlyn merged 1 commit into
mainfrom
codex/startup-cleanup-retention
Jul 2, 2026
Merged

[mcp] fix: retain cleanup debt after failed start#292
Wangmerlyn merged 1 commit into
mainfrom
codex/startup-cleanup-retention

Conversation

@Wangmerlyn

@Wangmerlyn Wangmerlyn commented Jul 2, 2026

Copy link
Copy Markdown
Owner

Summary

  • Retain a visible service session when startup fails after a controller was constructed and cleanup cannot be proven complete.
  • Preserve the original startup error for callers while marking cleanup timeout as stopping and cleanup errors as stop_failed.
  • Add MCP/HTTP regressions, lifecycle docs, AGENTS guidance, and the implementation plan under docs/plans/.

Verification

  • RED: PYTHONPATH=src pytest tests/mcp/test_server.py::test_failed_start_with_cleanup_timeout_remains_visible_as_stopping tests/mcp/test_server.py::test_failed_start_with_cleanup_error_remains_visible_as_stop_failed -q failed before the service fix because status returned inactive.
  • PYTHONPATH=src pytest tests/mcp/test_server.py::test_failed_start_with_cleanup_timeout_remains_visible_as_stopping tests/mcp/test_server.py::test_failed_start_with_cleanup_error_remains_visible_as_stop_failed tests/mcp/test_http_api.py::test_http_start_failure_cleanup_timeout_keeps_session_visible -q -> 3 passed.
  • PYTHONPATH=src pytest tests/mcp/test_server.py tests/mcp/test_http_api.py -q -> 347 passed.
  • mkdocs build --strict -> passed, with the known Material for MkDocs 2.0 warning.
  • pre-commit run --all-files --show-diff-on-failure -> passed.

Summary by CodeRabbit

  • New Features

    • Improved handling of failed session startup so cleanup status stays visible when shutdown cannot be confirmed immediately.
    • Sessions may now remain shown as stopping during delayed cleanup, or stop_failed if cleanup encounters an error.
  • Bug Fixes

    • Preserved the original startup error for callers while preventing premature removal of affected sessions.
    • Added coverage to ensure sessions remain visible during cleanup delays and that job IDs are not reused too early.

@coderabbitai

coderabbitai Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Warning

Review limit reached

@Wangmerlyn, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 44 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 1bfa0a15-1f25-411c-a57b-88ee94b2e0a1

📥 Commits

Reviewing files that changed from the base of the PR and between e7dd2bc and d7fc4f7.

📒 Files selected for processing (6)
  • AGENTS.md
  • docs/concepts/architecture.md
  • docs/guides/mcp.md
  • src/keep_gpu/mcp/server.py
  • tests/mcp/test_http_api.py
  • tests/mcp/test_server.py
📝 Walkthrough

Walkthrough

Adds a controller-startup cleanup retention contract to KeepGPUServer.start_keep(): sessions failing after controller work begins are retained as "stopping" (cleanup timeout) or "stop_failed" (cleanup error) instead of being silently discarded, preserving the original startup error. Includes new tests and documentation.

Changes

Startup Cleanup Retention

Layer / File(s) Summary
start_keep() retention implementation
src/keep_gpu/mcp/server.py
Introduces controller = None sentinel and _retain_failed_start_for_cleanup() helper that keeps a failed session visible as stopping/stop_failed, releases via _release_with_timeout() with a late-release finalizer, and updates the exception handler to branch on whether a controller was created.
Unit and HTTP API test coverage
tests/mcp/test_server.py, tests/mcp/test_http_api.py
Adds tests validating session retention on cleanup timeout and cleanup error, restart rejection while job-id is reserved, late-callback deactivation, and an HTTP regression test confirming REST visibility of the stopping state.
Documentation and implementation plan
AGENTS.md, docs/concepts/architecture.md, docs/guides/mcp.md, docs/plans/2026-07-02-startup-cleanup-retention.md
Documents the new state-retention rule and adds a plan document detailing implementation tasks, tests, and verification steps.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant StartKeep as start_keep()
  participant Retain as _retain_failed_start_for_cleanup()
  participant Controller

  Client->>StartKeep: request start
  StartKeep->>Controller: keep()
  Controller-->>StartKeep: raises exception
  alt controller is None
    StartKeep->>StartKeep: discard starting/pending-stop tracking
  else controller exists
    StartKeep->>Retain: retain session
    Retain->>Retain: set state=stopping, remove from starting/pending-stop
    Retain->>Controller: _release_with_timeout()
    alt cleanup succeeds
      Controller-->>Retain: success
      Retain->>Retain: remove retained session
    else cleanup times out or errors
      Controller-->>Retain: timeout/TimeoutError
      Retain->>Retain: mark state=stopping/stop_failed, log
    end
  end
  StartKeep-->>Client: return original startup error
Loading

Possibly related PRs

  • Wangmerlyn/KeepGPU#77: Modifies KeepGPUServer.start_keep failure-path handling around the same session/job-id reservation and release logic.
  • Wangmerlyn/KeepGPU#147: Coordinates start_keep()/stop_keep() around the same _release_with_timeout and late-callback machinery for session visibility on timeout/failure.
  • Wangmerlyn/KeepGPU#149: Alters how startup sessions are retained/transitioned and how timeout-related state/last_error are reported in the same file.

Poem

A job that stumbled, mid-controller stride,
No longer vanishes — it stays, doesn't hide.
"Stopping," it whispers, "or stop_failed," it groans,
While the true error still rides home alone.
🐇 Cleanup debt tracked, no rabbit left behind.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is concise and accurately summarizes the main change: retaining cleanup debt after a failed MCP start.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/startup-cleanup-retention

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements a mechanism to retain a visible session when a GPU controller fails startup after construction and cleanup cannot be proven complete immediately. Specifically, it updates KeepGPUServer.start_keep() to transition the session to stopping or stop_failed states depending on whether cleanup times out or raises an error, while still propagating the original startup exception to the caller. The changes also include updated documentation, a detailed implementation plan, and comprehensive unit and HTTP API tests to verify this retention behavior. There are no review comments to address, and I have no additional feedback to provide.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
tests/mcp/test_server.py (1)

2184-2211: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Duplicate FailsAfterWorkController definition.

An almost identical FailsAfterWorkController is defined inline in the previous test (Line 2143) and again here, and a third variant appears in tests/mcp/test_http_api.py. Consider hoisting a shared helper/fixture to reduce duplication, otherwise the logic itself is correct.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/mcp/test_server.py` around lines 2184 - 2211, The test logic is fine,
but FailsAfterWorkController is duplicated across multiple tests, including this
KeepGPUServer cleanup-error case and the earlier inline variant. Hoist the
shared controller behavior into a reusable helper or fixture in the test module,
and update the tests to reference that shared symbol so the duplicated class
definition is removed while preserving the current assertions in
test_failed_start_with_cleanup_error_remains_visible_as_stop_failed.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@tests/mcp/test_server.py`:
- Around line 2184-2211: The test logic is fine, but FailsAfterWorkController is
duplicated across multiple tests, including this KeepGPUServer cleanup-error
case and the earlier inline variant. Hoist the shared controller behavior into a
reusable helper or fixture in the test module, and update the tests to reference
that shared symbol so the duplicated class definition is removed while
preserving the current assertions in
test_failed_start_with_cleanup_error_remains_visible_as_stop_failed.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 25ffbd44-90a6-4ffa-b02f-5b320ea12fbc

📥 Commits

Reviewing files that changed from the base of the PR and between ec1fa40 and e7dd2bc.

📒 Files selected for processing (7)
  • AGENTS.md
  • docs/concepts/architecture.md
  • docs/guides/mcp.md
  • docs/plans/2026-07-02-startup-cleanup-retention.md
  • src/keep_gpu/mcp/server.py
  • tests/mcp/test_http_api.py
  • tests/mcp/test_server.py

@Wangmerlyn
Wangmerlyn force-pushed the codex/startup-cleanup-retention branch from e7dd2bc to 1601658 Compare July 2, 2026 08:31
@Wangmerlyn
Wangmerlyn force-pushed the codex/startup-cleanup-retention branch from 1601658 to d7fc4f7 Compare July 2, 2026 08:38
@Wangmerlyn

Copy link
Copy Markdown
Owner Author

Resolved the remaining CodeRabbit nit in d7fc4f7 by replacing the duplicated test-local failing controller classes in tests/mcp/test_server.py with a small _failing_after_work_factory helper. Re-verified with targeted cleanup-debt tests, pre-commit, and the full MCP/HTTP suite (347 passed).

@Wangmerlyn
Wangmerlyn merged commit ec36b5e into main Jul 2, 2026
6 checks passed
@Wangmerlyn
Wangmerlyn deleted the codex/startup-cleanup-retention branch July 2, 2026 08:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant