`create_sandbox()` does not clean up a newly created sandbox when setup fails

[wangjiawei-vegetable]

Submission checklist

• This is a bug, not a usage question.
• I added a clear and descriptive title.
• I searched existing issues and didn't find this.
• I can reproduce this with the latest released version.
• I included a minimal reproducible example and steps to reproduce.

Area (Required)

• deepagents (SDK)
• dcode
• talon
• acp
• evals
• harbor
• daytona
• modal
• quickjs
• runloop
• vercel
• langsmith-sandbox
• Other / not sure / general

Related Issues / PRs

• dcode --sandbox <provider> fails server startup with Blocking call to socket.socket.connect (blockbuster guard kills sync sandbox creation on the event loop) #6018 reports a separate sandbox lifecycle problem in which synchronous sandbox creation performs blocking I/O on the event loop.
• This issue concerns cleanup after a provider has already returned a newly created sandbox: if the optional setup script fails, the sandbox is not deleted.
• fix(code): run sandbox creation and entry off the guarded event loop #6017 is a test PR associated with dcode --sandbox <provider> fails server startup with Blocking call to socket.socket.connect (blockbuster guard kills sync sandbox creation on the event loop) #6018 and does not cover cleanup after setup failure.

Reproduction Steps / Example Code (Python)

The lifecycle bug can be reproduced without cloud credentials using a mock provider:

from types import SimpleNamespace
from unittest.mock import MagicMock, patch

import pytest

from deepagents_code.integrations.sandbox_factory import create_sandbox


def test_fresh_sandbox_is_deleted_when_setup_fails(tmp_path):
    setup_script = tmp_path / "setup.sh"
    setup_script.write_text("exit 1", encoding="utf-8")

    backend = MagicMock()
    backend.id = "sb-new"
    backend.execute.return_value = SimpleNamespace(
        exit_code=1,
        output="setup failed",
    )

    provider = MagicMock()
    provider.get_or_create.return_value = backend

    registry = MagicMock()
    registry.get_metadata.return_value = None
    registry.get_params.return_value = {}

    with (
        patch(
            "deepagents_code.integrations.sandbox_factory._get_registry",
            return_value=registry,
        ),
        patch(
            "deepagents_code.integrations.sandbox_factory._get_provider",
            return_value=provider,
        ),
        pytest.raises(RuntimeError, match="Setup failed"),
    ):
        with create_sandbox(
            "fake",
            setup_script_path=str(setup_script),
        ):
            pass

    provider.get_or_create.assert_called_once()
    provider.delete.assert_called_once_with(sandbox_id="sb-new")

The final assertion fails:

AssertionError: Expected 'delete' to be called once. Called 0 times.

I also reproduced the issue using a real remote OpenSandbox instance. The sanitized validation output was:

setup_error=Setup failed - aborting
automatic_cleanup_calls=0
post_failure_probe_exit_code=0
post_failure_probe_stdout=REAL_SANDBOX_STILL_ALIVE
safety_cleanup=completed

The successful post-failure probe confirms that the remote sandbox remained active after setup failed. The sandbox was manually deleted immediately after validation.

Description

create_sandbox() determines whether the current call owns the sandbox using:

should_cleanup = sandbox_id is None

It then creates or connects to the sandbox:

backend = provider_obj.get_or_create(
    sandbox_id=sandbox_id,
    **provider_kwargs,
)

However, the optional setup script runs before the cleanup-protected try/finally begins:

if setup_script_path:
    _run_sandbox_setup(backend, setup_script_path)

try:
    yield backend
finally:
    if should_cleanup:
        provider_obj.delete(sandbox_id=backend.id)

If _run_sandbox_setup() raises, execution never reaches the try/finally. When the sandbox was newly created by this call, provider_obj.delete() is therefore not called.

The sandbox is neither returned to the caller nor cleaned up. For a remote provider, it may continue consuming quota or incurring charges until it expires or is deleted manually.

The same control-flow gap applies when setup fails because:

• the setup-script path does not exist;
• reading the local setup script fails;
• backend.execute() raises;
• the setup command returns a non-zero exit code.

The problem is in the common create_sandbox() lifecycle rather than a specific provider implementation.

Expected Behavior

After provider_obj.get_or_create(sandbox_id=None) successfully returns a backend, subsequent setup failures should trigger a best-effort call to:

provider_obj.delete(sandbox_id=backend.id)

The existing ownership behavior should remain unchanged:

• A newly created sandbox, where sandbox_id is None, should be deleted.
• A caller-provided existing sandbox, where sandbox_id is not None, should not be deleted.
• A cleanup failure should not replace the original setup exception. It should continue to be reported through the existing cleanup warning behavior.

The setup phase should be included in the same cleanup-protected lifecycle as the context-manager body.

Suggested Regression Tests

1. A newly created sandbox is deleted when the setup command returns a non-zero exit code.
2. A newly created sandbox is deleted when setup raises an exception.
3. An existing sandbox passed through sandbox_id is not deleted when setup fails.
4. If both setup and deletion fail, the original setup exception is preserved and the cleanup failure is reported as a warning.
5. Normal context-manager exit and an exception raised inside the context body continue to delete a newly created sandbox exactly once.

Environment / System Info

OS: macOS
Python: 3.14.0
deepagents-code: 0.1.65
main commit: 03436b369c0324498602fe6b7918cf36f3629d76
real provider validation: OpenSandbox

[langchain-oss-automated-triage]

This issue was automatically closed because it appears to have been submitted programmatically — issue types are automatically set when using the GitHub web interface, and this issue has none.

We do not allow automated issue submission at this time.

Please use one of the issue templates.