fix: persist the sandbox namespace with a single dill dump - #101
Open
YoanSallami wants to merge 1 commit into
Open
fix: persist the sandbox namespace with a single dill dump#101YoanSallami wants to merge 1 commit into
YoanSallami wants to merge 1 commit into
Conversation
Pickling the namespace item by item was quadratic: dill pickles a function together with a copy of its globals, so N sandbox-defined functions cost N full-namespace pickles (120 definitions ~3.8 s, a few hundred hit RecursionError and lost the namespace outright). One dill.dumps of the whole namespace memoizes the shared globals: the same 120 definitions now persist in ~0.4 s. The per-item filter stays as a fallback for a namespace holding something unpicklable (a generator, an open socket), and now pickles with recurse=True so a function carries only the globals it references. Without that, one unpicklable value dragged every sandbox-defined function down with it: the offending object was reachable through the shared globals every function copied. Restored functions are re-homed onto the live namespace on the next run, so the reduced globals are never observable. Also write the state through a sibling temp file and os.replace, so a run killed mid-write (host timeout) cannot leave a truncated state file. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GfeWSiwmNYwhbPNLR75mer
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
MirageSandboxpersists the REPL namespace betweenruncalls by pickling it withdill. It did so item by item, which is quadratic: dill pickles a function together with a copy of its globals, so N sandbox-defined functions cost N full-namespace pickles.Measured on this branch's test snippet (120
defs in one run):RecursionError, namespace lostA second, sharper bug hid in the same loop: one unpicklable value dropped every sandbox-defined function. The offending object was reachable through the shared globals each function copied, so
dill.dumps(item)failed for all of them:Fix
dill.dumpsof the whole namespace, which memoizes the shared globals once.recurse=Trueso a function carries only the globals it references rather than the whole namespace. Restored functions are re-homed onto the live namespace on the next run, so the reduced globals are never observable.os.replaced, so a run killed mid-write (host timeout) cannot leave a truncated state file.Tests
Two regression tests in
mirage_sandbox_test.py:test_many_functions_persistandtest_unpicklable_value_does_not_drop_the_namespace(the latter fails on the old code with theNameErrorabove).uv run pytest synalinks/src/sandboxes/mirage_sandbox_test.py→ 124 passed, 9 skipped.uvx ruff checkclean.🤖 Generated with Claude Code
https://claude.ai/code/session_01GfeWSiwmNYwhbPNLR75mer