feat: engine-ops async folding workspace builder#1075
Draft
tony wants to merge 14 commits into
Draft
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1075 +/- ##
==========================================
- Coverage 82.56% 82.20% -0.37%
==========================================
Files 31 32 +1
Lines 2770 3017 +247
Branches 518 563 +45
==========================================
+ Hits 2287 2480 +193
- Misses 346 385 +39
- Partials 137 152 +15 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
why: the EngineOpsWorkspaceBuilder consumes libtmux's unreleased experimental workspace API (analyze/Workspace.abuild/WorkspaceSet, AsyncSubprocessEngine.for_server). This branch-only [tool.uv.sources] pin makes it importable; not for release. what: - Add [tool.uv.sources] libtmux -> engine-ops git branch - uv lock
why: the classic builder issues one libtmux call per window/pane/command. libtmux's engine-ops workspace API folds a whole session into a handful of ;-chained tmux dispatches and builds it asynchronously; wiring it as an opt-in builder gives tmuxp that speedup without touching the classic path. what: - Add EngineOpsWorkspaceBuilder (WorkspaceBuilderProtocol): analyze the config to a Workspace, abuild it over AsyncSubprocessEngine.for_server, bridge BuildEvents to on_build_event, populate .session - Register the `engine-ops` entry point (select via workspace_builder key) - Cover build shape, protocol conformance, entry-point resolution, event forwarding, and the append/empty guards
why: Let users route a single load through the async folding builder from the command line, without editing the workspace file's workspace_builder key. what: - Add --engine-ops flag to `tmuxp load` (CLILoadNamespace field, argparse switch, command_load pass-through) - load_workspace() gains builder_override= to inject the workspace_builder key after expansion - Enrich the builder's session_created event with name, window_total and session_pane_total so the load spinner renders instead of raising KeyError - Add a CLI-flag build test
why: Build several workspaces as one folded engine-ops set so N sessions render in a few ;-chained tmux dispatches sharing one preflight, instead of one detached load per file. what: - Add _load_parallel(): lower each file to the Workspace IR, group into a WorkspaceSet, and abuild the compiled plan - Add --parallel (implies engine-ops), --dry-run (print the compiled plan without building) and --no-fold (SequentialPlanner) - Reject -a/--append and -s; require --parallel for --dry-run and --no-fold; surface duplicate-name and on_exists errors - Attach the last declared session unless detached - Add CLI tests for build shape, dry-run, dup names and flag guards
why: Turn one parameterized workspace file into a session per value -- a matrix over multiple axes -- reusing the folded set build instead of copy-pasting near-identical workspace files. what: - Add --expand KEY=V1,V2 (repeatable) to expand a single file over variants, substituting $KEY placeholders via expand() - Add _parse_expand_specs(): parse specs into a Cartesian product of variant dicts; reject malformed specs - Route --expand through the set-build path (it implies it) and require exactly one workspace file - Extend the --dry-run/--no-fold guard to accept --expand - Add unit + live tests for matrix build, substitution, rejections
why: --parallel/--expand skipped all plugins. before_script is a post-build, whole-session hook the fold already exposes, so running it costs nothing and restores real plugin value; the mid-build hooks can't fold and are skipped with a warning instead of silently. what: - Load each file's plugins before the fold (keeps the version pre-flight gate) and memoize per file so --expand variants that share a file load and warn once - Run before_script on each freshly built session after abuild, in the file's import sandbox; a raising hook is logged, not fatal to the other built sessions - Warn once per plugin overriding a mid-build hook (before_workspace_builder/on_window_create/after_window_finished) - Do not fire reattach on fresh set builds, matching a fresh sequential load - Document the contract in the docstring and plugins topic doc - Add parametrized hook, leniency, and no-load tests
why: The libtmux engine-ops branch advanced (bug fixes + doc hygiene); re-resolve the pin so this branch builds against the current tip.
why: The libtmux engine-ops branch advanced (supervised reconnect, drain-restart error fix, reconnect backoff + proc-cleanup hardening, and the experimental facade->objects package rename), so re-resolve the git pin to pick it up. what: - uv.lock: libtmux engine-ops 4393b02a -> f52b2d3f.
why: --engine-ops hardcoded AsyncSubprocessEngine; the persistent
control-mode engine builds larger workspaces much faster but was not
reachable from the CLI.
what:
- Add --engine-ops-engine {subprocess,control_mode} (default
subprocess), threaded through the expanded-workspace config key like
builder_override.
- Select AsyncSubprocessEngine or AsyncControlModeEngine (an async
context manager) at both build sites: the single builder and the
--parallel/--expand workspace set.
- Cover both engines with a parametrized build test.
why: Track the latest engine-ops branch tip so the pinned libtmux matches the branch HEAD used downstream. what: - Bump locked libtmux rev f52b2d3f -> be88f08d (scripts-only delta: hermetic engine benchmark grid + results)
why: Track the latest engine-ops branch tip so the pinned libtmux matches the branch HEAD used downstream. what: - Bump locked libtmux rev be88f08d -> aa6d822d (scripts-only delta: bench_engines mypy fix)
why: Track the latest engine-ops branch tip so the pinned libtmux matches the branch HEAD used downstream. what: - Bump locked libtmux rev aa6d822d -> 1fe8fc67 (ConcreteEngine renamed to MockEngine upstream; no downstream code references it)
why: libtmux's engine-ops branch was rebased onto master (absorbing the gp-sphinx docs-dep bump); the prior pinned commit is now unreachable and GC-eligible on the remote. what: - Bump locked libtmux rev 1fe8fc67 -> e88e87b6
why: libtmux's engine-ops branch advanced -- the engines now hold a ServerConnection (one owner for the connection flags, tmux-bin resolution and argv assembly), the chain folder is guarded by ensure_chainable, shared helpers are imported rather than re-inlined, and the unread surface is gone. what: - Relock libtmux to the current engine-ops tip (22557d98).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an opt-in async workspace builder backed by libtmux's experimental engine-ops stack, plus CLI switches for single-session, parallel, and matrix loads.
What's here
EngineOpsWorkspaceBuilder(workspace/builder/engine_ops.py): aWorkspaceBuilderProtocolthat lowers an expanded workspace dict to libtmux's declarativeWorkspaceIR andabuilds it asynchronously. The build folds: a multi-pane window's commands collapse into a handful of;-chained dispatches instead of onesend-keysper line, so a session renders in a few round-trips rather than dozens. Registered as theengine-opsentry point.workspace_builder: engine-opsconfig key selects the builder.tmuxp load --engine-opsselects it for a single load without editing the workspace file; the load spinner keeps working via a structural-event bridge.tmuxp load --engine-ops-engine {subprocess,control_mode}picks the async engine the fold runs on (defaultsubprocess).subprocessforks atmuxper dispatch;control_modedrives the whole fold over one persistenttmux -Ccontrol-mode client, trading per-call process spawns for a single long-lived connection. Applies to--engine-ops,--parallel, and--expandalike. (imsg has no async engine, so it isn't offered here.)tmuxp load --parallelbuilds every WORKSPACE_FILE as oneWorkspaceSet: all sessions compile into a single rebased, folded plan sharing one preflight, instead of one detached load per file.--dry-runprints the compiled tmux plan without touching the server;--no-folddispatches one command per operation for debugging.-a/--appendand-sare rejected (they describe single-session semantics).tmuxp load --expand KEY=V1,V2expands one parameterized file into a session per value, substituting$KEYplaceholders (e.g.session_name: app-$env). Repeat--expandfor a matrix (Cartesian product). Implies the folded set build.Plugin support on the set path
--parallel/--expandrun the post-buildbefore_scriptplugin hook on each built session (loaded before the fold so the version pre-flight still gates; run in the file's import sandbox; a raising hook is logged, not fatal to the other sessions). Mid-build hooks (before_workspace_builder,on_window_create,after_window_finished) can't fire without unfolding the plan, so a plugin that overrides one is skipped with a warning;reattachisn't fired on fresh builds (as on any fresh sequential load). Load a workspace on its own for the full hook sequence. Seedocs/topics/plugins.md.Notes
FileExistsError(on_exists='error'); duplicate session names in a set are rejected up front.abuild(on_event=...)is intentionally not wired here: the plugin hooks run afterasyncio.runreturns and never touch the event stream. (A future streaming PR should routeon_eventthrough a bounded non-blocking pump — libtmux'ssets.pycurrentlyawaits the sink inline, which would head-of-line-block the fold.)Try it via uvx / uv tool
This branch pins
libtmuxto its unreleasedengine-opsbranch via[tool.uv.sources], and a recentuv(tested with 0.11.25) honors that pin when installing from git — so no extra flags are needed. Do not add--with git+…libtmux…: the pin already supplies it, and a second URL fails with a "conflicting URLs" error. You'll also needtmuxon your PATH. These commands attach to the new session; add-d(--detach) to build it in the background instead.Run it once, ephemerally, with
uvx:Pick the engine the fold runs on. The default is
subprocess(atmuxfork per dispatch); pass--engine-ops-engine control_modeto drive the whole fold over one persistenttmux -Cclient instead:Build several workspaces as one folded set:
Or expand one parameterized file into a matrix of sessions:
The
--engine-ops-engineswitch works on the set path too:Install it persistently as
tmuxpon your PATH:Then load any workspace through the folding builder:
Dependency
Pins
libtmuxto theengine-opsbranch via[tool.uv.sources]— the engine-ops workspace API is unreleased (libtmux.experimental). Draft until that lands.