feat(workflows): add dynamic declarative workflows - #618
Conversation
Add durable sequential workflow execution with bounded outputs, human review gates, cancellation, restart recovery, lineage-scoped locking, and SSE updates. Expose workflow creation and monitoring in the right panel and through the CodeNomad plugin bridge. Keep approval mutations on the authenticated user API so plugin credentials cannot bypass a human gate, and preserve workspace lineage across desktop restore. Cover runtime transitions, restart rebinding, abort containment, route scoping, client reconciliation, restore identity, and plugin messaging with focused tests.
Resolve right-panel customization conflicts by registering Workflows through the modular tab registry while retaining keyboard tab navigation and ARIA panel relationships. Combine workflow and customization translations, keep generic persisted tab IDs, and preserve all upstream settings, auth, filesystem, permission, and Linux launch fixes. Validated with the UI typecheck and 78 focused registry, workflow, reconciliation, snapshot, and codec tests.
Add versioned JSON/YAML workflow definitions with branching, bounded parallelism, loops, nested workflows, gates, retries, structured outputs, budgets, pause/resume, crash recovery, and managed-worktree execution. Keep workflow mutation behind authenticated host APIs, scope plugin callbacks with per-workspace capabilities, restrict agent tools and plugin operations, and harden cancellation, persistence, CORS, process cleanup, ownership leases, and stale revision handling. Extend the right-panel workflow UI with declarative authoring, localized execution monitoring, durable drafts, recovery controls, safe output rendering, and retained-workspace session navigation while preserving the simple builder. Validated with server typechecks and the full 335-pass server suite, UI/plugin typechecks and builds, focused workflow/UI/Markdown tests, and a zero-finding gatekeeper review.
pascalandr
changed the title
|
PR builds are available as GitHub Actions artifacts: https://github.com/NeuralNomadsAI/CodeNomad/actions/runs/30383020047 Artifacts expire in 7 days.
|
Merge the background-process client-factory refactor from dev while preserving workflow cancellation semantics. The shared instance client now supports explicit session/worktree directories and composes Request, caller, and fallback timeout signals. Combined tests cover loopback routing, authorization, directory scoping, pre-dispatch cancellation, active cancellation, and timeouts. Validated with server and UI typechecks plus the full server suite: 345 passed, 4 platform-skipped, 0 failed.
|
PR builds are available as GitHub Actions artifacts: https://github.com/NeuralNomadsAI/CodeNomad/actions/runs/30665339657 Artifacts expire in 7 days.
|
pascalandr
changed the title
Merge upstream/dev at c16cc00 into the workflow host branch. This adopts the first-party right-panel manifest runtime from NeuralNomadsAI#617 and registers Workflows through that contract while preserving stable tab IDs, keyboard navigation, localization, and existing customization behavior. The merge also carries the latest invalid OpenCode configuration diagnostics and retains both workspace lifecycle test paths. Resolutions were validated with server and UI typechecks, the UI production build, the full server suite, focused right-panel/workflow/workspace tests, and an independent regression review.
Keep the event loop referenced while the mocked stuck fetch waits for AbortSignal.timeout. Real fetch requests retain a socket handle, but the test double did not, allowing Node's test runner to cancel the suite before the unrefed timeout fired in CI. The timer is cleared after each test. The focused instance-client test and full server suite pass with no failures or cancellations.
|
PR builds are available as GitHub Actions artifacts: https://github.com/NeuralNomadsAI/CodeNomad/actions/runs/30809608845 Artifacts expire in 7 days.
|
Expose saved-definition create and update operations to the workspace plugin, and attach plugin-started runs to the calling OpenCode session so existing family permissions and Yolo state apply naturally. Let omitted agent tool lists inherit normal OpenCode access, retain explicit installed-tool allowlists, and add an opt-in repeat exhaustion failure policy for bounded self-correction loops. Cover schema parsing, runtime behavior, plugin payloads, and route boundaries. Validated with both TypeScript projects, 62 focused tests, and the full 355-test server suite.
A concurrently starting modern Electron client can publish its legacy running marker just before its cross-host participant. On slower Windows runners, Tauri could observe that transient state and both clients could remain secondary. Recheck a blocking legacy marker once after a bounded 50 ms grace period while preserving PID, process identity, and executable validation. Add a deterministic participant-publication race test. Validated with rustfmt, five cross-host startup runs, and the complete 85-test Tauri suite on Windows.
|
PR builds are available as GitHub Actions artifacts: https://github.com/NeuralNomadsAI/CodeNomad/actions/runs/30886836822 Artifacts expire in 7 days.
|
Allow agent nodes to declare an optional sessionKey. Nodes and repeat iterations sharing a key now continue the same OpenCode conversation, while nodes without a key retain the existing one-session-per-action behavior. Persist bounded key-to-session bindings in the workflow journal and serialize prompts per key so parallel branches cannot mutate one conversation concurrently. Existing crash recovery and confirmed session abort handling remain the side-effect boundary. Cover portable schema validation, persisted-state validation, repeat and cross-node reuse, parallel serialization, and restored bindings. Validated with server/UI/plugin typechecks and the complete server suite: 354 passed, 4 skipped.
|
PR builds are available as GitHub Actions artifacts: https://github.com/NeuralNomadsAI/CodeNomad/actions/runs/30890442848 Artifacts expire in 7 days.
|
1 similar comment
|
PR builds are available as GitHub Actions artifacts: https://github.com/NeuralNomadsAI/CodeNomad/actions/runs/30890442848 Artifacts expire in 7 days.
|
pascalandr
left a comment
pascalandr
left a comment
There was a problem hiding this comment.
Gatekeeper review
Findings
-
High: Shared
sessionKeycancellation can falsely confirm termination without aborting the active OpenCode generation.packages/server/src/workflows/interpreter.ts:192,packages/server/src/workflows/interpreter.ts:657-670,packages/server/src/workflows/manager.ts:1085-1089,packages/server/src/workflows/manager.ts:1222-1229. The session limiter is released before the prior node callssessionFinished(). A waiting node can start another prompt on the same session, then the prior node deletes that session ID from the sharedSet. Cancellation subsequently treats the active session as already finished and skips the explicitsession.abort, allowing remote work or side effects to continue after the run is persisted as cancelled. -
High: Caller-controlled ancestry can borrow another session family's Yolo permissions.
packages/server/src/server/routes/workflows.ts:71-75,packages/server/src/server/routes/workflows.ts:247-251,packages/server/src/workflows/interpreter.ts:98-103,packages/server/src/permissions/auto-accept-store.ts:29-55. The plugin start route accepts anyinitiatorSessionIdunder the workspace bearer and uses it directly asparentID. A callback-token holder can nominate a different Yolo-enabled session, causing workflow descendants and their inherited tools to be auto-approved under that family. -
High: A workspace-scoped plugin capability can replace workflows used by every workspace.
packages/server/src/server/http-server.ts:247-269,packages/server/src/server/routes/workflows.ts:184-230,packages/server/src/workflows/manager.ts:128-131,packages/server/src/workflows/manager.ts:431-445. Plugin definition routes ignorerequest.params.idand mutate one global store. Because omitted tool lists inherit all plugin tools, a compromised or prompt-injected agent in workspace A can replace a trusted definition that later executes in workspace B. -
High: Worktree deletion races unpublished workspace startup.
packages/server/src/server/routes/worktrees.ts:165-176,packages/server/src/workspaces/manager.ts:166-174,packages/server/src/workspaces/manager.ts:392,packages/server/src/workspaces/manager.ts:496. Starting workspaces are reserved before publication but excluded fromlist()andget(). DELETE can therefore remove the worktree whileruntime.launchis using it; a new startup can also begin after the route's scan and beforegit worktree remove. -
High: A recovery marker with completed actions permanently reserves the workspace.
packages/server/src/workflows/manager.ts:991-1007,packages/server/src/workflows/manager.ts:1470-1476,packages/server/src/workflows/manager.ts:1612-1624,packages/server/src/workflows/manager.ts:661-666,packages/server/src/workflows/manager.ts:750-767. If terminal persistence fails after all actions complete, restart createsrecovery_requiredwith no ambiguous session IDs. Resume rejects it for lacking IDs, and cancellation also refuses to confirm termination, leaving no API path to release the reservation. -
Medium: Crashes before action admission are misclassified as ambiguous side effects.
packages/server/src/workflows/interpreter.ts:144-147,packages/server/src/workflows/interpreter.ts:451-455,packages/server/src/workflows/manager.ts:1627-1658. An action is persisted asrunningbefore its attempt is incremented. A crash while waiting for a limiter or listing tools leavesattempt === 0and no session, yet restart marks it unrecoverable; resume and cancel then cannot release the run despite no side effect having started. -
Medium: Repeat results and
onExhaustedoutcomes are not recovery-stable.packages/server/src/workflows/interpreter.ts:251-266,packages/server/src/workflows/manager.ts:1646-1654. The loop always resumes from index zero and evaluateswhilebefore consulting completed iterations. If the last completed body changes the condition to false and the process crashes before the repeat parent is persisted, recovery can omit that body's output and turn an exhaustion failure into success. A normal run also fails immediately after the last body without checking whether that body satisfied the exit condition. -
Medium: Confirmed abort/recovery resends a named prompt into the interrupted conversation.
packages/server/src/workflows/interpreter.ts:327-332,packages/server/src/workflows/interpreter.ts:404-422,packages/server/src/workflows/interpreter.ts:742-746,packages/server/src/workflows/manager.ts:1488-1499. Abort handling removes execution-node session references but retainssessionBindings. Retry or confirmed recovery therefore reuses the aborted session and appends the identical prompt again, potentially duplicating instructions or side effects already represented by the first user message. -
Medium: Human confirmations can authorize a different state from the one displayed.
packages/ui/src/components/instance/shell/right-panel/tabs/WorkflowRunList.tsx:49-71,packages/server/src/server/routes/workflows.ts:98,packages/server/src/workflows/manager.ts:639-668. Approval reads the reactive current gate only after the dialog closes; if another client advances from gate A to gate B, the stale dialog approves B. Recovery confirmation carries neither an expected revision nor interrupted-node identity, so a stale dialog can authorize a later, unrelated ambiguous side effect. -
Medium: The Tauri 50 ms retry does not close the participant-publication race.
packages/tauri-app/src-tauri/src/client_state/cross_host.rs:229-276,packages/tauri-app/src-tauri/src/client_state/cross_host.rs:296-320,packages/tauri-app/src-tauri/src/client_state/cross_host.rs:726-751. If modern Electron publishes its legacy marker but is suspended for longer than 50 ms before publishing its participant, Tauri remains permanently secondary. If Electron then crashes before acquiring ownership, Tauri never retries acquisition and client-state ownership remains vacant until restart. -
Medium: Side-effect checkpoints are atomic but not crash-durable.
packages/server/src/workflows/manager.ts:1328-1334,packages/server/src/workflows/manager.ts:1706-1720,packages/server/src/workflows/definition-store.ts:205-214. Temporary files are renamed without syncing the file or containing directory. A power or OS crash afterpersist()returns but after a prompt starts can restore an older checkpoint and cause recovery to repeat an effect it believes never started. -
Medium: Valid definition-store limits permit multi-gigabyte concurrent reads.
packages/server/src/workflows/definition-store.ts:24-25,packages/server/src/workflows/definition-store.ts:125-140. The store permits 1,000 histories of approximately 4 MiB each, whilelist()reads and parses every file concurrently. A catalog built entirely through valid plugin requests can make ordinary definition listing allocate several gigabytes and terminate the server. -
Medium: Workflow SSE delivery has unbounded buffering and quadratic snapshot traffic.
packages/server/src/workflows/manager.ts:1725-1729,packages/server/src/server/routes/events.ts:39-50,packages/server/src/workflows/definition-schema.ts:16,packages/server/src/workflows/interpreter.ts:18. Every checkpoint publishes the complete accumulated run, potentially containing 10,000 nodes and 4 MiB of output. The SSE writer ignores backpressure, so slow clients retain increasingly large snapshots in memory while large runs repeatedly serialize and transmit the entire history.
Open questions / assumptions
- Callback credentials are assumed not to confer arbitrary session-family identity.
- Per-workspace callback tokens are assumed not to be host-wide workflow-administrator credentials.
- OpenCode abort is assumed not to transactionally remove the original user message and partial response.
onExhausted: failis assumed to mean the exit condition remains true after all permitted attempts.- Power/OS-crash recovery is assumed to be in scope.
- Electron is assumed to have no enforced upper bound guaranteeing participant publication within 50 ms.
Validation gaps
- No cancellation test covers the handoff between two serialized prompts sharing one
sessionKey. - No restart test covers an action persisted as
runningwithattempt === 0. - No recovery test covers a completed repeat body whose parent node was not persisted.
- No test resolves or cancels marker-backed recovery containing only completed actions.
- No integration test combines plugin-start ancestry with the real
AutoAcceptManager. - No cross-workspace test defines plugin definition-mutation scope.
- No mounted UI test changes gate/recovery state while confirmation dialogs are open.
- The Tauri retry test does not cover publication after 50 ms or producer crash.
- No power-loss/fsync, maximum-catalog, slow-SSE-client, or unpublished-workspace deletion-race test exists.
Existing checks at review time: 354/358 server tests with 4 skipped, 85/85 Tauri tests, server/UI typechecks, plugin build, and all PR CI jobs green.
1 participant
Summary
Implements Claude Code-style dynamic workflows through a safe declarative YAML/JSON runtime backed by OpenCode agents. It provides comparable dynamic orchestration without evaluating generated JavaScript or TypeScript.
User experience
Users can build and operate workflows from the Workflows tab in the customizable right panel:
Example:
Compared with Claude Code dynamic workflows
evalor generated-code execution; commands cross an explicit shell-node boundaryThe user outcome is intentionally similar: workflows are created dynamically and can branch, loop, fan out, invoke agents, pause for people, and compose other workflows. The execution model is deliberately different: CodeNomad favors predictable validation, visual operation, persistence, and recovery over arbitrary generated-code flexibility.
Tradeoffs
toolsis omitted; an explicittoolslist remains an installed-tool allowlist. Shell nodes remain the deterministic command primitive.Safety
Persistent named sessions and probable follow-ups
Agent nodes may declare an optional
sessionKey. The first use creates an OpenCode session; later nodes and repeat iterations with the same key continue that conversation. Bindings are persisted for restart recovery, and prompts sharing a key are serialized so parallel branches cannot mutate one conversation concurrently. Omitting the key preserves one session per action.Probable follow-ups remain intentionally separate: per-session worktrees for independently isolated workers, provider-specific reasoning controls such as a max-thinking variant, and richer PR authorization primitives if generic human gates prove insufficient.
Validation
Closes #435