Skip to content

feat(runtime): add scoped workload identities and agent tools - #129

Open
MarcStdt wants to merge 14 commits into
masterfrom
codex/workload-agent-tools
Open

MarcStdt wants to merge 14 commits into
masterfrom
codex/workload-agent-tools

Conversation

@MarcStdt

@MarcStdt MarcStdt commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Add opt-in Kubernetes workload tools using projected Pod-bound ServiceAccount tokens, a dedicated audience, TokenReview and live Pod/ownership/deployment checks. Permissions come from daemon-created Pod grants; authentication does not load daemon sessions or stored command status.
  • Remove the Kubernetes ConfigMap/watch registry, token Secrets and Docker FileRegistry. Kubernetes uses native token rotation and Pod lifecycle revocation; Docker tools are deliberately anonymous and unrestricted.
  • Expose all eleven tools through MCP at /:id/mcp and normal resource REST routes through the same permission-checked service. No generic tool RPC route or tool-name-dependent owner fallback.
  • Add explicit commands:remove: stop/remove a command's definition, runtime containers and queue/status. Preserve deployment data. No temporary-command tool.
  • Remove execution IDs from scheduling, status, resource labels and tool payloads. Wait by command name; restarts/replacements are not distinguished.
  • Keep command creation derived from the existing model, with concrete nested schema types for OpenHands. Creation rejects duplicates and identity grants; definition updates do not implicitly stop commands.
  • Preserve bounded logs/search/stdin, targeted JWT sanitization, existing dashboard/infrastructure authentication and UI package publication. No global logging scanner or duplicate MCP response scan.

REST mappings

Paths are relative to /:id/api/v1.

MCP tool REST
list_commands GET /commands
create_command POST /commands
remove_command DELETE /scroll/commands/:command
run_command POST /command
wait_for_command GET /commands/:command/wait
get_queue GET /queue
read_logs GET /consoles/:console/logs
search_logs POST /consoles/:console/logs/search
send_stdin POST /consoles/:console/stdin
get_deployment_status GET /status
publish_ui POST /ui/packages/:scope/publish

Existing queue, run, removal and publication routes are registered once and preserve their original owner-authenticated handlers, responses and status codes. Workload callers use the shared tool service.

Verification

  • Full Go suite: go test -p 1 ./....
  • Focused command-service, HTTP handler and Kubernetes race tests; final handler race suite passed.
  • Real Docker and Kubernetes MCP fixtures passed using the exact SPA launcher, without an LLM request: discovery, creation/execution/removal, waiting, logs/search/stdin, completed logs, browser-independent execution and daemon reattachment.
  • REST coverage includes every tool, strict request validation, deployment/permission denial, single registration of shared routes and unchanged owner queue/run/remove/publication behavior.
  • Kubernetes coverage retains TokenReview audience, bound Pod verification, captured grants, missing-grant denial and authentication without daemon session state.
  • Helm rendering in both namespace modes and runtime image import were verified during the ServiceAccount implementation. Latest fixtures use disposable resources, which were removed.
  • gofmt and git diff --check passed.

Accepted behavior and rollout

  • Breaking change: no execution IDs or exact-run waiting. A wait observes the current command; delayed old events are not distinguished from replacements.
  • Waits authorize once when accepted and may finish after revocation within the 30-second maximum. Subsequent requests authenticate again.
  • Kubernetes API failures fail closed. Revocation follows Pod termination/deletion and native token validity, not stored command status alone. Grants change only when Pods are explicitly recreated.
  • Existing opaque-token Kubernetes agents require explicit recreation; no legacy fallback. New agents need commands:remove for removal. No automatic recreation.
  • Docker caveat: any reachable caller can invoke every workload tool for any deployment, ignoring permissions and credentials. Trusted single-user local use only; keep the proxy away from untrusted networks/workloads. Unrelated owner-route authentication remains unchanged.
  • UI publishing approval is a prompt instruction, not server-enforced approval. Existing shell consoles grant shell-like control through stdin.
  • No database migrations, production deployment, merge or existing workload restart.

Related PRs

Backend-owned workload setup

  • Create worker and agent ServiceAccounts together during deployment materialization. Command execution no longer provisions or edits ServiceAccounts.
  • Session command execution passes the configured workload API URL to the backend; remove backend-name checks, optional preparation assertions and identity-related copying from the session.
  • Kubernetes owns opt-in credential metadata, token projection and environment injection. Docker retains credential-free API URL injection. Stored command definitions and request authentication remain unchanged.
  • No migrations, legacy detection, automatic repair or existing workload conversion.
  • Verified full Go suite, focused race tests, real Docker/Kubernetes MCP fixtures and diff checks. Only disposable test resources were used; no existing agents or game workloads restarted.

Give opted-in command executions isolated opaque credentials and scoped
HTTP/MCP tools without inheriting dashboard or infrastructure privileges.

Persist identities across reconnects and revoke them with execution
lifecycle changes. Redact workload credentials from output.
Replace the Kubernetes credential registry with ServiceAccount tokens,
TokenReview and current execution checks. Keep Docker file credentials.

Kubernetes agents using opaque tokens require explicit recreation; no
legacy credential fallback is retained.
Snapshot Kubernetes grants on runtime-created Pods and authenticate
without loading daemon sessions or execution records. Native Pod
lifecycle controls Kubernetes revocation; Docker retains its file-based
credential registry and captured grants.
Docker tools deliberately allow anonymous access to every deployment and
ignore permission grants. Restrict this runtime to trusted local use.

Remove the file registry, token mounts and credential lifecycle machinery.
Kubernetes Pod-bound authentication and unrelated route auth stay unchanged.
Restore existing log encoders and remove obsolete opaque-token matching.
Keep JWT sanitization at workload log/error output instead of scanning
all logs and rescanning every serialized MCP result.

Verify HTTP/MCP sanitization parity without changing Kubernetes isolation,
Docker tool access or execution tracking.
Delete parallel agent REST routes and their owner-auth fallthrough.
Keep existing owner REST endpoints registered once with normal auth;
OpenHands continues using the unchanged MCP tools.

BREAKING CHANGE: agent tools are available only at /:id/mcp.
Kubernetes permissions and anonymous Docker MCP behavior are unchanged.
Updating a stored command definition must not implicitly stop its current
execution or discard queue/status state. Pod grants are immutable for the
running execution; stop and restart remain explicit operations.

MCP create_command continues rejecting existing command names.
Remove recovery-only flags, duplicate queue execution IDs, unused workload
environment metadata and the no-op runtime closer. Narrow authenticated
identity to deployment binding and permissions; retain Pod ownership
checks and execution tracking independently.

Authorize a bounded command wait at acceptance, not on every poll. An
accepted wait can finish after revocation within 30 seconds; subsequent
requests still require current credentials. Cover both behaviors and
hydration without legacy recovery flags.
Expose every workload tool through resource REST routes and MCP.
Reuse existing queue, run, removal and publication owner handlers.

BREAKING CHANGE: remove execution IDs from runtime status and tool
payloads. Waiting observes the current command, including later runs.
New Kubernetes agents need commands:remove to remove commands.
Provision worker and agent ServiceAccounts together during
materialization, not command execution. Keep token metadata and
environment injection inside each runtime backend.

No migrations, legacy handling or automatic repair.
Use local Auth JWKS so supplied dashboard credentials validate instead of failing under anonymous-only local configuration.
Comment thread apps/druid/adapters/cli/daemon.go Outdated
return err
}
callbacks := appservices.NewWorkerCallbackManager()
if err := supervisor.SetWorkloadAPIURL(runtimeWorkloadAPIURL); err != nil {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we not just add to NewRuntimeSupervisor?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 7a498dc. NewRuntimeSupervisor now accepts and validates the workload API URL before creating the backend. Removed SetWorkloadAPIURL and updated all callers, including integration fixtures. Added constructor tests for URL validation, normalization and configuration being available during backend creation.

@@ -0,0 +1,198 @@
package handlers

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

id dont like this file. its splits routes to somewhere else. mcp and http routes should be more distinged. for mcp stuff i want multiple files for different handlers. and a mcp.add function instead. this needs to be solved cleaner!

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 7a498dc. Removed command_tools.go and its hidden route-registration function. All REST and MCP routes are now declared directly in routes.go with named REST handlers. MCP has a separate entrypoint and command, console, status, UI and schema files; commandMCP.add registers tools and applies permission-filtered discovery. HTTP and MCP still share the existing operation module, without changing paths, schemas or authorization. Verification: go test ./..., focused handler race tests, integration-tagged compile check, gofmt and diff checks all pass.

Configure the workload API URL during supervisor construction so backend creation cannot observe incomplete configuration.
Share upload-only and build-dependent publication across HTTP and MCP. Keep initialization upload-only and delegate live rebuilds to declared commands.

BREAKING CHANGE: UI declarations require wasm/openui instead of path. Publication accepts with_build instead of a path override; no legacy fallback.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant