diff --git a/ARCHITECTURE_V4.md b/ARCHITECTURE_V4.md index 4638f5a..0ed6e0c 100644 --- a/ARCHITECTURE_V4.md +++ b/ARCHITECTURE_V4.md @@ -825,6 +825,13 @@ attack surface for "convince the model to execute something dangerous" bounded by what a human operator pre-approved, not by what the model can be talked into requesting. +> **Implemented in v4.0.0-alpha.7** as `crates/aarambh-studio-agent/src/{sandbox,authorization}.rs`. +> `SandboxedToolProvider` implements `ToolResultProvider`, so execution plugs +> into the existing `ToolChain` with zero chain changes. See +> `docs/phase47_sandbox.md` for the runbook and the honesty boundary +> (pure-Rust CPU sandbox: wall-clock timeout + output/argument-size ceilings; +> OS-level isolation is out of scope for the source release). + --- ## 62. Multi-Agent Orchestration diff --git a/CHANGELOG.md b/CHANGELOG.md index 2fef0b4..2db064c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,92 @@ > From first principles. From zero. From Rust. +## [4.0.0-alpha.7] - 2026-08-23 + +### Added + +- **Phase 47 — Tool Execution With Sandboxing:** Closes the boundary v2 + §30 opened (tool calls are emitted, never executed) and v3 §46 extended + (multi-step chains, still emit-only): the model's tool calls can now be + **actually executed** by aarambh-studio itself, but only inside a strict, + closed-world sandbox. This is the highest-risk phase before Phase 51 and + is scoped conservatively on purpose — there is no generic "run a shell + command" or "eval this code" executor anywhere in the crate, by design. + - New `sandbox` module (`aarambh-studio-agent`: `sandbox.rs`): the + `ToolExecutor` trait (one specific, named capability per implementor), + `ToolSandbox` (closed-world registry + compiled JSON Schemas + + authorization + limits), `SandboxLimits` (wall-clock `timeout_ms`, + `max_output_bytes`, `max_args_bytes`), `ExecContext` (limits + + cooperative cancellation flag + deadline), `ValidatedArgs` (newtype + guarantee — executors never receive untrusted input), `ExecError` + (`UnknownTool`/`Unauthorized`/`InvalidArgs`/`Timeout`/ + `ResourceLimitExceeded`/`Executor`), and `SandboxedToolProvider` + (implements `ToolResultProvider`, so execution plugs into the existing + `ToolChain` with **zero chain changes** — results re-enter via the + unchanged `result_ingestion` path, execution is purely additive to what + v3 §46 built). + - New `authorization` module (`aarambh-studio-agent`: `authorization.rs`): + `AuthorizationScope` — the closed set of tool names an operator + explicitly enabled at startup. Per-tool authorization is an **operator** + decision, not a model decision: the model can request execution of + anything in its declared schema, but only authorized tools are ever + executed. `AuthorizationScope::intersect` supports Phase 48's + multi-agent orchestration, where a sub-agent's scope can only be a + *subset* of its orchestrator's — orchestration can never escalate tool + access beyond what the operator enabled at the top level. + - Two reference `ToolExecutor` implementations: `ReadFileInWorkdir` (the + milestone executor — a read-only file lookup confined to a fixed working + directory; refuses absolute paths and `..` traversal, caps output bytes, + no network or write access by construction) and `StaticLookup` (an + in-memory key→text executor for deterministic tests and smoke runs). + - The full §61 execution pipeline is enforced by `ToolSandbox::execute`: + (1) closed-world allowlist — the name must match a registered executor; + (2) operator authorization — distinct from `UnknownTool`, a capability + can exist and be declared but unauthorized; (3) argument-size ceiling; + (4) schema re-validation, defense-in-depth on top of the + grammar-constrained decoder — malformed or schema-invalid calls are + never executed; (5) bounded envelope — worker thread with + `recv_timeout` wall-clock timeout, cooperative cancellation via an + `AtomicBool` flag, detached-on-timeout since safe Rust cannot + force-kill a thread; (6) output-size ceiling. Every failure yields a + fail-closed `ToolResult{status:Error, error:...}` — the chain records + the refusal and continues, it never silently drops a call. + - New `agent` CLI flags: `--execute-tools` (switch from caller-executed + stdin/replay results to sandboxed execution), `--allow-tool ` + (repeatable operator authorization), `--exec-timeout-ms`, + `--exec-max-output-bytes`, `--exec-workdir ` (binds the + `ReadFileInWorkdir` executor). + - Six roadmap-named acceptance tests in `sandbox.rs` (plus supporting + tests for authorization intersect, read-file traversal refusal, and + limits validation): `unlisted_tool_name_is_hard_refused_never_attempted`, + `unauthorized_but_declared_tool_is_refused_at_execution_not_declaration`, + `execution_timeout_kills_a_hanging_tool_call`, + `execution_respects_configured_memory_and_cpu_ceiling`, + `malformed_tool_call_json_is_never_executed`, and + `execution_result_re_ingests_correctly_into_the_next_chain_step` (this + last one drives the real `ToolChain` with a `FakeDecoder` + + `SandboxedToolProvider` + `StaticLookup` and asserts the executed + result is re-ingested into the chain state). + - New `scripts/phase47_smoke.sh`: runs the agent-crate sandbox unit tests, + verifies `agent --help` surfaces the new flags, and writes a scorecard + to `artifacts/phase47_sandbox_smoke.json`. + - New `docs/phase47_sandbox.md`: dedicated Phase 47 runbook (mirrors + `docs/phase46_rlaif.md` structure). + +### Honesty boundary + +Phase 47's sandbox is pure-Rust and CPU-only: wall-clock timeout +(cooperative cancellation + thread-detachment on timeout, since safe Rust +cannot force-kill a thread), output-size ceiling, argument-size ceiling, +closed-world allowlist, operator authorization, and schema re-validation. +OS-level isolation (seccomp, cgroups, namespaces) is out of scope for the +source release, consistent with the project's CPU-first posture. The +safety-relevant property — a runaway or hung call never blocks the chain +and always produces a fail-closed result — holds under every tested failure +condition. A general-purpose code-execution sandbox remains explicitly out +of scope: Phase 47's execution is strictly closed-world, named-capability +tool execution, never arbitrary code or shell execution. + ## [4.0.0-alpha.6] - 2026-08-16 ### Added diff --git a/Cargo.lock b/Cargo.lock index 229977a..d391c55 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4,7 +4,7 @@ version = 4 [[package]] name = "aarambh-studio" -version = "4.0.0-alpha.6" +version = "4.0.0-alpha.7" dependencies = [ "aarambh-studio-agent", "aarambh-studio-audio", @@ -36,7 +36,7 @@ dependencies = [ [[package]] name = "aarambh-studio-agent" -version = "4.0.0-alpha.6" +version = "4.0.0-alpha.7" dependencies = [ "aarambh-studio-core", "aarambh-studio-inference", @@ -47,7 +47,7 @@ dependencies = [ [[package]] name = "aarambh-studio-audio" -version = "4.0.0-alpha.6" +version = "4.0.0-alpha.7" dependencies = [ "aarambh-studio-core", "candle-core", @@ -58,7 +58,7 @@ dependencies = [ [[package]] name = "aarambh-studio-core" -version = "4.0.0-alpha.6" +version = "4.0.0-alpha.7" dependencies = [ "candle-core", "serde", @@ -68,7 +68,7 @@ dependencies = [ [[package]] name = "aarambh-studio-data" -version = "4.0.0-alpha.6" +version = "4.0.0-alpha.7" dependencies = [ "aarambh-studio-core", "candle-core", @@ -79,7 +79,7 @@ dependencies = [ [[package]] name = "aarambh-studio-distill" -version = "4.0.0-alpha.6" +version = "4.0.0-alpha.7" dependencies = [ "aarambh-studio-core", "aarambh-studio-inference", @@ -96,7 +96,7 @@ dependencies = [ [[package]] name = "aarambh-studio-eval" -version = "4.0.0-alpha.6" +version = "4.0.0-alpha.7" dependencies = [ "aarambh-studio-agent", "aarambh-studio-audio", @@ -118,7 +118,7 @@ dependencies = [ [[package]] name = "aarambh-studio-finetune" -version = "4.0.0-alpha.6" +version = "4.0.0-alpha.7" dependencies = [ "aarambh-studio-audio", "aarambh-studio-core", @@ -139,7 +139,7 @@ dependencies = [ [[package]] name = "aarambh-studio-inference" -version = "4.0.0-alpha.6" +version = "4.0.0-alpha.7" dependencies = [ "aarambh-studio-core", "aarambh-studio-model", @@ -155,7 +155,7 @@ dependencies = [ [[package]] name = "aarambh-studio-kernel" -version = "4.0.0-alpha.6" +version = "4.0.0-alpha.7" dependencies = [ "aarambh-studio-core", "candle-core", @@ -169,7 +169,7 @@ dependencies = [ [[package]] name = "aarambh-studio-model" -version = "4.0.0-alpha.6" +version = "4.0.0-alpha.7" dependencies = [ "aarambh-studio-core", "aarambh-studio-nn", @@ -180,7 +180,7 @@ dependencies = [ [[package]] name = "aarambh-studio-nn" -version = "4.0.0-alpha.6" +version = "4.0.0-alpha.7" dependencies = [ "aarambh-studio-core", "aarambh-studio-kernel", @@ -191,7 +191,7 @@ dependencies = [ [[package]] name = "aarambh-studio-quant" -version = "4.0.0-alpha.6" +version = "4.0.0-alpha.7" dependencies = [ "aarambh-studio-core", "candle-core", @@ -202,7 +202,7 @@ dependencies = [ [[package]] name = "aarambh-studio-safety" -version = "4.0.0-alpha.6" +version = "4.0.0-alpha.7" dependencies = [ "aarambh-studio-core", "aarambh-studio-inference", @@ -213,7 +213,7 @@ dependencies = [ [[package]] name = "aarambh-studio-selflearn" -version = "4.0.0-alpha.6" +version = "4.0.0-alpha.7" dependencies = [ "aarambh-studio-core", "aarambh-studio-eval", @@ -234,7 +234,7 @@ dependencies = [ [[package]] name = "aarambh-studio-serve" -version = "4.0.0-alpha.6" +version = "4.0.0-alpha.7" dependencies = [ "aarambh-studio-core", "aarambh-studio-inference", @@ -258,7 +258,7 @@ dependencies = [ [[package]] name = "aarambh-studio-tokenizer" -version = "4.0.0-alpha.6" +version = "4.0.0-alpha.7" dependencies = [ "aarambh-studio-core", "serde", @@ -268,7 +268,7 @@ dependencies = [ [[package]] name = "aarambh-studio-train" -version = "4.0.0-alpha.6" +version = "4.0.0-alpha.7" dependencies = [ "aarambh-studio-audio", "aarambh-studio-core", @@ -286,7 +286,7 @@ dependencies = [ [[package]] name = "aarambh-studio-vision" -version = "4.0.0-alpha.6" +version = "4.0.0-alpha.7" dependencies = [ "aarambh-studio-core", "candle-core", @@ -302,7 +302,7 @@ dependencies = [ [[package]] name = "aarambh-studio-weights" -version = "4.0.0-alpha.6" +version = "4.0.0-alpha.7" dependencies = [ "aarambh-studio-core", "aarambh-studio-model", diff --git a/Cargo.toml b/Cargo.toml index fa562f4..7fd2eb9 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -24,7 +24,7 @@ members = [ resolver = "2" [workspace.package] -version = "4.0.0-alpha.6" +version = "4.0.0-alpha.7" edition = "2024" rust-version = "1.89" description = "From first principles. From zero. From Rust." diff --git a/README.md b/README.md index 985807b..c18a29f 100644 --- a/README.md +++ b/README.md @@ -34,11 +34,17 @@ Best-of-N / self-consistency / verifier-guided / process-reward selection that generates N independent candidate completions and selects the best one at inference time — a new axis alongside the existing thinking-mode budget system, distinct from controlling how many tokens -one generation spends reasoning — and RLAIF, a third alignment signal +one generation spends reasoning — RLAIF, a third alignment signal alongside GRPO and DPO where a frozen judge model scores pairs of self-sampled completions (judged in both orderings to correct position bias) and the resulting `(chosen, rejected)` pairs feed the existing -unmodified `finetune dpo` pipeline. +unmodified `finetune dpo` pipeline — and **sandboxed tool execution** +(Phase 47), which closes the boundary v2 §30 opened (emit-only) and +v3 §46 extended (multi-step, still emit-only): the model's tool calls can +now be **actually executed** by aarambh-studio itself, but only inside a +strict, closed-world sandbox (registered named executors, operator +authorization, schema re-validation, wall-clock timeout, and output-size +ceiling — there is no generic shell or eval executor, ever, by design). > [!IMPORTANT] > This is a source and engineering project. It does not publish crates to @@ -136,6 +142,7 @@ See the phase-specific docs for full walkthroughs with smoke fixtures: | Train (CPU/GPU, MTP, MoE, distillation, QAT) | `aarambh-studio train --help` + [configs/](configs/) TOML examples | | Inference (text, thinking, image, video, document, audio) | [docs/aarambh-studio-complete-guide.md](docs/aarambh-studio-complete-guide.md) | | Tool-use agent chains | [docs/phase37_agent.md](docs/phase37_agent.md) | +| Sandboxed tool execution | [docs/phase47_sandbox.md](docs/phase47_sandbox.md) | | Video understanding | [docs/phase35_video.md](docs/phase35_video.md) | | Document understanding | [docs/phase36_document.md](docs/phase36_document.md) | | Audio understanding | [docs/phase42_audio.md](docs/phase42_audio.md) | @@ -281,7 +288,13 @@ CUDA checks require a CUDA-capable environment and are intentionally opt-in. Multi-node training is data-parallel only (Phase 44), not model/pipeline-parallel. Test-time compute scaling (Phase 45) is text-only and ships a heuristic process-reward scorer plus a trait for a future trained head. -- Tool chains are generated and orchestrated but never executed by the runtime. +- Tool chains were historically generated and orchestrated but never + executed by the runtime; Phase 47 adds opt-in sandboxed execution + (`agent --execute-tools`) that executes only operator-authorized, + closed-world named capabilities (e.g. `read_file_in_workdir`) inside a + bounded envelope. A general-purpose code-execution sandbox remains out + of scope — execution is strictly closed-world, never arbitrary code or + shell execution. - Video is visual-only H.264 MP4; audio is WAV PCM only (no MP3/FLAC/Ogg). - Documents are pixel-based (no OCR/table parser). - The server is local/single-model; vision, audio, and self-learning are CLI workflows. @@ -303,7 +316,7 @@ reproducible bugs and scoped feature requests. Report vulnerabilities through author = {Aarambh Dev Hub}, year = {2026}, url = {https://github.com/AarambhDevHub/aarambh-studio}, - version = {4.0.0-alpha.6}, + version = {4.0.0-alpha.7}, license = {Apache-2.0} } ``` diff --git a/ROADMAP_V4.md b/ROADMAP_V4.md index 8c077a0..b05e9ff 100644 --- a/ROADMAP_V4.md +++ b/ROADMAP_V4.md @@ -746,7 +746,7 @@ and is scoped conservatively on purpose. **`aarambh-studio-agent`** *(extends the crate v3 §37 already scaffolded for chain orchestration)*: ``` -[ ] src/sandbox.rs +[x] src/sandbox.rs ToolExecutor trait — implementors register one specific, named capability (e.g. "http_get_whitelisted_domain", "read_file_in_workdir") — there is no generic "run shell command" @@ -756,7 +756,7 @@ and is scoped conservatively on purpose. Every execution wrapped with an explicit timeout and a resource ceiling (memory, CPU, wall-clock) — a runaway or hung tool call is killed, not allowed to block the chain indefinitely -[ ] src/authorization.rs +[x] src/authorization.rs Per-tool authorization is an operator decision, not a model decision — the model can request execution of anything in its declared tool schema, but only tools the operator has explicitly @@ -766,10 +766,10 @@ and is scoped conservatively on purpose. **Composability:** ``` -[ ] Execution happens only after the grammar-constrained JSON (v2 §30) +[x] Execution happens only after the grammar-constrained JSON (v2 §30) validates cleanly against the declared schema — malformed or partially-streamed tool calls are never executed -[ ] Execution results re-enter the chain via the existing +[x] Execution results re-enter the chain via the existing ToolResult/result_ingestion.rs path from v3 §46 — no new ingestion mechanism, execution is additive to what v3 already built ``` @@ -801,12 +801,33 @@ A whitelisted, sandboxed tool (e.g. a read-only file lookup within a fixed working directory) executes correctly end-to-end inside a multi-step chain, with every safety boundary (allowlist, timeout, resource cap) independently tested and verified to fail closed, not -open, under every tested failure condition. +open, under every tested failure condition. The 6 roadmap-named +acceptance tests in aarambh-studio-agent/src/sandbox.rs (plus +supporting tests) pass; the CLI `agent --execute-tools` path surfaces +the new flags; scripts/phase47_smoke.sh writes a scorecard. git commit -m "feat: Phase 47 — sandboxed tool execution" git tag v4.0.0-alpha.7 ``` +> **Status: Implemented in v4.0.0-alpha.7.** `crates/aarambh-studio-agent` +> ships `src/sandbox.rs` (`ToolExecutor` trait, `ToolSandbox`, `SandboxLimits`, +> `ExecContext`, `ExecError`, `SandboxedToolProvider`, and the reference +> `ReadFileInWorkdir` + `StaticLookup` executors) and `src/authorization.rs` +> (`AuthorizationScope`, with `intersect` for Phase 48 sub-agent scope +> narrowing). Execution is closed-world (registered executor + declared +> schema), operator-authorized, schema-re-validated, and bounded by a +> wall-clock timeout (worker thread + `recv_timeout`, cooperative +> cancellation flag, detached-on-timeout) and output/argument-size ceilings. +> `SandboxedToolProvider` implements `ToolResultProvider`, so execution plugs +> into the existing `ToolChain` with zero chain changes — results re-enter via +> the unchanged `result_ingestion` path. The CLI `agent --execute-tools` +> flag (with `--allow-tool`, `--exec-timeout-ms`, `--exec-max-output-bytes`, +> `--exec-workdir`) wires the operator's authorization. See +> `docs/phase47_sandbox.md` and `scripts/phase47_smoke.sh`. No new crate and +> no new dependency were added (std `thread`/`sync`/`mpsc` + existing +> `serde`/`thiserror` only); the release audit's 20-package invariant holds. + --- ## Phase 48 — Multi-Agent Orchestration diff --git a/SELF_LEARNING_V4.md b/SELF_LEARNING_V4.md index 9dcafb5..bdaf7cf 100644 --- a/SELF_LEARNING_V4.md +++ b/SELF_LEARNING_V4.md @@ -281,6 +281,13 @@ inappropriate, the same way any other incorrect proposal would be scored low and replayed, rather than the turn being silently dropped from the loop. +> **Implemented in v4.0.0-alpha.7.** `crates/aarambh-studio-agent/src/sandbox.rs` +> ships the closed-world `ToolExecutor` trait, `ToolSandbox`, and +> `SandboxedToolProvider` (a `ToolResultProvider`), so a self-learning +> session that enables sandboxed execution uses the exact same allowlist, +> authorization, timeout, and resource-ceiling boundaries as any other +> execution path — it is not a privileged or exempted context. + ## 48. Self-Learning With Multi-Agent Orchestration Extends v3 §33's chain-aware replay pattern one level further. A diff --git a/aarambh-studio/src/cmd/agent.rs b/aarambh-studio/src/cmd/agent.rs index 833243b..a43735a 100644 --- a/aarambh-studio/src/cmd/agent.rs +++ b/aarambh-studio/src/cmd/agent.rs @@ -3,9 +3,10 @@ use std::io::{self, BufReader}; use std::path::PathBuf; use aarambh_studio_agent::{ - AgentError, AgentResult, ChainDecoder, ChainEvent, EvictionPolicy, ReplayResultProvider, + AgentError, AgentResult, AuthorizationScope, ChainDecoder, ChainEvent, EvictionPolicy, + ReadFileInWorkdir, ReplayResultProvider, SandboxLimits, SandboxedToolProvider, StdinResultProvider, ToolChain, ToolChainConfig, ToolExchange, ToolResult, ToolResultContent, - ToolResultProvider, ToolResultRequest, + ToolResultProvider, ToolResultRequest, ToolSandbox, }; use aarambh_studio_core::{AarambhError, Configurable, TokenizerLike}; use aarambh_studio_inference::{ @@ -98,11 +99,32 @@ pub struct AgentArgs { /// JSONL safety audit path. #[arg(long, default_value = "safety_audit.jsonl")] pub safety_audit_log: PathBuf, + /// Execute tool calls inside the sandbox instead of reading caller + /// results from stdin/replay (Phase 47). Only tools listed via + /// `--allow-tool` are executable; everything else is a hard refusal. + #[arg(long)] + pub execute_tools: bool, + /// Operator-authorized tool name. Repeat to enable multiple tools. + /// Only these names may ever execute when `--execute-tools` is set. + #[arg(long = "allow-tool", value_name = "NAME")] + pub allow_tool: Vec, + /// Per-call wall-clock ceiling for sandboxed execution, in milliseconds. + #[arg(long, default_value_t = aarambh_studio_agent::DEFAULT_TIMEOUT_MS)] + pub exec_timeout_ms: u64, + /// Maximum output payload bytes a sandboxed executor may return. + #[arg(long, default_value_t = aarambh_studio_agent::DEFAULT_MAX_OUTPUT_BYTES)] + pub exec_max_output_bytes: usize, + /// Working directory for the `read_file_in_workdir` executor. The + /// executor is registered only when this is set; it can never read + /// outside this directory. + #[arg(long)] + pub exec_workdir: Option, } enum CliResultProvider { Replay(ReplayResultProvider), Stdin(StdinResultProvider>), + Sandbox(SandboxedToolProvider), } impl ToolResultProvider for CliResultProvider { @@ -110,6 +132,7 @@ impl ToolResultProvider for CliResultProvider { match self { Self::Replay(provider) => provider.next_result(request), Self::Stdin(provider) => provider.next_result(request), + Self::Sandbox(provider) => provider.next_result(request), } } @@ -117,6 +140,7 @@ impl ToolResultProvider for CliResultProvider { match self { Self::Replay(provider) => provider.finish(), Self::Stdin(provider) => provider.finish(), + Self::Sandbox(provider) => provider.finish(), } } } @@ -456,6 +480,11 @@ impl ChainDecoder for CliChainDecoder { /// Execute the agent CLI command. pub fn run(args: AgentArgs) -> anyhow::Result<()> { + // Validate sandboxed-execution config first so operator errors surface + // before any model is loaded or checkpoint is resolved. + if args.execute_tools { + validate_sandbox_config(&args)?; + } let run_config = TrainingRunConfig::from_toml(&args.config)?; let run_device = run_config.device()?; let dtype = run_config.dtype_for_device(&run_device)?.to_candle(); @@ -512,9 +541,13 @@ pub fn run(args: AgentArgs) -> anyhow::Result<()> { document_runtime: None, projected_media: None, }; - let provider = match args.results { - Some(path) => CliResultProvider::Replay(ReplayResultProvider::from_jsonl(path)?), - None => CliResultProvider::Stdin(StdinResultProvider::new(BufReader::new(io::stdin()))), + let provider = if args.execute_tools { + CliResultProvider::Sandbox(build_sandbox_provider(&args, &definitions)?) + } else { + match args.results { + Some(path) => CliResultProvider::Replay(ReplayResultProvider::from_jsonl(path)?), + None => CliResultProvider::Stdin(StdinResultProvider::new(BufReader::new(io::stdin()))), + } }; let eviction_policy = parse_eviction(&args.eviction)?; let chain_config = ToolChainConfig { @@ -597,3 +630,67 @@ fn parse_eviction(value: &str) -> anyhow::Result { )), } } + +/// Build the sandboxed-execution provider from the operator's CLI flags. +/// +/// Authorization is an operator decision: only `--allow-tool` names may +/// execute. The closed-world allowlist is the set of registered executors +/// (currently `read_file_in_workdir`, bound to `--exec-workdir`). A name +/// that is authorized but has no registered executor is still a hard +/// refusal at execution time (`ExecError::UnknownTool`). +fn build_sandbox_provider( + args: &AgentArgs, + definitions: &[ToolDefinition], +) -> anyhow::Result { + // Defense-in-depth: validate_sandbox_config already ran at the top of + // run(), so these checks are redundant here but kept so a programmatic + // caller cannot bypass them. + validate_sandbox_config(args)?; + let mut authorization = AuthorizationScope::empty(); + for name in &args.allow_tool { + authorization + .enable(name) + .map_err(|error| anyhow::anyhow!("invalid --allow-tool value: {error}"))?; + } + let limits = SandboxLimits { + timeout_ms: args.exec_timeout_ms, + max_output_bytes: args.exec_max_output_bytes, + max_args_bytes: aarambh_studio_agent::DEFAULT_MAX_ARGS_BYTES, + }; + let mut sandbox = ToolSandbox::new(authorization, limits) + .map_err(|error| anyhow::anyhow!("sandbox configuration error: {error}"))?; + sandbox + .register_definitions(definitions) + .map_err(|error| anyhow::anyhow!("sandbox definition error: {error}"))?; + if let Some(workdir) = &args.exec_workdir { + let executor = ReadFileInWorkdir::new(workdir) + .map_err(|error| anyhow::anyhow!("--exec-workdir error: {error}"))?; + sandbox + .register_executor(Box::new(executor)) + .map_err(|error| anyhow::anyhow!("executor registration error: {error}"))?; + } + Ok(SandboxedToolProvider::new(sandbox)) +} + +/// Validate the sandboxed-execution config before any model is loaded, so +/// operator errors (missing `--allow-tool`, unauthorized `--exec-workdir`) +/// surface immediately rather than after a checkpoint is resolved. +fn validate_sandbox_config(args: &AgentArgs) -> anyhow::Result<()> { + if args.allow_tool.is_empty() { + return Err(anyhow::anyhow!( + "--execute-tools requires at least one --allow-tool to authorize execution" + )); + } + if args.exec_workdir.is_some() + && !args + .allow_tool + .iter() + .any(|name| name == ReadFileInWorkdir::NAME) + { + return Err(anyhow::anyhow!( + "--exec-workdir registers the {:?} executor but it was not authorized via --allow-tool", + ReadFileInWorkdir::NAME + )); + } + Ok(()) +} diff --git a/artifacts/phase47_sandbox_smoke.json b/artifacts/phase47_sandbox_smoke.json new file mode 100644 index 0000000..0757423 --- /dev/null +++ b/artifacts/phase47_sandbox_smoke.json @@ -0,0 +1,21 @@ +{ + "phase": 47, + "title": "Tool Execution With Sandboxing", + "agent_unit_tests": "passed", + "cli_flags_surface": [ + "--execute-tools", + "--allow-tool", + "--exec-timeout-ms", + "--exec-max-output-bytes", + "--exec-workdir" + ], + "closed_world_allowlist": true, + "operator_authorization": true, + "schema_revalidation": true, + "wall_clock_timeout": true, + "output_and_args_ceilings": true, + "fail_closed_on_every_failure": true, + "no_new_crate": true, + "no_new_dependency": true, + "honesty_note": "The sandbox is pure-Rust and CPU-only: wall-clock timeout (cooperative cancellation + thread-detachment), output/argument size ceilings, closed-world allowlist, operator authorization, and schema re-validation. OS-level isolation (seccomp/cgroups) is out of scope for the source release. The 6 roadmap-named acceptance tests prove the safety-relevant property \u2014 a runaway or hung call never blocks the chain and always produces a fail-closed result \u2014 under every tested failure condition. A general-purpose code-execution sandbox remains explicitly out of scope: execution is strictly closed-world, named-capability tool execution." +} \ No newline at end of file diff --git a/crates/aarambh-studio-agent/src/authorization.rs b/crates/aarambh-studio-agent/src/authorization.rs new file mode 100644 index 0000000..849208d --- /dev/null +++ b/crates/aarambh-studio-agent/src/authorization.rs @@ -0,0 +1,176 @@ +//! Operator-controlled per-tool authorization. +//! +//! Phase 47 (`ARCHITECTURE_V4.md` §61) makes tool execution an operator +//! decision, not a model decision: a model can *declare* any tool in its +//! schema and *request* execution of anything it declares, but only the +//! tools an operator explicitly enabled at server or CLI startup are ever +//! actually executed. This module is the data structure that carries that +//! operator decision. +//! +//! An [`AuthorizationScope`] is a closed set of tool names. It is built once +//! at startup from the operator's allowlist and never widened by the model. +//! [`AuthorizationScope::intersect`] supports Phase 48's multi-agent +//! orchestration, where a sub-agent's authorized scope can only be a +//! *subset* of its orchestrator's — orchestration can never escalate tool +//! access beyond what the operator enabled at the top level. + +use std::collections::BTreeSet; + +use serde::{Deserialize, Serialize}; + +use crate::AgentError; + +/// Validate a tool name against the same character rule +/// `aarambh-studio-inference` enforces for tool declarations, so an +/// operator cannot enable a name the model could never legally declare. +fn validate_tool_name(name: &str) -> Result<(), AgentError> { + let mut chars = name.chars(); + let first = chars + .next() + .ok_or_else(|| AgentError::Config("tool name must not be empty".into()))?; + if name.len() > 64 + || !(first.is_ascii_alphabetic() || first == '_') + || !chars.all(|char| char.is_ascii_alphanumeric() || matches!(char, '_' | '.' | '-')) + { + return Err(AgentError::Config(format!( + "invalid tool name {name:?}; expected [A-Za-z_][A-Za-z0-9_.-]{{0,63}}" + ))); + } + Ok(()) +} + +/// The closed set of tool names an operator explicitly enabled at startup. +/// +/// Authorization is intentionally separate from the closed-world allowlist +/// of registered [`crate::ToolExecutor`] implementations: a tool can be +/// authorized (the operator said "yes, this name may execute") without a +/// matching executor being registered, in which case execution is still a +/// hard refusal via [`crate::ExecError::UnknownTool`]. The two checks are +/// applied in sequence, mirroring `ARCHITECTURE_V4.md` §61's pipeline. +#[derive(Debug, Clone, Default, Serialize, Deserialize)] +pub struct AuthorizationScope { + enabled: BTreeSet, +} + +impl AuthorizationScope { + /// Build an empty scope (no tool is authorized). + pub fn empty() -> Self { + Self::default() + } + + /// Build a scope from an iterator of tool names, validating each. + pub fn new(names: I) -> Result + where + I: IntoIterator, + S: Into, + { + let mut scope = Self::empty(); + for name in names { + scope.enable(&name.into())?; + } + Ok(scope) + } + + /// Enable one tool name, validating its format. Idempotent. + pub fn enable(&mut self, name: &str) -> Result<(), AgentError> { + validate_tool_name(name)?; + self.enabled.insert(name.to_string()); + Ok(()) + } + + /// Returns true when the operator enabled this tool name. + pub fn is_authorized(&self, name: &str) -> bool { + self.enabled.contains(name) + } + + /// The sorted, deduplicated set of authorized tool names. + pub fn allowed(&self) -> &BTreeSet { + &self.enabled + } + + /// Number of tools the operator authorized. + pub fn len(&self) -> usize { + self.enabled.len() + } + + /// Whether no tool is authorized (execution is fully disabled). + pub fn is_empty(&self) -> bool { + self.enabled.is_empty() + } + + /// Restrict this scope to the intersection with `other`. + /// + /// Used by Phase 48 multi-agent orchestration so a sub-agent's scope + /// can only narrow, never widen, its orchestrator's scope. The result + /// is always a subset of both inputs. + pub fn intersect(&self, other: &AuthorizationScope) -> AuthorizationScope { + let enabled = self + .enabled + .intersection(&other.enabled) + .cloned() + .collect::>(); + AuthorizationScope { enabled } + } +} + +#[cfg(test)] +mod tests { + use super::AuthorizationScope; + + #[test] + fn empty_scope_authorizes_nothing() { + let scope = AuthorizationScope::empty(); + assert!(scope.is_empty()); + assert!(!scope.is_authorized("read_file_in_workdir")); + } + + #[test] + fn enable_authorizes_named_tool() { + let mut scope = AuthorizationScope::empty(); + scope.enable("read_file_in_workdir").unwrap(); + assert!(scope.is_authorized("read_file_in_workdir")); + assert!(!scope.is_authorized("lookup")); + assert_eq!(scope.len(), 1); + } + + #[test] + fn invalid_tool_names_are_rejected() { + let mut scope = AuthorizationScope::empty(); + assert!(scope.enable("").is_err()); + assert!(scope.enable("1starts_with_digit").is_err()); + assert!(scope.enable("has space").is_err()); + assert!(scope.enable(&"a".repeat(65)).is_err()); + // Valid forms. + assert!(scope.enable("lookup").is_ok()); + assert!(scope.enable("read_file_in_workdir").is_ok()); + assert!(scope.enable("http.get_v2").is_ok()); + } + + #[test] + fn enable_is_idempotent() { + let mut scope = AuthorizationScope::empty(); + scope.enable("lookup").unwrap(); + scope.enable("lookup").unwrap(); + assert_eq!(scope.len(), 1); + } + + #[test] + fn intersect_is_subset_of_both() { + let parent = + AuthorizationScope::new(["read_file_in_workdir", "lookup", "shipping_quote"]).unwrap(); + let child = AuthorizationScope::new(["lookup", "dangerous_shell"]).unwrap(); + let sub = parent.intersect(&child); + assert!(sub.is_authorized("lookup")); + assert!(!sub.is_authorized("read_file_in_workdir")); + assert!(!sub.is_authorized("dangerous_shell")); + // A sub-scope can never reach a tool the parent lacked. + assert!(sub.allowed().iter().all(|name| parent.is_authorized(name))); + } + + #[test] + fn intersect_with_empty_disables_everything() { + let scope = AuthorizationScope::new(["read_file_in_workdir", "lookup"]).unwrap(); + let sub = scope.intersect(&AuthorizationScope::empty()); + assert!(sub.is_empty()); + } +} diff --git a/crates/aarambh-studio-agent/src/lib.rs b/crates/aarambh-studio-agent/src/lib.rs index 7a6ec6e..276f28e 100644 --- a/crates/aarambh-studio-agent/src/lib.rs +++ b/crates/aarambh-studio-agent/src/lib.rs @@ -1,13 +1,23 @@ //! Bounded, caller-executed, long-horizon tool-use chains. +//! +//! Phase 47 (`ARCHITECTURE_V4.md` §61) extends this crate with sandboxed +//! tool execution: the model's tool calls can now be actually executed by +//! aarambh-studio itself, inside a strict, closed-world sandbox. See the +//! [`sandbox`] and [`authorization`] modules for the execution envelope. #![deny(missing_docs)] +/// Operator-controlled per-tool authorization. +pub mod authorization; /// Multi-turn tool-chain orchestration. pub mod chain; /// Interactive and replay-based tool-result ingestion. pub mod result_ingestion; +/// Sandboxed tool execution (Phase 47). +pub mod sandbox; /// Exact-token chain state and result protocol types. pub mod state; +pub use authorization::AuthorizationScope; pub use chain::{ AgentError, AgentResult, ChainDecoder, ChainEvent, ChainMetrics, ChainOutput, ToolChain, ToolChainConfig, @@ -15,6 +25,11 @@ pub use chain::{ pub use result_ingestion::{ ReplayEntry, ReplayResultProvider, StdinResultProvider, ToolResultProvider, }; +pub use sandbox::{ + DEFAULT_MAX_ARGS_BYTES, DEFAULT_MAX_OUTPUT_BYTES, DEFAULT_TIMEOUT_MS, ExecContext, ExecError, + ReadFileInWorkdir, ResourceKind, SandboxLimits, SandboxedToolProvider, StaticLookup, + ToolExecutor, ToolSandbox, ValidatedArgs, +}; pub use state::{ ChainState, EvictionPolicy, ToolExchange, ToolResult, ToolResultContent, ToolResultRequest, ToolResultStatus, diff --git a/crates/aarambh-studio-agent/src/sandbox.rs b/crates/aarambh-studio-agent/src/sandbox.rs new file mode 100644 index 0000000..64a3f6f --- /dev/null +++ b/crates/aarambh-studio-agent/src/sandbox.rs @@ -0,0 +1,1391 @@ +//! Sandboxed tool execution. +//! +//! Phase 47 (`ARCHITECTURE_V4.md` §61) closes the boundary v2 §30 opened +//! (tool calls are emitted, never executed) and v3 §46 extended (multi-step +//! chains, still emit-only): the model's tool calls can now be **actually +//! executed** by aarambh-studio itself, but only inside a strict, explicit +//! sandbox. +//! +//! ## Closed-world execution +//! +//! Every executor ([`ToolExecutor`]) implements one specific, named +//! capability. There is no generic "run a shell command" or "eval this +//! code" executor anywhere in this crate, by design. An unrecognised tool +//! name is a **hard refusal** ([`ExecError::UnknownTool`]), never a +//! best-effort fallback attempt. +//! +//! ## The execution envelope +//! +//! Each call runs inside a bounded envelope enforced by [`ToolSandbox`]: +//! +//! 1. **Closed-world allowlist** — the name must match a registered +//! executor. +//! 2. **Operator authorization** — the name must be in the operator's +//! [`AuthorizationScope`] (an operator decision, not a model decision). +//! 3. **Schema re-validation** — arguments are independently re-validated +//! against the declared JSON Schema, defense-in-depth on top of the +//! grammar-constrained decoder that already produced a valid call. +//! Malformed or schema-invalid calls are never executed. +//! 4. **Wall-clock timeout** — execution runs on a worker thread with a +//! `recv_timeout` join; exceeding the ceiling yields +//! [`ExecError::Timeout`] and the call is abandoned (fail-closed). +//! 5. **Resource ceiling** — argument and output payload sizes are capped. +//! +//! ## Composability +//! +//! [`SandboxedToolProvider`] implements [`crate::ToolResultProvider`], so +//! execution plugs into the existing [`crate::ToolChain`] with **no changes +//! to the chain**: the chain calls `next_result`, the provider executes the +//! call through the sandbox, and the resulting [`crate::ToolResult`] re-enters +//! the chain via the existing `result_ingestion` path — execution is purely +//! additive to what v3 §46 already built. +//! +//! ## Honesty boundary +//! +//! This is a pure-Rust, CPU-only sandbox: wall-clock timeout (cooperative +//! cancellation via an [`std::sync::atomic::AtomicBool`] flag, plus +//! thread-abandonment on timeout since safe Rust cannot force-kill a +//! thread), output-size ceiling, args-size ceiling, closed-world allowlist, +//! operator authorization, and schema re-validation. OS-level isolation +//! (seccomp, cgroups, namespaces) is out of scope for the source release, +//! consistent with the project's CPU-first posture. The safety-relevant +//! property — a runaway or hung call never blocks the chain and always +//! produces a fail-closed result — holds under every tested failure +//! condition. + +use std::collections::HashMap; +use std::fs; +use std::io::Read; +use std::path::{Path, PathBuf}; +use std::sync::Arc; +use std::sync::atomic::{AtomicBool, Ordering}; +use std::sync::mpsc; +use std::thread; +use std::time::{Duration, Instant}; + +use aarambh_studio_inference::{JsonSchema, ToolCall, ToolDefinition}; +use serde::{Deserialize, Serialize}; +use thiserror::Error; + +use crate::authorization::AuthorizationScope; +use crate::{ + AgentError, ToolResult, ToolResultContent, ToolResultProvider, ToolResultRequest, + ToolResultStatus, +}; + +/// Maximum bytes accepted for the serialized tool-call arguments. +pub const DEFAULT_MAX_ARGS_BYTES: usize = 8 * 1024; +/// Maximum bytes accepted for a tool-execution output payload. +pub const DEFAULT_MAX_OUTPUT_BYTES: usize = 64 * 1024; +/// Default per-call wall-clock ceiling, in milliseconds. +pub const DEFAULT_TIMEOUT_MS: u64 = 5_000; + +/// Schema-validated tool-call arguments. +/// +/// Constructed only by [`ToolSandbox::execute`] after the arguments have +/// been independently re-validated against the declared JSON Schema. The +/// newtype guarantee means a [`ToolExecutor`] never receives untrusted, +/// unvalidated input — by construction, [`ValidatedArgs::value`] is +/// schema-valid for the tool named by [`ValidatedArgs::name`]. +#[derive(Debug, Clone)] +pub struct ValidatedArgs { + name: String, + arguments: serde_json::Value, +} + +impl ValidatedArgs { + /// The tool name these arguments belong to. + pub fn name(&self) -> &str { + &self.name + } + + /// The schema-valid argument object. + pub fn value(&self) -> &serde_json::Value { + &self.arguments + } +} + +/// The kind of resource ceiling an executor exceeded. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +#[serde(rename_all = "snake_case")] +pub enum ResourceKind { + /// The output payload exceeded `max_output_bytes`. + OutputBytes, + /// The argument payload exceeded `max_args_bytes`. + ArgumentBytes, +} + +impl std::fmt::Display for ResourceKind { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match self { + Self::OutputBytes => f.write_str("output_bytes"), + Self::ArgumentBytes => f.write_str("argument_bytes"), + } + } +} + +/// Resource and time ceilings for one execution envelope. +/// +/// All fields are operator-set configuration; an executor cannot influence +/// them. The ceilings are enforced by [`ToolSandbox`], not by individual +/// executors, so a buggy or adversarial executor cannot escape them. +#[derive(Debug, Clone, Copy, Serialize, Deserialize)] +pub struct SandboxLimits { + /// Wall-clock ceiling per call, in milliseconds. A call exceeding it + /// is abandoned and reported as [`ExecError::Timeout`] (fail-closed). + pub timeout_ms: u64, + /// Maximum output payload bytes an executor may return. + pub max_output_bytes: usize, + /// Maximum argument payload bytes accepted from the model. + pub max_args_bytes: usize, +} + +impl Default for SandboxLimits { + fn default() -> Self { + Self { + timeout_ms: DEFAULT_TIMEOUT_MS, + max_output_bytes: DEFAULT_MAX_OUTPUT_BYTES, + max_args_bytes: DEFAULT_MAX_ARGS_BYTES, + } + } +} + +impl SandboxLimits { + /// Validate that all ceilings are positive and bounded. + pub fn validate(&self) -> Result<(), AgentError> { + if self.timeout_ms == 0 { + return Err(AgentError::Config( + "sandbox timeout_ms must be greater than zero".into(), + )); + } + if self.max_output_bytes == 0 { + return Err(AgentError::Config( + "sandbox max_output_bytes must be greater than zero".into(), + )); + } + if self.max_args_bytes == 0 { + return Err(AgentError::Config( + "sandbox max_args_bytes must be greater than zero".into(), + )); + } + Ok(()) + } +} + +/// Execution context handed to every [`ToolExecutor`]. +/// +/// Carries the [`SandboxLimits`] in force and a cooperative cancellation +/// flag set when the wall-clock deadline passes. Long-running executors +/// should poll [`ExecContext::is_cancelled`] inside their loops and return +/// [`ExecError::Timeout`] promptly when it becomes true. +#[derive(Debug, Clone)] +pub struct ExecContext { + limits: SandboxLimits, + cancel: Arc, + deadline: Instant, +} + +impl ExecContext { + /// The resource and time ceilings in force for this call. + pub fn limits(&self) -> SandboxLimits { + self.limits + } + + /// The wall-clock deadline after which the call is considered timed out. + pub fn deadline(&self) -> Instant { + self.deadline + } + + /// True once the wall-clock deadline has passed or the sandbox raised + /// the cancellation flag. Cooperative executors should check this + /// inside long loops and return [`ExecError::Timeout`]. + pub fn is_cancelled(&self) -> bool { + self.cancel.load(Ordering::Acquire) || Instant::now() >= self.deadline + } + + /// Remaining wall-clock time before the deadline, or zero once passed. + pub fn remaining(&self) -> Duration { + self.deadline.saturating_duration_since(Instant::now()) + } +} + +/// Why a sandboxed execution did not produce a result. +/// +/// Every variant maps to a fail-closed [`ToolResult`] with +/// `status = Error`: the chain records the refusal and continues, it never +/// silently drops a call or attempts a best-effort fallback. +#[derive(Debug, Error)] +pub enum ExecError { + /// The tool name is not in the closed-world allowlist of registered + /// executors. Hard refusal — no execution attempt of any kind. + #[error("tool {0:?} is not in the closed-world execution allowlist")] + UnknownTool(String), + /// The tool is registered (in the allowlist) but the operator did not + /// authorize it. Distinct from `UnknownTool`: the capability exists, + /// the operator simply did not enable it. + #[error("tool {0:?} is not authorized by the operator")] + Unauthorized(String), + /// The tool-call arguments failed independent schema re-validation. + /// Malformed or schema-invalid calls are never executed. + #[error("tool {0:?} arguments failed schema validation: {1}")] + InvalidArgs(String, String), + /// Execution exceeded the configured wall-clock ceiling. The worker + /// thread was abandoned (fail-closed). + #[error("execution of {0:?} exceeded the {1} ms wall-clock ceiling")] + Timeout(String, u64), + /// Execution exceeded a configured resource ceiling. + #[error("execution of {name:?} exceeded the {kind} ceiling ({limit} bytes)")] + ResourceLimitExceeded { + /// The tool name. + name: String, + /// Which ceiling was exceeded. + kind: ResourceKind, + /// The configured ceiling value. + limit: usize, + }, + /// The executor returned an error from its own logic. + #[error("executor {0:?} failure: {1}")] + Executor(String, String), +} + +/// Maximum bytes of error text retained in a fail-closed `ToolResult`. +const MAX_ERROR_TEXT_BYTES: usize = 4 * 1024; + +impl ExecError { + /// Render the error as the bounded `error` text of a fail-closed + /// [`ToolResult`]. The result is always non-empty and well under the + /// [`crate::state::MAX_RESULT_TEXT_BYTES`] ceiling. + pub fn to_result_error(&self) -> String { + let text = self.to_string(); + if text.len() > MAX_ERROR_TEXT_BYTES { + // Keep the head and an ellipsis so the message stays bounded. + let mut head = text + .chars() + .take(MAX_ERROR_TEXT_BYTES - 1) + .collect::(); + head.push('…'); + head + } else { + text + } + } +} + +/// One named, closed-world execution capability. +/// +/// Every implementor represents one specific capability (for example a +/// read-only file lookup confined to a fixed working directory). There is +/// no generic "run a shell command" or "eval this code" executor — by +/// design, that capability does not exist in this crate. +/// +/// Implementors must be [`Send + Sync`] so the sandbox can dispatch on a +/// worker thread. They should respect the [`ExecContext`] cancellation +/// flag inside long loops and must not return more than +/// `limits.max_output_bytes` of payload (the sandbox enforces this as +/// defense-in-depth, but an executor that respects it avoids producing a +/// needless [`ExecError::ResourceLimitExceeded`]). +pub trait ToolExecutor: Send + Sync { + /// The exact capability name. Must match a declared [`ToolDefinition`] + /// name and an entry in the operator's [`AuthorizationScope`]. Exact + /// match only — no pattern or fuzzy resolution. + fn name(&self) -> &'static str; + + /// Execute one validated call. + /// + /// On success, returns a [`ToolResultContent`] that the sandbox wraps + /// into a `status = Ok` [`ToolResult`]. On error, returns an + /// [`ExecError`] that becomes a `status = Error` [`ToolResult`]. + fn execute( + &self, + args: &ValidatedArgs, + ctx: &ExecContext, + ) -> Result; +} + +/// A `ToolResultProvider` that executes tool calls through the sandbox. +/// +/// This is the composability bridge between Phase 47's execution capability +/// and v3 §46's existing chain: the [`crate::ToolChain`] calls +/// `next_result`, the provider executes the call via the sandbox, and the +/// resulting [`ToolResult`] re-enters the chain through the unchanged +/// `result_ingestion` path. No new ingestion mechanism is introduced. +pub struct SandboxedToolProvider { + sandbox: ToolSandbox, +} + +impl SandboxedToolProvider { + /// Build a provider backed by the given sandbox. + pub fn new(sandbox: ToolSandbox) -> Self { + Self { sandbox } + } + + /// Borrow the underlying sandbox (for inspection in tests). + pub fn sandbox(&self) -> &ToolSandbox { + &self.sandbox + } +} + +impl ToolResultProvider for SandboxedToolProvider { + fn next_result(&mut self, request: &ToolResultRequest) -> Result { + let result = self.sandbox.execute(&request.call, &request.call_id); + // Belt-and-braces: the sandbox already produced a valid result, but + // re-validate against the chain's call-id contract so any future + // drift is caught at the provider boundary, not deeper in the chain. + result.validate_for(&request.call_id)?; + Ok(result) + } +} + +#[derive(Debug)] +struct CompiledTool { + schema: JsonSchema, +} + +/// The closed-world sandbox. +/// +/// Holds the registry of named [`ToolExecutor`] implementations, the +/// compiled JSON Schemas for every declared tool, the operator's +/// [`AuthorizationScope`], and the [`SandboxLimits`] in force. Built once +/// at startup; immutable thereafter. +pub struct ToolSandbox { + executors: HashMap<&'static str, Arc>, + compiled: HashMap, + authorization: AuthorizationScope, + limits: SandboxLimits, +} + +impl ToolSandbox { + /// Build an empty sandbox with the given authorization and limits. + pub fn new( + authorization: AuthorizationScope, + limits: SandboxLimits, + ) -> Result { + limits.validate()?; + Ok(Self { + executors: HashMap::new(), + compiled: HashMap::new(), + authorization, + limits, + }) + } + + /// Register a tool definition so its schema can be compiled and its + /// name recognised as "declared". Must be called for every tool the + /// model may request, including those without a registered executor. + pub fn register_definition(&mut self, definition: ToolDefinition) -> Result<(), AgentError> { + let schema = JsonSchema::compile(&definition.parameters).map_err(|error| { + AgentError::Config(format!( + "tool {:?} parameters schema failed to compile: {error}", + definition.name + )) + })?; + if !schema.is_object() { + return Err(AgentError::Config(format!( + "tool {:?} parameters schema must have object type", + definition.name + ))); + } + self.compiled + .insert(definition.name.clone(), CompiledTool { schema }); + Ok(()) + } + + /// Register a batch of tool definitions. + pub fn register_definitions( + &mut self, + definitions: &[ToolDefinition], + ) -> Result<(), AgentError> { + for definition in definitions { + self.register_definition(definition.clone())?; + } + Ok(()) + } + + /// Register an executor implementing one named capability. The name + /// must exactly match the executor's [`ToolExecutor::name`] and a + /// previously-registered [`ToolDefinition`] of the same name. + pub fn register_executor(&mut self, executor: Box) -> Result<(), AgentError> { + let name = executor.name(); + if !self.compiled.contains_key(name) { + return Err(AgentError::Config(format!( + "cannot register executor {name:?}: no matching tool definition was registered" + ))); + } + if self.executors.insert(name, Arc::from(executor)).is_some() { + return Err(AgentError::Config(format!( + "duplicate executor registration for {name:?}" + ))); + } + Ok(()) + } + + /// The configured limits. + pub fn limits(&self) -> SandboxLimits { + self.limits + } + + /// The operator's authorization scope. + pub fn authorization(&self) -> &AuthorizationScope { + &self.authorization + } + + /// Whether an executor is registered for this capability name. + pub fn has_executor(&self, name: &str) -> bool { + self.executors.contains_key(name) + } + + /// Execute one tool call through the full §61 pipeline. + /// + /// Returns a fully-formed [`ToolResult`] ready to re-enter the chain. + /// Success yields `status = Ok` with content; every failure yields + /// `status = Error` with a bounded error message (fail-closed). + pub fn execute(&self, call: &ToolCall, call_id: &str) -> ToolResult { + match self.dispatch(call) { + Ok(content) => ToolResult { + call_id: call_id.to_string(), + status: ToolResultStatus::Ok, + content: Some(content), + error: None, + }, + Err(error) => ToolResult { + call_id: call_id.to_string(), + status: ToolResultStatus::Error, + content: None, + error: Some(error.to_result_error()), + }, + } + } + + fn dispatch(&self, call: &ToolCall) -> Result { + // 1. The name must be a declared tool (the model's schema). + let compiled = self + .compiled + .get(&call.name) + .ok_or_else(|| ExecError::UnknownTool(call.name.clone()))?; + + // 2. Closed-world allowlist: an executor must be registered. + let executor = self + .executors + .get(call.name.as_str()) + .ok_or_else(|| ExecError::UnknownTool(call.name.clone()))? + .clone(); + + // 3. Operator authorization: the operator must have enabled it. + // Distinct from UnknownTool: the capability exists and is declared, + // the operator simply did not authorize it. + if !self.authorization.is_authorized(&call.name) { + return Err(ExecError::Unauthorized(call.name.clone())); + } + + // 4. Argument size ceiling. + let args_len = serde_json::to_vec(&call.arguments) + .map_err(|error| ExecError::Executor(call.name.clone(), error.to_string()))? + .len(); + if args_len > self.limits.max_args_bytes { + return Err(ExecError::ResourceLimitExceeded { + name: call.name.clone(), + kind: ResourceKind::ArgumentBytes, + limit: self.limits.max_args_bytes, + }); + } + + // 5. Schema re-validation (defense-in-depth on top of the + // grammar-constrained decoder). + if let Err(error) = compiled.schema.validate(&call.arguments) { + return Err(ExecError::InvalidArgs(call.name.clone(), error.to_string())); + } + + let validated = ValidatedArgs { + name: call.name.clone(), + arguments: call.arguments.clone(), + }; + + // 6. Bounded envelope: run on a worker thread with a wall-clock + // timeout. A cooperative executor checks the cancellation flag; a + // non-cooperative one is abandoned (detached) on timeout. + let cancel = Arc::new(AtomicBool::new(false)); + let ctx = ExecContext { + limits: self.limits, + cancel: cancel.clone(), + deadline: Instant::now() + Duration::from_millis(self.limits.timeout_ms), + }; + let result = self.run_with_timeout(&call.name, executor, validated, &ctx)?; + + // 7. Output size ceiling (defense-in-depth). + let output_len = output_bytes(&result); + if output_len > self.limits.max_output_bytes { + return Err(ExecError::ResourceLimitExceeded { + name: call.name.clone(), + kind: ResourceKind::OutputBytes, + limit: self.limits.max_output_bytes, + }); + } + + Ok(result) + } + + fn run_with_timeout( + &self, + name: &str, + executor: Arc, + args: ValidatedArgs, + ctx: &ExecContext, + ) -> Result { + let (tx, rx) = mpsc::channel::>(); + let cancel = ctx.cancel.clone(); + let deadline = ctx.deadline; + let limits = ctx.limits; + let ctx_for_thread = ExecContext { + limits, + cancel: cancel.clone(), + deadline, + }; + // The executor is `Send + Sync` and stored behind an `Arc`, so it + // can be moved into the worker thread without any `unsafe`. + let builder = thread::Builder::new().name(format!("sandbox:{name}")); + let handle = builder + .spawn(move || { + let _ = tx.send(executor.execute(&args, &ctx_for_thread)); + }) + .map_err(|error| ExecError::Executor(name.to_string(), error.to_string()))?; + + match rx.recv_timeout(Duration::from_millis(self.limits.timeout_ms)) { + Ok(result) => { + // The worker finished; joining reaps the thread. + let _ = handle.join(); + result + } + Err(mpsc::RecvTimeoutError::Timeout) => { + // Raise the cancellation flag so a cooperative executor + // winds down. The thread is detached (safe Rust cannot + // force-kill it); the chain is unblocked and the result is + // fail-closed. + cancel.store(true, Ordering::Release); + Err(ExecError::Timeout(name.to_string(), self.limits.timeout_ms)) + } + Err(mpsc::RecvTimeoutError::Disconnected) => { + // The worker panicked before sending. Join to recover the + // panic, then report a fail-closed executor error. + if let Err(panic_payload) = handle.join() { + let msg = panic_payload + .downcast_ref::() + .map(String::as_str) + .or_else(|| panic_payload.downcast_ref::<&str>().copied()) + .unwrap_or("worker thread panicked"); + return Err(ExecError::Executor(name.to_string(), msg.to_string())); + } + Err(ExecError::Executor( + name.to_string(), + "worker thread ended without a result".into(), + )) + } + } + } +} + +/// Approximate byte cost of a result content payload, for ceiling checks. +fn output_bytes(content: &ToolResultContent) -> usize { + match content { + ToolResultContent::Text { text } => text.len(), + ToolResultContent::Image { path, description } => path.len() + description.len(), + ToolResultContent::Video { path, description } => path.len() + description.len(), + ToolResultContent::Document { + path, + pages, + description, + } => { + let pages_cost: usize = pages.iter().map(|p| *p as usize).sum(); + path.len() + description.len() + pages_cost + } + } +} + +// --------------------------------------------------------------------------- +// Reference executors +// --------------------------------------------------------------------------- + +/// A read-only file lookup confined to a fixed working directory. +/// +/// This is the milestone executor: "a read-only file lookup within a fixed +/// working directory." It reads a file by relative name, refuses any path +/// that escapes the workdir (absolute paths or `..` traversal), and caps +/// the returned bytes at the sandbox's `max_output_bytes`. It has no +/// network access and no write access — by construction. +pub struct ReadFileInWorkdir { + workdir: PathBuf, +} + +impl ReadFileInWorkdir { + /// The capability name this executor registers under. + pub const NAME: &'static str = "read_file_in_workdir"; + + /// Build an executor confined to `workdir`. The directory must exist. + pub fn new(workdir: impl Into) -> Result { + let workdir = workdir.into(); + let canonical = fs::canonicalize(&workdir).map_err(|error| { + AgentError::Config(format!( + "sandbox workdir {} cannot be canonicalized: {error}", + workdir.display() + )) + })?; + if !canonical.is_dir() { + return Err(AgentError::Config(format!( + "sandbox workdir {} is not a directory", + canonical.display() + ))); + } + Ok(Self { workdir: canonical }) + } + + /// The canonicalized working directory this executor is confined to. + pub fn workdir(&self) -> &Path { + &self.workdir + } + + /// Resolve a relative path against the workdir, refusing any escape. + fn resolve(&self, relative: &str) -> Result { + if relative.is_empty() { + return Err(ExecError::Executor( + Self::NAME.into(), + "path argument must not be empty".into(), + )); + } + let relative_path = Path::new(relative); + // Reject absolute paths and any `..` component in the user input + // before joining, so a missing file cannot be probed for + // path-structure information and an absolute path cannot replace + // the workdir base (`PathBuf::join` replaces the base when given an + // absolute path). + if relative_path.is_absolute() { + return Err(ExecError::Executor( + Self::NAME.into(), + "path must be relative and stay inside the workdir".into(), + )); + } + for component in relative_path.components() { + use std::path::Component; + if matches!( + component, + Component::ParentDir | Component::RootDir | Component::Prefix(_) + ) { + return Err(ExecError::Executor( + Self::NAME.into(), + "path must be relative and stay inside the workdir".into(), + )); + } + } + let candidate = self.workdir.join(relative_path); + let canonical = fs::canonicalize(&candidate).map_err(|error| { + ExecError::Executor( + Self::NAME.into(), + format!("cannot resolve path {relative:?}: {error}"), + ) + })?; + if !canonical.starts_with(&self.workdir) { + return Err(ExecError::Executor( + Self::NAME.into(), + "resolved path escapes the workdir".into(), + )); + } + Ok(canonical) + } +} + +impl ToolExecutor for ReadFileInWorkdir { + fn name(&self) -> &'static str { + Self::NAME + } + + fn execute( + &self, + args: &ValidatedArgs, + ctx: &ExecContext, + ) -> Result { + let path = args + .value() + .get("path") + .and_then(serde_json::Value::as_str) + .ok_or_else(|| { + ExecError::Executor(Self::NAME.into(), "missing string argument `path`".into()) + })?; + let resolved = self.resolve(path)?; + let mut file = fs::File::open(&resolved).map_err(|error| { + ExecError::Executor(Self::NAME.into(), format!("open failed: {error}")) + })?; + // Read up to the output ceiling (plus a small slack byte) so a huge + // file cannot exhaust memory before the ceiling fires. The sandbox's + // own output check enforces the exact bound. + let cap = ctx.limits().max_output_bytes; + let mut text = String::new(); + let mut buf = [0u8; 8192]; + loop { + if ctx.is_cancelled() { + return Err(ExecError::Timeout( + Self::NAME.into(), + ctx.limits().timeout_ms, + )); + } + let n = file.read(&mut buf).map_err(|error| { + ExecError::Executor(Self::NAME.into(), format!("read failed: {error}")) + })?; + if n == 0 { + break; + } + if text.len() + n > cap { + // Take only what fits under the ceiling and stop. + let keep = cap.saturating_sub(text.len()); + let slice = &buf[..n.min(keep)]; + match std::str::from_utf8(slice) { + Ok(s) => text.push_str(s), + Err(_) => { + return Err(ExecError::Executor( + Self::NAME.into(), + "file is not valid UTF-8".into(), + )); + } + } + break; + } + match std::str::from_utf8(&buf[..n]) { + Ok(s) => text.push_str(s), + Err(_) => { + return Err(ExecError::Executor( + Self::NAME.into(), + "file is not valid UTF-8".into(), + )); + } + } + } + Ok(ToolResultContent::Text { text }) + } +} + +impl std::fmt::Debug for ReadFileInWorkdir { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("ReadFileInWorkdir") + .field("workdir", &self.workdir) + .finish() + } +} + +/// An in-memory key-to-text lookup executor. +/// +/// Useful for deterministic tests and smoke runs that need a closed-world +/// capability without touching the filesystem. The capability name defaults +/// to `"lookup"` but may be overridden at construction so multiple static +/// tables can coexist. +pub struct StaticLookup { + name: &'static str, + table: HashMap, +} + +impl StaticLookup { + const DEFAULT_NAME: &'static str = "lookup"; + + /// Build a `"lookup"` executor from an owned key→text map. + pub fn new(table: HashMap) -> Self { + Self { + name: Self::DEFAULT_NAME, + table, + } + } + + /// Build a named static-lookup executor (e.g. `"lookup_config"`). + pub fn with_name(name: &'static str, table: HashMap) -> Self { + Self { name, table } + } +} + +impl ToolExecutor for StaticLookup { + fn name(&self) -> &'static str { + self.name + } + + fn execute( + &self, + args: &ValidatedArgs, + _ctx: &ExecContext, + ) -> Result { + let key = args + .value() + .get("key") + .and_then(serde_json::Value::as_str) + .ok_or_else(|| { + ExecError::Executor(self.name.into(), "missing string argument `key`".into()) + })?; + match self.table.get(key) { + Some(text) => Ok(ToolResultContent::Text { text: text.clone() }), + None => Err(ExecError::Executor( + self.name.into(), + format!("no entry for key {key:?}"), + )), + } + } +} + +impl std::fmt::Debug for StaticLookup { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("StaticLookup") + .field("name", &self.name) + .field("entries", &self.table.len()) + .finish() + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::authorization::AuthorizationScope; + use aarambh_studio_inference::ToolCall; + use serde_json::json; + use std::collections::HashMap; + use std::fs; + use std::path::PathBuf; + use std::sync::atomic::{AtomicUsize, Ordering}; + use std::time::Instant; + + fn lookup_definition() -> ToolDefinition { + ToolDefinition { + name: "lookup".into(), + description: "Look up a value by key.".into(), + parameters: json!({ + "type": "object", + "properties": {"key": {"type": "string"}}, + "required": ["key"], + "additionalProperties": false + }), + } + } + + fn read_file_definition() -> ToolDefinition { + ToolDefinition { + name: "read_file_in_workdir".into(), + description: "Read a UTF-8 text file from the working directory.".into(), + parameters: json!({ + "type": "object", + "properties": {"path": {"type": "string"}}, + "required": ["path"], + "additionalProperties": false + }), + } + } + + // --- Roadmap acceptance test 1 ---------------------------------------- + + #[test] + fn unlisted_tool_name_is_hard_refused_never_attempted() { + let mut sandbox = ToolSandbox::new( + AuthorizationScope::new(["lookup"]).unwrap(), + SandboxLimits::default(), + ) + .unwrap(); + // Declare a tool "ghost" the model can request, but never register + // an executor for it. The name is not in the closed-world execution + // allowlist. + let ghost = ToolDefinition { + name: "ghost".into(), + description: String::new(), + parameters: json!({"type":"object","properties":{}}), + }; + sandbox.register_definitions(&[ghost]).unwrap(); + let call = ToolCall { + name: "ghost".into(), + arguments: json!({}), + }; + let result = sandbox.execute(&call, "call_0001"); + assert_eq!(result.status, ToolResultStatus::Error); + assert!( + result + .error + .as_deref() + .unwrap() + .contains("not in the closed-world") + ); + assert_eq!(result.content, None); + } + + // --- Roadmap acceptance test 2 ---------------------------------------- + + #[test] + fn unauthorized_but_declared_tool_is_refused_at_execution_not_declaration() { + // The capability is registered (in the allowlist) and declared, but + // the operator did NOT authorize "lookup". + let mut sandbox = + ToolSandbox::new(AuthorizationScope::empty(), SandboxLimits::default()).unwrap(); + sandbox + .register_definitions(&[lookup_definition()]) + .unwrap(); + let table = HashMap::from([("alpha".to_string(), "value-alpha".to_string())]); + sandbox + .register_executor(Box::new(StaticLookup::new(table))) + .unwrap(); + let call = ToolCall { + name: "lookup".into(), + arguments: json!({"key":"alpha"}), + }; + let result = sandbox.execute(&call, "call_0001"); + assert_eq!(result.status, ToolResultStatus::Error); + assert!(result.error.as_deref().unwrap().contains("not authorized")); + assert_eq!(result.content, None); + // And the distinct-from-unknown property: an authorized version + // succeeds, proving the refusal was authorization-specific. + let mut sandbox_ok = ToolSandbox::new( + AuthorizationScope::new(["lookup"]).unwrap(), + SandboxLimits::default(), + ) + .unwrap(); + sandbox_ok + .register_definitions(&[lookup_definition()]) + .unwrap(); + let table2 = HashMap::from([("alpha".to_string(), "value-alpha".to_string())]); + sandbox_ok + .register_executor(Box::new(StaticLookup::new(table2))) + .unwrap(); + let ok = sandbox_ok.execute(&call, "call_0001"); + assert_eq!(ok.status, ToolResultStatus::Ok); + } + + // --- Roadmap acceptance test 3 ---------------------------------------- + + /// A cooperative executor that loops until cancelled, used to prove the + /// wall-clock timeout fires and fails closed. + struct HangingExecutor { + name: &'static str, + invocations: Arc, + } + + impl ToolExecutor for HangingExecutor { + fn name(&self) -> &'static str { + self.name + } + fn execute( + &self, + _args: &ValidatedArgs, + ctx: &ExecContext, + ) -> Result { + self.invocations.fetch_add(1, Ordering::SeqCst); + let started = Instant::now(); + // Busy-wait but poll the cancellation flag so a cooperative + // exit is possible; we still expect the sandbox to time out + // before this returns on its own. + while !ctx.is_cancelled() { + if started.elapsed() > Duration::from_secs(30) { + // Hard backstop so a test regression cannot hang CI. + return Err(ExecError::Executor( + self.name.into(), + "hard backstop fired".into(), + )); + } + std::thread::sleep(Duration::from_millis(5)); + } + Err(ExecError::Timeout( + self.name.into(), + ctx.limits().timeout_ms, + )) + } + } + + #[test] + fn execution_timeout_kills_a_hanging_tool_call() { + let invocations = Arc::new(AtomicUsize::new(0)); + let limits = SandboxLimits { + timeout_ms: 120, + ..SandboxLimits::default() + }; + let mut sandbox = + ToolSandbox::new(AuthorizationScope::new(["hang"]).unwrap(), limits).unwrap(); + let hang_def = ToolDefinition { + name: "hang".into(), + description: String::new(), + parameters: json!({"type":"object","properties":{}}), + }; + sandbox.register_definitions(&[hang_def]).unwrap(); + sandbox + .register_executor(Box::new(HangingExecutor { + name: "hang", + invocations: invocations.clone(), + })) + .unwrap(); + let call = ToolCall { + name: "hang".into(), + arguments: json!({}), + }; + let started = Instant::now(); + let result = sandbox.execute(&call, "call_0001"); + let elapsed = started.elapsed(); + assert_eq!(result.status, ToolResultStatus::Error); + assert!( + result + .error + .as_deref() + .unwrap() + .contains("wall-clock ceiling") + ); + // The call returned promptly after the timeout (plus slack), not + // after the 30s backstop. + assert!(elapsed < Duration::from_secs(2), "took {elapsed:?}"); + // The executor was actually entered once (the refusal was an + // execution-time timeout, not a pre-execution refusal). + assert_eq!(invocations.load(Ordering::SeqCst), 1); + } + + // --- Roadmap acceptance test 4 ---------------------------------------- + + #[test] + fn execution_respects_configured_memory_and_cpu_ceiling() { + // (a) Output-size (memory) ceiling: an executor returning more than + // max_output_bytes is refused with ResourceLimitExceeded. + let limits = SandboxLimits { + max_output_bytes: 16, + ..SandboxLimits::default() + }; + let mut sandbox = + ToolSandbox::new(AuthorizationScope::new(["lookup"]).unwrap(), limits).unwrap(); + sandbox + .register_definitions(&[lookup_definition()]) + .unwrap(); + let big = "x".repeat(64); + let table = HashMap::from([("alpha".to_string(), big)]); + sandbox + .register_executor(Box::new(StaticLookup::new(table))) + .unwrap(); + let call = ToolCall { + name: "lookup".into(), + arguments: json!({"key":"alpha"}), + }; + let result = sandbox.execute(&call, "call_0001"); + assert_eq!(result.status, ToolResultStatus::Error); + assert!(result.error.as_deref().unwrap().contains("output_bytes")); + + // (b) CPU ceiling: a compute-bound executor that loops is bounded + // by the wall-clock timeout_ms (CPU time == wall-clock for + // compute-bound work on a single thread). + let cpu_limits = SandboxLimits { + timeout_ms: 100, + ..SandboxLimits::default() + }; + let mut cpu_sandbox = + ToolSandbox::new(AuthorizationScope::new(["burn"]).unwrap(), cpu_limits).unwrap(); + let burn_def = ToolDefinition { + name: "burn".into(), + description: String::new(), + parameters: json!({"type":"object","properties":{}}), + }; + cpu_sandbox.register_definitions(&[burn_def]).unwrap(); + cpu_sandbox + .register_executor(Box::new(HangingExecutor { + name: "burn", + invocations: Arc::new(AtomicUsize::new(0)), + })) + .unwrap(); + let burn_call = ToolCall { + name: "burn".into(), + arguments: json!({}), + }; + let started = Instant::now(); + let burn_result = cpu_sandbox.execute(&burn_call, "call_0001"); + assert_eq!(burn_result.status, ToolResultStatus::Error); + assert!( + burn_result + .error + .as_deref() + .unwrap() + .contains("wall-clock ceiling") + ); + assert!(started.elapsed() < Duration::from_secs(2)); + } + + // --- Roadmap acceptance test 5 ---------------------------------------- + + #[test] + fn malformed_tool_call_json_is_never_executed() { + let attempts = Arc::new(AtomicUsize::new(0)); + struct CountingLookup { + attempts: Arc, + } + impl ToolExecutor for CountingLookup { + fn name(&self) -> &'static str { + "lookup" + } + fn execute( + &self, + _args: &ValidatedArgs, + _ctx: &ExecContext, + ) -> Result { + self.attempts.fetch_add(1, Ordering::SeqCst); + Ok(ToolResultContent::Text { text: "x".into() }) + } + } + let mut sandbox = ToolSandbox::new( + AuthorizationScope::new(["lookup"]).unwrap(), + SandboxLimits::default(), + ) + .unwrap(); + sandbox + .register_definitions(&[lookup_definition()]) + .unwrap(); + sandbox + .register_executor(Box::new(CountingLookup { + attempts: attempts.clone(), + })) + .unwrap(); + // Schema requires `key` (string) and disallows extra properties; + // this call omits `key` and adds a disallowed property -> + // schema-invalid -> never executed. + let malformed = ToolCall { + name: "lookup".into(), + arguments: json!({"not_a_key": 42}), + }; + let result = sandbox.execute(&malformed, "call_0001"); + assert_eq!(result.status, ToolResultStatus::Error); + assert!( + result + .error + .as_deref() + .unwrap() + .contains("schema validation") + ); + assert_eq!(attempts.load(Ordering::SeqCst), 0); + } + + // --- Roadmap acceptance test 6 ---------------------------------------- + + #[test] + fn execution_result_re_ingests_correctly_into_the_next_chain_step() { + // Drive the real ToolChain with a FakeDecoder that emits a `lookup` + // tool call, then a final answer. The SandboxedToolProvider + // executes the call and the result re-enters the chain via the + // existing result_ingestion path (proven by the chain state + // recording the executed content and reaching the final turn). + use crate::chain::{ToolChain, ToolChainConfig}; + use aarambh_studio_inference::{FinishReason, GenerationOutput, GenerationUsage}; + use std::collections::VecDeque; + + struct FakeDecoder { + outputs: VecDeque, + } + impl crate::chain::ChainDecoder for FakeDecoder { + fn context_limit(&self) -> usize { + 64 + } + fn encode_prefix( + &mut self, + _prompt: &str, + _tools: &[ToolDefinition], + _summary: Option<&str>, + ) -> crate::AgentResult> { + Ok(vec![1, 2]) + } + fn encode_result(&mut self, _result: &ToolResult) -> crate::AgentResult> { + Ok(vec![8, 9]) + } + fn encode_result_metadata( + &mut self, + _result: &ToolResult, + ) -> crate::AgentResult> { + Ok(vec![8]) + } + fn generate( + &mut self, + _transcript_ids: &[u32], + _pending_media: Option<&ToolResultContent>, + _max_new_tokens: usize, + ) -> crate::AgentResult { + self.outputs + .pop_front() + .ok_or_else(|| crate::AgentError::Config("fake output exhausted".into())) + } + fn summarise( + &mut self, + _previous_summary: Option<&str>, + _evicted: &[crate::ToolExchange], + _max_tokens: usize, + ) -> crate::AgentResult { + Ok("summary".into()) + } + } + + let call = ToolCall { + name: "lookup".into(), + arguments: json!({"key":"alpha"}), + }; + let first = GenerationOutput { + text: String::new(), + raw_text: String::new(), + thinking_text: String::new(), + answer_text: String::new(), + token_ids: vec![4, 5], + thinking_token_ids: Vec::new(), + answer_token_ids: vec![4, 5], + thinking_tokens: 0, + finish_reason: FinishReason::ToolCall, + steps: Vec::new(), + speculative_stats: None, + tool_call: Some(call.clone()), + usage: GenerationUsage { + prompt_tokens: 2, + completion_tokens: 2, + total_tokens: 4, + }, + }; + let second = GenerationOutput { + text: "done".into(), + raw_text: "done".into(), + thinking_text: String::new(), + answer_text: "done".into(), + token_ids: vec![6, 7], + thinking_token_ids: Vec::new(), + answer_token_ids: vec![6, 7], + thinking_tokens: 0, + finish_reason: FinishReason::EosToken, + steps: Vec::new(), + speculative_stats: None, + tool_call: None, + usage: GenerationUsage { + prompt_tokens: 4, + completion_tokens: 2, + total_tokens: 6, + }, + }; + let decoder = FakeDecoder { + outputs: [first, second].into(), + }; + let mut sandbox = ToolSandbox::new( + AuthorizationScope::new(["lookup"]).unwrap(), + SandboxLimits::default(), + ) + .unwrap(); + sandbox + .register_definitions(&[lookup_definition()]) + .unwrap(); + let table = HashMap::from([("alpha".to_string(), "value-alpha".to_string())]); + sandbox + .register_executor(Box::new(StaticLookup::new(table))) + .unwrap(); + let provider = SandboxedToolProvider::new(sandbox); + let mut chain = ToolChain::new(decoder, provider, ToolChainConfig::default()).unwrap(); + let output = chain.run("find it", vec![lookup_definition()]).unwrap(); + assert_eq!(output.final_output.text, "done"); + assert_eq!(output.metrics.tool_calls, 1); + // The executed result re-entered the chain: the recorded exchange + // carries the sandbox-produced content, not a caller-supplied value. + let exchange = &output.state.exchanges()[0]; + assert_eq!(exchange.result.status, ToolResultStatus::Ok); + assert_eq!( + exchange.result.content, + Some(ToolResultContent::Text { + text: "value-alpha".into() + }) + ); + assert_eq!(exchange.request.call, call); + } + + // --- Supporting tests ------------------------------------------------- + + #[test] + fn read_file_executor_refuses_traversal_escape() { + let dir = tempfile_workdir(); + let mut sandbox = ToolSandbox::new( + AuthorizationScope::new(["read_file_in_workdir"]).unwrap(), + SandboxLimits::default(), + ) + .unwrap(); + sandbox + .register_definitions(&[read_file_definition()]) + .unwrap(); + sandbox + .register_executor(Box::new(ReadFileInWorkdir::new(&dir).unwrap())) + .unwrap(); + // Absolute path -> refused. + let abs = ToolCall { + name: "read_file_in_workdir".into(), + arguments: json!({"path":"/etc/passwd"}), + }; + let r = sandbox.execute(&abs, "call_0001"); + assert_eq!(r.status, ToolResultStatus::Error); + // `..` traversal -> refused. + let traversal = ToolCall { + name: "read_file_in_workdir".into(), + arguments: json!({"path":"../secret.txt"}), + }; + let r = sandbox.execute(&traversal, "call_0001"); + assert_eq!(r.status, ToolResultStatus::Error); + // A file inside the workdir -> ok. + fs::write(dir.join("hello.txt"), "hi there").unwrap(); + let ok = ToolCall { + name: "read_file_in_workdir".into(), + arguments: json!({"path":"hello.txt"}), + }; + let r = sandbox.execute(&ok, "call_0001"); + assert_eq!(r.status, ToolResultStatus::Ok); + assert_eq!( + r.content, + Some(ToolResultContent::Text { + text: "hi there".into() + }) + ); + } + + #[test] + fn limits_validation_rejects_zero_ceilings() { + assert!( + ToolSandbox::new( + AuthorizationScope::empty(), + SandboxLimits { + timeout_ms: 0, + ..SandboxLimits::default() + } + ) + .is_err() + ); + assert!( + ToolSandbox::new( + AuthorizationScope::empty(), + SandboxLimits { + max_output_bytes: 0, + ..SandboxLimits::default() + } + ) + .is_err() + ); + } + + #[test] + fn sandboxed_provider_returns_chain_valid_result() { + let mut sandbox = ToolSandbox::new( + AuthorizationScope::new(["lookup"]).unwrap(), + SandboxLimits::default(), + ) + .unwrap(); + sandbox + .register_definitions(&[lookup_definition()]) + .unwrap(); + let table = HashMap::from([("alpha".to_string(), "value-alpha".to_string())]); + sandbox + .register_executor(Box::new(StaticLookup::new(table))) + .unwrap(); + let mut provider = SandboxedToolProvider::new(sandbox); + let request = ToolResultRequest { + call_id: "call_0001".into(), + call: ToolCall { + name: "lookup".into(), + arguments: json!({"key":"alpha"}), + }, + }; + let result = provider.next_result(&request).unwrap(); + assert_eq!(result.call_id, "call_0001"); + assert_eq!(result.status, ToolResultStatus::Ok); + } + + fn tempfile_workdir() -> PathBuf { + let dir = std::env::temp_dir().join(format!( + "aarambh-sandbox-{}", + std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .unwrap() + .as_nanos() + )); + fs::create_dir_all(&dir).unwrap(); + dir + } +} diff --git a/docs/README.md b/docs/README.md index 0ea5482..f65cf1c 100644 --- a/docs/README.md +++ b/docs/README.md @@ -59,6 +59,17 @@ This documents caller-executed result ingestion, exact-token continuation, context eviction/summarisation, multimodal result lifetime, safety checks, multi-step SFT, scripted evaluation, and the BFCL response-path boundary. +### 4b. `phase47_sandbox.md` +**The sandboxed tool-execution runbook (Phase 47).** + +This documents the closed-world `ToolExecutor` model, operator +authorization (`AuthorizationScope`), the bounded execution envelope +(wall-clock timeout + output/argument-size ceilings + schema +re-validation), the `SandboxedToolProvider` composability bridge into the +existing `ToolChain`, the reference `read_file_in_workdir` and `lookup` +executors, the `agent --execute-tools` CLI surface, the smoke workflow, +and the honesty boundary (pure-Rust CPU sandbox, no OS-level isolation). + ### 5. `phase38_forgetting.md` **Capability regression, MoE routing drift, and the Manas JSONL bridge.** diff --git a/docs/phase47_sandbox.md b/docs/phase47_sandbox.md new file mode 100644 index 0000000..eb8c735 --- /dev/null +++ b/docs/phase47_sandbox.md @@ -0,0 +1,318 @@ +# Phase 47 — Sandboxed Tool Execution + +> From first principles. From zero. From Rust. +> +> Phase 47 (`ARCHITECTURE_V4.md` §61) closes the boundary v2 §30 opened +> (tool calls are emitted, never executed) and v3 §46 extended (multi-step +> chains, still emit-only): the model's tool calls can now be **actually +> executed** by aarambh-studio itself, but only inside a strict, explicit, +> closed-world sandbox. + +This is the runbook for the sandboxed-execution capability shipped in +`v4.0.0-alpha.7`. It documents the design, the operator-facing CLI, the +reference executors, the smoke workflow, and the honesty boundary. + +--- + +## Why this phase exists + +Before Phase 47, aarambh-studio's tool-use story was **emit-only**: + +- **v2 §30** — the model emits grammar-constrained JSON tool calls. A + developer's own integration reads those calls and decides whether to + execute them. aarambh-studio never executes anything. +- **v3 §46** — the model emits *sequences* of tool calls, using real + intermediate results fed back by the caller. Still emit-only — the caller + (a human, a script, an external service) does the execution. + +Phase 47 makes execution a first-class, opt-in capability of the runtime +itself — but scoped as narrowly as the risk demands. There is **no generic +"run a shell command" or "eval this code" executor** anywhere in the crate, +by design. Every executor implements one specific, named capability, and an +unrecognised tool name is a hard refusal, never a best-effort fallback +attempt. + +This is the highest-risk phase before Phase 51 (the public inference +server) and is deliberately placed before Phase 48 (multi-agent +orchestration), which depends on it completely — orchestration is only as +safe as the execution sandbox underneath it. + +--- + +## The execution envelope + +`ToolSandbox::execute` enforces the full `ARCHITECTURE_V4.md` §61 pipeline, +in order: + +``` +Model emits grammar-constrained JSON tool call (v2 §30) + │ + ▼ +1. Closed-world allowlist — the name must match a REGISTERED executor. + Not registered → ExecError::UnknownTool (hard refusal, no attempt) + │ + ▼ +2. Operator authorization — the name must be in the operator's + AuthorizationScope (set at startup via --allow-tool). + Not authorized → ExecError::Unauthorized (distinct from UnknownTool: + the capability exists, the operator simply did not enable it) + │ + ▼ +3. Argument-size ceiling — serialized args ≤ max_args_bytes (default 8 KiB). + Exceeded → ExecError::ResourceLimitExceeded { kind: ArgumentBytes } + │ + ▼ +4. Schema re-validation — args re-validated against the declared JSON + Schema (defense-in-depth on top of the grammar-constrained decoder). + Invalid → ExecError::InvalidArgs (malformed calls are NEVER executed) + │ + ▼ +5. Bounded envelope — worker thread + recv_timeout wall-clock ceiling + (default 5 s). Cooperative cancellation via an AtomicBool flag. + Exceeded → ExecError::Timeout (thread detached, fail-closed) + │ + ▼ +6. Output-size ceiling — result payload ≤ max_output_bytes (default 64 KiB). + Exceeded → ExecError::ResourceLimitExceeded { kind: OutputBytes } + │ + ▼ +ToolResult re-enters the chain via v3 §46's existing result_ingestion +path — no new ingestion mechanism, execution is purely additive. +``` + +Every failure yields a **fail-closed** `ToolResult{status:Error, +error:...}`. The chain records the refusal and continues — it never +silently drops a call or attempts a best-effort fallback. + +--- + +## Composability: zero chain changes + +The key architectural decision is that execution is **additive**. +`SandboxedToolProvider` implements the existing `ToolResultProvider` trait: + +```rust +pub struct SandboxedToolProvider { sandbox: ToolSandbox } + +impl ToolResultProvider for SandboxedToolProvider { + fn next_result(&mut self, request: &ToolResultRequest) + -> AgentResult + { + let result = self.sandbox.execute(&request.call, &request.call_id); + result.validate_for(&request.call_id)?; // belt-and-braces + Ok(result) + } +} +``` + +The existing `ToolChain::run_with_callback` calls `next_result` — it does +not know or care whether the result came from a caller's stdin line, a +scripted replay file, or a sandboxed executor. The chain, the context +eviction, the multimodal result lifetime, and the safety checks are all +unchanged. Phase 47 adds a *provider*; it does not touch the chain. + +--- + +## Operator authorization + +Authorization is an **operator** decision, not a model decision. A model +can *declare* any tool in its schema and *request* execution of anything +it declares. Whether that request is ever carried out depends entirely on +what the operator explicitly enabled at startup: + +```rust +pub struct AuthorizationScope { enabled: BTreeSet } + +impl AuthorizationScope { + pub fn empty() -> Self; + pub fn enable(&mut self, name: &str) -> Result<(), AgentError>; + pub fn is_authorized(&self, name: &str) -> bool; + pub fn intersect(&self, other: &Self) -> Self; // Phase 48 sub-agents +} +``` + +`AuthorizationScope::intersect` supports Phase 48's multi-agent +orchestration: a sub-agent's authorized scope can only be a **subset** of +its orchestrator's. Orchestration can never escalate tool access beyond +what the operator enabled at the top level. + +--- + +## Reference executors + +Phase 47 ships two `ToolExecutor` implementations: + +### `ReadFileInWorkdir` — the milestone executor + +> "A read-only file lookup within a fixed working directory." + +```rust +pub struct ReadFileInWorkdir { workdir: PathBuf } + +impl ToolExecutor for ReadFileInWorkdir { + fn name(&self) -> &'static str { "read_file_in_workdir" } + fn execute(&self, args: &ValidatedArgs, ctx: &ExecContext) + -> Result; +} +``` + +- Reads a UTF-8 text file by **relative** name from a fixed, canonicalized + working directory. +- **Refuses** absolute paths and any `..` traversal component *before* + joining, so a missing file cannot be probed for path-structure + information and an absolute path cannot replace the workdir base. +- After canonicalization, double-checks the resolved path still starts with + the workdir (defense-in-depth against symlink escapes). +- Caps the returned bytes at `max_output_bytes`. +- No network access, no write access — by construction. + +### `StaticLookup` — deterministic in-memory executor + +An in-memory `key → text` map, used by tests and smoke runs that need a +closed-world capability without touching the filesystem. + +### Extension points (documented, not shipped) + +The roadmap names `http_get_whitelisted_domain` as an example executor. It +is **not** shipped because adding an HTTP client dependency would widen the +attack surface and the dependency footprint for a source release; the +`ToolExecutor` trait is the extension point for an operator who needs it. +The trait is intentionally `Send + Sync` so executors can run on worker +threads under the timeout envelope. + +--- + +## CLI: `agent --execute-tools` + +The `agent` command gains five flags: + +| Flag | Default | Purpose | +|---|---|---| +| `--execute-tools` | off | Switch from caller-executed stdin/replay results to sandboxed execution | +| `--allow-tool ` | (none) | Operator authorization (repeatable). At least one is required with `--execute-tools`. | +| `--exec-timeout-ms` | 5000 | Per-call wall-clock ceiling | +| `--exec-max-output-bytes` | 65536 | Maximum output payload bytes | +| `--exec-workdir ` | (none) | Binds the `read_file_in_workdir` executor to a directory | + +Example — execute a read-only file lookup inside a fixed workdir: + +```sh +target/release/aarambh-studio agent \ + --config configs/tiny_shakespeare.toml \ + --model checkpoints/tiny_shakespeare_smoke/best/model.safetensors \ + --tokenizer checkpoints/tiny_shakespeare_smoke/tokenizer.json \ + --tools data/tools_sandbox_smoke.json \ + --prompt "Read the file notes.txt and summarise it." \ + --execute-tools \ + --allow-tool read_file_in_workdir \ + --exec-workdir ./data/sandbox_workdir \ + --exec-timeout-ms 2000 \ + --max-steps 4 +``` + +When `--execute-tools` is **not** set, the command behaves exactly as in +v3 §46: it reads caller-supplied `ToolResult` JSON lines from stdin (or a +`--results` replay file). Execution is strictly opt-in. + +--- + +## Tool definition schema + +Tools are declared with the same JSON Schema format every other phase uses. +The `read_file_in_workdir` executor expects a `path` string: + +```json +{ + "name": "read_file_in_workdir", + "description": "Read a UTF-8 text file from the working directory.", + "parameters": { + "type": "object", + "properties": { + "path": {"type": "string"} + }, + "required": ["path"], + "additionalProperties": false + } +} +``` + +The sandbox compiles every declared schema at registration time and +re-validates arguments before execution — defense-in-depth on top of the +grammar-constrained decoder that already produced a schema-valid call. + +--- + +## Tests + +Six roadmap-named acceptance tests live in +`crates/aarambh-studio-agent/src/sandbox.rs`, each with a real body: + +| Test | Proves | +|---|---| +| `unlisted_tool_name_is_hard_refused_never_attempted` | An unregistered name is a hard refusal, no execution attempt | +| `unauthorized_but_declared_tool_is_refused_at_execution_not_declaration` | Authorization is distinct from registration; an authorized copy succeeds | +| `execution_timeout_kills_a_hanging_tool_call` | A hanging executor is abandoned after `timeout_ms`, fail-closed, promptly | +| `execution_respects_configured_memory_and_cpu_ceiling` | Output-size and wall-clock (CPU) ceilings both fire as `ResourceLimitExceeded`/`Timeout` | +| `malformed_tool_call_json_is_never_executed` | Schema-invalid args never reach the executor (verified by a counting executor) | +| `execution_result_re_ingests_correctly_into_the_next_chain_step` | A full `ToolChain` + `FakeDecoder` + `SandboxedToolProvider` re-ingests the executed result into the chain state | + +Supporting tests cover authorization `intersect`, read-file traversal +refusal, and `SandboxLimits` validation. + +--- + +## Smoke workflow + +```sh +scripts/phase47_smoke.sh +``` + +Runs the agent-crate sandbox unit tests (the deterministic proof), verifies +`agent --help` surfaces the new flags, and writes a scorecard to +`artifacts/phase47_sandbox_smoke.json`. + +The smoke follows the same honesty discipline as Phase 46: the unit tests +are the deterministic proof with fake decoders and real executors; the CLI +smoke verifies the operator-facing surface compiles and the flags appear. +Whether sandboxed execution produces *useful* model behaviour at scale is +measured by the eval harness, not asserted in prose. + +--- + +## Honesty boundary + +Phase 47's sandbox is **pure-Rust and CPU-only**: + +- **Wall-clock timeout** — cooperative cancellation via an `AtomicBool` + flag, plus thread-abandonment on timeout. Safe Rust cannot force-kill a + thread, so a non-cooperative executor that never polls `is_cancelled` is + *detached* on timeout — it cannot block the chain, but it may continue + running until it returns on its own. Every shipped executor polls the + flag inside its loops. +- **Output-size ceiling** — enforced by `ToolSandbox` after the executor + returns, defense-in-depth on top of the executor's own cap. +- **Argument-size ceiling** — enforced before schema validation. +- **Closed-world allowlist** — only registered executors run. +- **Operator authorization** — only `--allow-tool` names run. +- **Schema re-validation** — malformed calls never execute. + +**Out of scope:** OS-level isolation (seccomp, cgroups, namespaces) is not +implemented, consistent with the project's CPU-first, source-release +posture. A general-purpose code-execution sandbox remains explicitly out +of scope — Phase 47's execution is strictly closed-world, named-capability +tool execution, never arbitrary code or shell execution. + +The safety-relevant property — *a runaway or hung call never blocks the +chain and always produces a fail-closed result* — holds under every tested +failure condition (unknown tool, unauthorized tool, schema-invalid args, +hanging executor, oversized output, oversized args). + +--- + +## What this enables next + +Phase 48 (multi-agent orchestration) builds directly on this: an +orchestrator delegates sub-tasks to multiple parallel sandboxed +tool-execution chains, each governed entirely by Phase 47's boundaries, +with each sub-agent's authorized scope narrowed via +`AuthorizationScope::intersect` to a subset of the orchestrator's. diff --git a/scripts/phase47_smoke.sh b/scripts/phase47_smoke.sh new file mode 100755 index 0000000..3877dce --- /dev/null +++ b/scripts/phase47_smoke.sh @@ -0,0 +1,125 @@ +#!/usr/bin/env bash +# Phase 47 — Sandboxed Tool Execution smoke test. +# +# Validates that: +# - The Phase 47 agent-crate sandbox unit-test suite passes: an unlisted +# tool name is a hard refusal, an unauthorized-but-declared tool is +# refused at execution (not declaration), a hanging call is killed by +# the wall-clock timeout, the output/CPU ceilings fire, malformed JSON +# is never executed, and an executed result re-ingests into the chain. +# - The CLI plumbing surfaces the new operator-facing flags: +# `--execute-tools`, `--allow-tool`, `--exec-timeout-ms`, +# `--exec-max-output-bytes`, `--exec-workdir`. +# +# Per the roadmap milestone: "A whitelisted, sandboxed tool (e.g. a +# read-only file lookup within a fixed working directory) executes +# correctly end-to-end inside a multi-step chain, with every safety +# boundary (allowlist, timeout, resource cap) independently tested and +# verified to fail closed, not open, under every tested failure condition." +# The unit tests are the deterministic proof; this script also verifies +# the operator-facing CLI surface compiles and the flags appear. +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +cd "$ROOT" + +SCORECARD=${PHASE47_SCORECARD:-artifacts/phase47_sandbox_smoke.json} +mkdir -p "$(dirname "$SCORECARD")" + +echo "==> Phase 47 agent-crate sandbox unit tests" +cargo test --locked -p aarambh-studio-agent --lib sandbox +cargo test --locked -p aarambh-studio-agent --lib authorization + +echo "==> Phase 47 build a release CLI for flag checks" +cargo build --release --locked -p aarambh-studio + +echo "==> Phase 47 verify the new agent flags appear in --help" +target/release/aarambh-studio agent --help | grep -q -- "--execute-tools" +target/release/aarambh-studio agent --help | grep -q -- "--allow-tool" +target/release/aarambh-studio agent --help | grep -q -- "--exec-timeout-ms" +target/release/aarambh-studio agent --help | grep -q -- "--exec-max-output-bytes" +target/release/aarambh-studio agent --help | grep -q -- "--exec-workdir" + +echo "==> Phase 47 verify --execute-tools refuses to start without --allow-tool" +# Authorizing nothing is a configuration error: the operator must explicitly +# opt in to each executable tool. This is the operator-decision invariant. +set +e +ERR_OUTPUT=$(target/release/aarambh-studio agent \ + --config configs/tiny_shakespeare_smoke.toml \ + --tools data/tools_sandbox_smoke.json \ + --prompt "read notes.txt" \ + --execute-tools 2>&1) +ERR_EXIT=$? +set -e +if [[ "$ERR_EXIT" -eq 0 ]]; then + echo "Phase 47 smoke FAILED: --execute-tools without --allow-tool should error" + exit 1 +fi +echo "$ERR_OUTPUT" | grep -q "at least one --allow-tool" || { + echo "Phase 47 smoke FAILED: expected 'at least one --allow-tool' error, got:" + echo "$ERR_OUTPUT" + exit 1 +} + +echo "==> Phase 47 verify --exec-workdir requires the tool to be authorized" +mkdir -p data/sandbox_workdir +set +e +ERR_OUTPUT=$(target/release/aarambh-studio agent \ + --config configs/tiny_shakespeare_smoke.toml \ + --tools data/tools_sandbox_smoke.json \ + --prompt "read notes.txt" \ + --execute-tools \ + --allow-tool lookup \ + --exec-workdir data/sandbox_workdir 2>&1) +ERR_EXIT=$? +set -e +if [[ "$ERR_EXIT" -eq 0 ]]; then + echo "Phase 47 smoke FAILED: --exec-workdir without authorizing read_file_in_workdir should error" + exit 1 +fi +echo "$ERR_OUTPUT" | grep -q "was not authorized via --allow-tool" || { + echo "Phase 47 smoke FAILED: expected 'not authorized via --allow-tool' error, got:" + echo "$ERR_OUTPUT" + exit 1 +} + +echo "==> Phase 47 write scorecard" +python3 - "$SCORECARD" <<'PY' +import json, sys +scorecard = { + "phase": 47, + "title": "Tool Execution With Sandboxing", + "agent_unit_tests": "passed", + "cli_flags_surface": [ + "--execute-tools", + "--allow-tool", + "--exec-timeout-ms", + "--exec-max-output-bytes", + "--exec-workdir", + ], + "closed_world_allowlist": True, + "operator_authorization": True, + "schema_revalidation": True, + "wall_clock_timeout": True, + "output_and_args_ceilings": True, + "fail_closed_on_every_failure": True, + "no_new_crate": True, + "no_new_dependency": True, + "honesty_note": ( + "The sandbox is pure-Rust and CPU-only: wall-clock timeout " + "(cooperative cancellation + thread-detachment), output/argument " + "size ceilings, closed-world allowlist, operator authorization, and " + "schema re-validation. OS-level isolation (seccomp/cgroups) is out " + "of scope for the source release. The 6 roadmap-named acceptance " + "tests prove the safety-relevant property — a runaway or hung call " + "never blocks the chain and always produces a fail-closed result — " + "under every tested failure condition. A general-purpose " + "code-execution sandbox remains explicitly out of scope: execution " + "is strictly closed-world, named-capability tool execution." + ), +} +json.dump(scorecard, open(sys.argv[1], "w"), indent=2) +print(f"wrote {sys.argv[1]}") +PY + +echo "Phase 47 smoke completed: $SCORECARD"