Add runtime execution spine for safe chain execution - #10
Merged
Conversation
Turn the planner into an executor: POST /runs/{id}/chain/execute now actually
runs a registered chain end-to-end through the ChainExecutor (the outer
authority) and returns a real ChainExecutionResult. No merge, deploy, or PR; no
new forbidden endpoints; the harness invariants are preserved.
New packages (built on the existing LLM layer; no duplication):
- app/runtime/: WorkspacePolicy (blocks empty/nonexistent/outside-root/nested-
duplicate paths), ChainExecuteRequest, RuntimeExecutor + server settings.
- app/agents/: controlled provider runtime over LLMAdapter/LLMInvocationRecorder
(AgentRuntime, AgentInvocationRequest/Result, deterministic fake + manual
adapters). Provider unavailable/failed -> BLOCKED, never a fabricated PASS.
- app/tools/: read-only tool registry + policy + executor; output is hashed and
ledgered; unregistered/forbidden/mutating-in-read-only tools are BLOCKED.
- app/parsing/: strict structured-output parser (malformed -> None) and
agent-output quarantine helpers.
AI_READINESS_AUDIT chain gains an optional, read-only controlled analyst step
(AnalysisAgentInvocationHandler -> AgentOutputQuarantineHandler ->
StructuredOutputParserHandler). The agent only informs analysis: it never
decides PASS and never enters the evidence ledger as proof; deterministic
handlers remain the proof source and the independent AnalysisVerifier decides.
When no provider is configured the step SKIPs, so the deterministic chain (and
all existing tests) stay green.
Wiring: ChainContext/ChainExecutor thread optional agent specs/adapters + tool
registry/policy; api/store records the ChainExecutionResult; GET
/runs/{id}/chain/results returns it.
Tests (+34, total 252): runtime spine end-to-end via RuntimeExecutor and the
API, workspace policy, agent runtime, tool runtime, structured parser. conftest
disables ambient git commit-signing (sandbox signing server flakes with 503),
making the pre-existing git-based tests hermetic.
Docs: README + docs/chain_of_responsibility.md updated to current truth (chains
all implemented; runtime spine; only deterministic fake/manual/stub providers —
no live provider; test count 252).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JQW4FXw46KBvMCVc2EduXT
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0c7ec46e22
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
POST /runs/{run_id}/chain/execute now refuses to let the URL identity and the
body request identity diverge. The URL run_id is authoritative and the request
is bound to the registered RunRecord before any execution:
- body.request.run_id, if non-empty, must equal the URL run_id (else 422)
- registered manifest run_id, if set, must equal the URL run_id (else 422)
- body.request.task_id, when the manifest has a task_id, must match it (else 422)
- an empty body.request.run_id is adopted from the URL run_id; an omitted
task_id is adopted from the manifest
- on any mismatch the chain does not execute and no artifacts are written under
a divergent identity
Regression tests: run_id mismatch rejected, task_id mismatch rejected, empty
run_id bound to URL, matching identity executes, and no artifacts under the
divergent body run id. 256 tests pass; git diff --check clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JQW4FXw46KBvMCVc2EduXT
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements the runtime execution spine that enables safe, end-to-end execution of registered chains through a controlled API surface. This adds the missing layer between chain definition and actual execution, introducing workspace policy enforcement, controlled agent invocation, read-only tool runtime, and structured output parsing.
Key Changes
Runtime Executor & Workspace Policy
RuntimeExecutor(app/runtime/runtime_executor.py): Orchestrates chain execution by binding workspace policy, artifact storage, agent configuration, and tool registry to the existingChainExecutorWorkspacePolicy(app/runtime/workspace_policy.py): Validates execution paths against an allowed root, blocking empty paths, nonexistent paths, paths outside the root, and nested duplicate segmentsChainExecuteRequest(app/runtime/execution_request.py): Separates the routingChainRequest(canonical identity) from the real filesystem execution path (server-validated)Controlled Agent Runtime
AgentRuntime(app/agents/runtime.py): Single, role-bound model invocation with provider allowlisting and availability enforcement. ReturnsBLOCKED(never fabricatedPASS) if no model is bound to the role or provider is unavailableAgentInvocationRequest/Result(app/agents/provider.py): Thin types over the LLM layer for controlled invocation and configurationFakeAnalysisAdapter(app/agents/fake_provider.py): Deterministic, keyless stub provider for testing the full spineManualAnalysisAdapter(app/agents/manual_provider.py): Caller-supplied output path for deterministic executionRead-Only Tool Runtime
ToolRegistry(app/tools/registry.py): Immutable set of allowed read-only tools (list_repo_tree, discover_commands, inspect_ci_config, inspect_dependencies) with forbidden tool name enforcementToolExecutor(app/tools/executor.py): Runs registered, policy-approved tools and records output as hashed, ledgered artifacts. Blocks unregistered, forbidden, or policy-disallowed toolsToolPolicy(app/tools/policy.py): Decides tool execution based on registration status and run type (read-only modes block mutating tools)ToolResult(app/tools/results.py): Execution result record; success requires an artifact hashStructured Output Parsing
AnalysisAgentOutput&parse_structured_output()(app/parsing/structured_parser.py): Strict, repair-free schema validation of model output. ReturnsNoneon malformed JSON, wrong shape, or extra keys so the caller canBLOCKquarantine_agent_output()(app/parsing/quarantine.py): Routes raw model output and summaries throughChainContext.write_quarantined()so they never become evidence (Section 6.2 quarantine rule)API Endpoint
POST /runs/{run_id}/chain/execute(app/api/routes_chains.py): Safe execution endpoint that validates workspace path, executes the registered chain end-to-end, and returns a realChainExecutionResult. Blocks on workspace policy violations with explicit reasonsHandler Integration
AnalysisAgentInvocationHandler(app/handlers/analysis.py): New read-only handler that runs repo tools through the controlled tool runtime, invokes the declared analyst model, and quarantines output. Skips if no agent configured; blocks if provider unavailableNotable Implementation Details
used_as_evidence=False; only deterministic handlers produce evidenceBLOCKEDstatus, never fabricatedPASShttps://claude.ai/code/session_01JQW4FXw46KBvMCVc2EduXT