Skip to content

spec: cut OA 1.6.0 — runtime redefined around contracts, pipeline, cost observability, sandboxing#90

Merged
aswhitehouse merged 2 commits into
mainfrom
claude/oas-1-6-0-spec-runtime-3agp6o
Jul 21, 2026
Merged

spec: cut OA 1.6.0 — runtime redefined around contracts, pipeline, cost observability, sandboxing#90
aswhitehouse merged 2 commits into
mainfrom
claude/oas-1-6-0-spec-runtime-3agp6o

Conversation

@sgriffiths

@sgriffiths sgriffiths commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

What changed

Cuts the OA 1.6.0 formal specification (spec/open-agent-spec-1.6.md, status: Release) and canonical schema (spec/schema/oas-schema-1.6.json), superseding the 1.5 draft. The runtime is redefined around four pillars — typed contracts, a deterministic execution pipeline, first-class usage/cost observability, and declarative sandboxing — and the README is repositioned to lead with cost efficiency & effectiveness as headline advantages.

Why

After the 1.6.0 release, the formal spec lagged the actual runtime badly:

  • sandbox:/IIS, history threading, and input immutability were implemented (since 1.5.0) and marketed in the README, but defined nowhere in the formal spec or canonical schema — the 1.5.0 changelog claimed they were folded in, but they never were.
  • The 1.6.0 usage/cost work was folded into a doc still labelled "1.5.0 Draft", buried as an envelope footnote (§8.3).
  • Cost efficiency is a core differentiator for OA post-1.6.0, so the spec now states it as a design goal (§1.1) and the README calls it out up front.

Newly formalised in 1.6.0:

  • sandbox: declaration, per-key task-override resolution, the three SANDBOX_* error codes, and the honesty rule (§11, §13.2)
  • Reserved history input convention and its [system, history..., user] injection semantics (§6.4, §8.3)
  • Chain-wide input immutability — deep copy at every task boundary (§7.2)
  • Diamond-DAG legality and the linear-time cycle-detection requirement (from the 2^N lattice fix; §7.3)
  • Per-engine default-endpoint isolation (from the endpoint-clobbering fix; §5.2)
  • Usage/cost promoted to its own normative section (§10): normalisation, multi-call summation, no-guessed-rates, fail-closed pricing override
  • Certified conformance via the runtime-agnostic adapter protocol and capability registry (§14.2)

The schema adds root-level and task-level sandbox definitions. Everything is additive: every valid 1.5.x document remains a valid 1.6.0 document.

How tested

  • All existing tests pass (pytest tests/ — 498 passed, 10 skipped)
  • New tests added (if applicable) — docs/schema only, no runtime code changed
  • New schema passes Draft-07 meta-validation; all bundled example specs validated against it (examples/sandboxed-agent now validates formally; the 7 pre-existing failures are byte-identical under the 1.5 schema)
  • Confirmed both runtimes accept open_agent_spec: "1.6.0" (Python validator + npm loader pattern)
  • Verified no code/CI references pin the versioned 1.5 artifacts

Type of change

  • Documentation
  • Bug fix
  • New feature
  • Breaking change
  • Refactoring / CI / tooling

Checklist

  • Code follows project style (ruff check . && ruff format --check . — no Python touched)
  • Self-review completed
  • Version bumped if needed — spec artifacts and doc references moved to 1.6.0; pyproject.toml untouched (no runtime change)

@vercel

vercel Bot commented Jul 12, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
open-agent-spec Ready Ready Preview, Comment Jul 21, 2026 12:29am

Request Review

…st observability, sandboxing

Promote the 1.5 draft to a release-status 1.6.0 formal spec, restructured
so the runtime is defined as a deterministic pipeline with explicit design
goals (predictability, cost transparency, right-sized inference, bounded
execution). Newly formalised — previously implemented but spec'd nowhere:

- sandbox:/IIS declaration, per-key override resolution, SANDBOX_* codes
- reserved history input convention and message-list injection semantics
- chain-wide input immutability at every task boundary
- diamond-DAG legality and the linear-time cycle-detection requirement
- per-engine default-endpoint isolation
- certified conformance via the runtime-agnostic adapter protocol

Usage/cost reporting is promoted from an envelope footnote (old §8.3) to
its own normative section (§10) covering normalisation, multi-call
summation, no-guessed-rates, and fail-closed pricing overrides.

Cut oas-schema-1.6.json (1.5 + root/task sandbox definitions; additive —
every valid 1.5.x document remains valid; all bundled examples verified).
Reposition README to lead with cost efficiency & effectiveness, point the
spec table at the 1.6 artifacts, and mark the 1.5 doc superseded.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0166kWcJuF5ip5pbVmmGL4QU
@sgriffiths
sgriffiths force-pushed the claude/oas-1-6-0-spec-runtime-3agp6o branch from 9b685ce to 1b5346f Compare July 16, 2026 07:26
@sgriffiths
sgriffiths marked this pull request as ready for review July 16, 2026 07:34
@sgriffiths
sgriffiths requested a review from aswhitehouse as a code owner July 16, 2026 07:34

@aswhitehouse aswhitehouse left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good cut overall — this is the right moment to promote the formal spec out of the lag it's been in since 1.5, and the four-pillar framing (contracts / deterministic pipeline / cost observability / sandboxing) is a clean public story. Schema is Draft-07-valid, additive over 1.5, examples/sandboxed-agent validates against it, and CI is green.

One blocking normative contradiction, plus a few things that should land before calling this Status: Release.


Blocking: §7.3 contradicts itself on transitive execution

Step 2a says:

Execute the dependency task recursively (applying the same semantics).

Two paragraphs later:

Only the direct dependencies of the invoked task are executed; depends_on is not transitive execution.

Those cannot both be true. The second paragraph matches the reference runtimes (_resolve_chain_run_single_task, which does not re-enter _resolve_chain) and is what the depends-on/diamond-graph and depends-on/deep-lattice conformance cases encode. An independent implementor following step 2a literally would run transitive deps and fail both cases.

Please rewrite step 2 so it's unambiguous, e.g.:

For each direct dependency in listed order: execute that dependency as a single-task invocation (its own depends_on, if any, is not resolved as part of this call), then merge its output into the running input map.

Cycle detection remains transitive (walk the full graph); execution stays direct-only. That distinction is the whole point of the lattice work and should be crystal-clear in a Release-status doc.


Should fix before merge

1. Envelope §9.2 vs usage §10.1 — MUST vs SHOULD tension.
§9.2 lists usage inside the envelope a runtime "MUST return". §10.1 says a runtime "SHOULD include a usage key" and that it MUST be null/omitted when the engine reports no counts. Make §9.2 mark usage as optional (present when known, null/omitted otherwise) so the two sections agree — otherwise a local/Codex runtime that correctly omits usage is non-conformant by §9.2 and conformant by §10.

2. Bundled runtime schema has drifted from the canonical one.
oas_cli/schemas/oas-schema.json (what oa validate actually loads) still has no sandbox. So after this PR, the "canonical" schema accepts sandboxed specs and the CLI schema doesn't know they exist. Either:

  • sync the bundled schema to 1.6 (preferred — one source of truth), or
  • call out explicitly in the PR/docs that the formal schema and the CLI schema are deliberately different until a follow-up.

Leaving them silently divergent undercuts "an independent implementor can build from the formal spec alone" when our own reference CLI doesn't use that schema.

3. Changelog placement.
The 1.6.0 package already shipped (changelog has ## [1.6.0] - 2026-07-05). This formal-spec cut is currently under a buried ## [Unreleased] below that heading. Either fold it into the 1.6.0 block as a documentation amendment of the already-released version, or keep it under a top-of-file [Unreleased] for 1.6.1 — but don't leave Unreleased below a released section. Also, Status: Release on the doc while the changelog calls it Unreleased is a mild inconsistency.


Nits (non-blocking)

  • §8.3 history honesty. "MUST NOT silently drop… SHOULD reject or warn" is softer than the sandbox honesty rule in §11.3. Prefer the same shape: a runtime that does not claim history MUST refuse (or ignore-with-structured-warning) rather than silently drop. Silent drop is how chat agents "forget".
  • agent.role enum still rejects assistant / writer — the 7 example failures are correctly acknowledged as pre-existing, but if this is the Release cut, loosening that enum (or dropping it) would be a cheap win so the multi-agent examples validate against the formal schema.
  • Conformance cases still embed open_agent_spec: "1.5.0". Fine under the additive guarantee, but a one-line note in spec/conformance/README.md that cases pin the minimum version they require (not the suite version) would prevent future "why doesn't the 1.6 suite use 1.6?" confusion.

What's solid

  • Design goals (§1.1) tying cost transparency and cheap validation to the runtime model — this is the right public framing for post-1.6 OA.
  • Sandbox §11 is faithful to the runtime (per-key override, resolved paths, three error codes, honesty rule).
  • Linear-time cycle-detection requirement + diamond legality (§7.3) correctly captures #88.
  • Usage/cost promoted to its own section with fail-closed pricing — much better than the old envelope footnote.
  • Adapter protocol / capability registry called out in §14.2.
  • Schema additive; sandboxed-agent validates; CI green.

Happy to re-review once the §7.3 contradiction is resolved — that's the one that would actually mislead an implementor.

Comment thread spec/open-agent-spec-1.6.md Outdated
Comment thread spec/open-agent-spec-1.6.md
Comment thread CHANGELOG.md Outdated
…; sync bundled schema to 1.6

Address aswhitehouse's review on #90:

- §7.3 (blocking): dependency execution is now explicitly direct-only —
  each direct dependency runs as a single-task invocation whose own
  depends_on is never resolved — while cycle detection remains transitive.
  Also documents that each dependency receives a deep copy of the running
  merged input, matching both reference runtimes.
- §9.2: usage is CONDITIONAL in the envelope, reconciling with §10.1 so
  a runtime whose engine reports no counts is unambiguously conformant.
- Bundled runtime schema (oas_cli/schemas/oas-schema.json) synced to the
  canonical 1.6 schema — one source of truth; oa validate now knows
  sandbox:.
- CHANGELOG: [Unreleased] moved above [1.6.0]; stale pre-1.4 notes
  carried along, labelled as older.
- §8.3: history honesty rule strengthened to match the sandbox rule
  (MUST refuse or surface a structured warning; silent drop is a
  conformance violation).
- agent.role loosened to a free-form string in the canonical + bundled
  schemas and §4.1 (informational field, open set, well-known values
  RECOMMENDED); role test updated for the new semantics.
- Multi-agent persona examples gain the required agent.description —
  every bundled example now validates against the canonical 1.6 schema.
- Conformance README: cases pin the minimum spec version they require,
  not the suite version.

503 tests pass; all 16 example specs validate against oas-schema-1.6.json.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0166kWcJuF5ip5pbVmmGL4QU

Copy link
Copy Markdown
Contributor Author

All review items addressed in 6456af4 — ready for re-review.

Blocking — §7.3: rewritten per your suggested shape. Step 2 now reads "execute that dependency as a single-task invocation … its own depends_on declarations participate in cycle detection (step 1) but are NOT resolved or executed as part of this invocation", with an explicit "execution is direct-only; cycle detection is transitive" summary line. Also documents (verified against _resolve_chain) that each dependency receives a deep copy of the running merged map, so later-listed deps observe earlier outputs.

Should-fix:

  1. §9.2 now marks usage CONDITIONAL — required keys are everything else; omitting/nulling usage when the engine reports no counts is explicitly conformant.
  2. Bundled schema synced: oas_cli/schemas/oas-schema.json is now byte-identical to spec/schema/oas-schema-1.6.json (it was 1.5-equivalent apart from one description string, so the sync was clean). One source of truth; oa validate now knows sandbox:.
  3. CHANGELOG: [Unreleased] moved above [1.6.0]; the stranded pre-1.4 notes moved with it, labelled as older. Kept the spec doc at Status: Release since it documents the already-shipped 1.6.0 feature set — but happy to fold the entries into the 1.6.0 block instead if you'd prefer that reading.

Nits (took all three): §8.3 history honesty now mirrors §11.3 (MUST refuse or surface a structured warning; silent drop is a conformance violation). agent.role loosened to a free-form string in §4.1 + both schemas (well-known values RECOMMENDED), with the role test updated for open-set semantics; the personas also gained their required agent.description, so all 16 bundled examples now validate against the canonical 1.6 schema. Conformance README notes cases pin the minimum spec version they require.

503 tests pass locally; ruff clean.


Generated by Claude Code

@aswhitehouse aswhitehouse left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed 6456af4 — all requested changes addressed cleanly.

Item Status
§7.3 direct-only vs transitive contradiction ✅ Fixed — execution is explicit direct-only; cycle detection stays transitive; deep-copy of running input documented
§9.2 / §10.1 usage MUST vs SHOULD ✅ Fixed — usage is CONDITIONAL in the envelope
Bundled schema drift ✅ Fixed — oas_cli/schemas/oas-schema.jsonoas-schema-1.6.json, includes sandbox:
Changelog [Unreleased] placement ✅ Fixed — moved above [1.6.0]
History honesty rule ✅ Strengthened to match sandbox
agent.role enum ✅ Open string; persona examples get description; role test updated
Conformance cases pin note ✅ Added

CI green (test + conformance). Ship it.

One optional follow-up, not blocking: §8.3's "surface a structured warning" doesn't define a warning shape or code. Fine for now — if we ever grow a warnings channel on the envelope, wire it there.

@aswhitehouse
aswhitehouse merged commit 109eaf9 into main Jul 21, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants