Emit engine contracts as descriptors; stop prompts restating them - #86
Merged
Conversation
Root-causes the contract-drift bug class (3 rejected-control cycles across live runs, each patched by hand-editing prompt prose). The engine now emits a machine-readable contract descriptor per iteration, referenced from paths.json, derived from real engine symbols: - LayerHandoff schema from LayerHandoff.model_json_schema() - reference grammar from the LOGICAL_REFERENCE_* scope/marker constants - control fields from ControlSignal.model_json_schema()/accepted_field_names() - eval-receipt applicability from check_runner_roles ∩ receipt-producing expected_outputs (real field is eval_receipt_refs) The outer and planner prompts drop the hand-restated schema/grammar/ eval_refs prose and point at the authoritative descriptor instead, so they can no longer drift from validation. Implemented by codex (gpt-5.6-sol xhigh); pending Grok + human review. Co-Authored-By: Codex (gpt-5.6-sol) <noreply@openai.com> Claude-Session: https://claude.ai/code/session_01EPYwF6cRYs2mfsfHF1hibc
…mple, ref rules Addresses adversarial review (Grok) of the contract descriptor: 1. Stop presenting the raw multi-version ControlSignal.model_json_schema() as "conform exactly" (it under-described required-for-stopped fields and advertised fields not accepted in the active protocol, e.g. a singular eval_receipt_ref). The descriptor now exposes authoritative accepted_fields + required_fields for the active protocol, derived from ControlSignal validation; the raw schema is subordinate. 2. Restore load-bearing guidance inside the descriptor (drift-proof): a model-validated minimal goal_met example built only from accepted/required fields; evidence-ref rules incl. the path-segment restrictions the reference parser enforces (no empty/./.. segments, no backslash/NUL, not absolute after :/), and the file requirement matching the coordinator's real asserted-artifact validator. 3. Discovery: contracts path added to the diet header's key-paths line. 4. Tests: stock inner_outer_eval roster -> applicable False + eval fields omitted; older-roster compat; round-trip (build from descriptor -> ControlSignal.model_validate accepts, reject without control_id); worker asserts protocol/accepted/required/applicability; header budget rendered with a roster-carrying assignment. Co-Authored-By: Codex (gpt-5.6-sol) <noreply@openai.com> Claude-Session: https://claude.ai/code/session_01EPYwF6cRYs2mfsfHF1hibc
Documents that terminal control and the layer handoff must conform to the engine-emitted contracts file (paths.json -> contracts) rather than a prompt-restated schema: authoritative accepted/required fields + a validated goal_met example, eval-receipt applicability, the evidence-ref grammar, and the LayerHandoff schema — all derived from engine ground truth so they cannot drift. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EPYwF6cRYs2mfsfHF1hibc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Root-causes the contract-drift bug class. The engine validates several contracts (the
LayerHandoffschema, the logical-reference grammar, terminal-control fields incl. eval-receipt applicability) that the stock prompts only described in prose — and every time the prose drifted from the validator, agents produced artifacts the engine rejected, costing a rejected-terminal-control cycle per program. This session alone patched three such drifts by hand (the handoffstatus/refsfields, the reference grammar, andeval_refs).This makes the engine the single source of truth: it emits a machine-readable contract descriptor every iteration (referenced from
paths.jsonand the prompt header), derived directly from engine symbols:ControlSignalvalidation, a model-validated minimalgoal_metexample, and eval-receipt applicability fromcheck_runner_roles ∩ receipt-producing expected_outputs(the raw multi-version JSON schema is kept only as subordinate reference);LayerHandoffschema fromLayerHandoff.model_json_schema().The outer and planner prompts drop the hand-restated schema/grammar/eval prose and point at the descriptor, so they can no longer drift.
Review
Implemented by Codex (gpt-5.6-sol xhigh); adversarially reviewed by Grok (grok-4.5) and me over two rounds. Grok's first pass (SHIP-WITH-NITS) found the descriptor was shipping the raw multi-version schema as "conform exactly" (a foot-gun that under-described required-for-stopped fields and advertised fields the active version rejects), and that removed prose dropped the
goal_metexample + evidence-ref teaching. Both fixed; Grok's second pass verdict: SHIP — all findings CLOSED with file:line evidence, verified on stock protocol 3.Test plan
450 tests pass; ruff + pyright clean. New tests: stock roster → applicability False + eval fields omitted; round-trip (build from descriptor →
ControlSignal.model_validateaccepts, reject withoutcontrol_id); worker asserts protocol/accepted/required/applicability; header-budget rendered with a roster-carrying assignment. Docs:success-and-controlexplains the descriptor.🤖 Generated with Claude Code — implemented by Codex, reviewed by Grok + Claude.
https://claude.ai/code/session_01EPYwF6cRYs2mfsfHF1hibc