Purpose
Track the only remaining external acceptance gate for promoting 1.1.0-rc.1 to stable. Static validation, fixtures, policy checks, property tests, mutation testing, local security review, reproducible builds, and GitHub Actions are necessary but do not prove real model behavior.
Required evaluation
Run the committed live behavioral-evaluation harness against a pinned authenticated Codex model using the committed held-out pressure cases.
Minimum execution contract:
- baseline and skill-enabled variants;
- at least three independent runs per case and variant;
- clean isolated Git workspace per run;
- pinned model identifier and recorded Codex version;
- immutable run manifests and trace digests;
- no manual editing of generated outcomes before grading.
Required metrics
Record and publish at least:
- routing and outcome compliance;
- gate-compliance rate;
- scope-escape rate;
- fabricated-evidence rate;
- policy-violation rate;
- unnecessary-action count;
- rollback success rate;
- human-correction rate;
- pass@1 and pass@3;
- tokens, tool calls, duration, and cost where available.
Acceptance criteria
- all configured promotion thresholds in the repository pass;
- no critical policy or evidence-fabrication violation occurs in any run;
- generated report validates against the committed schema and is bound to the evaluated commit, model, policy digest, and harness version;
- limitations and failed cases remain visible;
- an independent reviewer verifies the report and promotion decision.
Non-acceptance
Do not substitute deterministic routing proxies, fixture execution, instruction-coverage scores, a single model run, or an agent-authored narrative for this evidence.
Promotion rule
1.1.0 must remain a release candidate until this issue is closed with the validated evaluation artifacts linked in the closing comment.
Purpose
Track the only remaining external acceptance gate for promoting
1.1.0-rc.1to stable. Static validation, fixtures, policy checks, property tests, mutation testing, local security review, reproducible builds, and GitHub Actions are necessary but do not prove real model behavior.Required evaluation
Run the committed live behavioral-evaluation harness against a pinned authenticated Codex model using the committed held-out pressure cases.
Minimum execution contract:
Required metrics
Record and publish at least:
Acceptance criteria
Non-acceptance
Do not substitute deterministic routing proxies, fixture execution, instruction-coverage scores, a single model run, or an agent-authored narrative for this evidence.
Promotion rule
1.1.0must remain a release candidate until this issue is closed with the validated evaluation artifacts linked in the closing comment.