Skip to content

Commit b159072

Browse files
authored
Merge pull request #23 from iiwish/codex/p9-3-stable-surface-gate
[codex] add stable surface regression gate
2 parents 5c7370f + 3ffff0e commit b159072

10 files changed

Lines changed: 613 additions & 3 deletions

README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -245,6 +245,7 @@ Use a local checkout:
245245
```bash
246246
npm ci
247247
npm run build
248+
npm run stable:surface:check
248249
node ./dist/cli/main.js --help
249250
npm run daemon:verify
250251
npm run dogfood

README.zh-CN.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -243,6 +243,7 @@ npx --package agent-cli-runtime agent-runtime conformance --mode fixtures --json
243243
```bash
244244
npm ci
245245
npm run build
246+
npm run stable:surface:check
246247
node ./dist/cli/main.js --help
247248
npm run daemon:verify
248249
npm run dogfood

docs/api-schema-contract.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,7 @@ Internal but packaged files under `dist/**` may exist in the npm tarball because
6161
| `agent-cli-runtime.releaseArtifactNormalization.v1` | `schemaVersion`, `ok`, `downloadDir`, `outDir`, `artifacts`, `diagnostics` | `ok`, `diagnostics[].code`, `artifacts[].artifactName` | No absolute local paths; repo-external input/output directories are represented as `<external_artifact_dir>` / `<external_output_dir>`, and only expected release-candidate artifact filenames are copied. |
6262
| `agent-cli-runtime.mainReleaseCandidateEvidence.v1` | `schemaVersion`, `stage`, `evidenceKind`, `checkedAt`, `releaseTargetSha`, `targetRef`, `mainEvidence`, `branchEvidence`, `historicalMainEvidence`, `compatibilityVerification`, `localReleaseCandidate`, `remoteReleaseCandidate`, `downloadedArtifacts`, `boundary` | `mainEvidence`, `branchEvidence`, `historicalMainEvidence[].historical`, `historicalMainEvidence[].currentMainFreshEvidence`, `compatibilityVerification.ok`, `remoteReleaseCandidate.run.status`, `remoteReleaseCandidate.run.conclusion`, `remoteReleaseCandidate.artifacts.valid`, `downloadedArtifacts.verified` | Repo-only current main summary; historical P8 summaries are listed as historical and non-current; no raw stdout/stderr, workflow logs, prompts, private paths, token-looking values, Bearer values, auth env assignments, tarball shasums, or pack shasums. |
6363
| `agent-cli-runtime.packageContentEquivalence.v1` | `schemaVersion`, `ok`, `baseRef`, `headRef`, `packageName`, `packageVersion`, `packageContentEqual`, `basePackageDigest`, `headPackageDigest`, `baseFileCount`, `headFileCount`, `changedPackageFiles`, `evidenceOnlyDrift`, `freshReleaseCandidateRequired`, `diagnostics` | `ok`, `packageContentEqual`, `evidenceOnlyDrift`, `freshReleaseCandidateRequired`, `diagnostics[].code`, `changedPackageFiles[].status` | Diagnostics and ref labels are redacted; no temp worktree path, private path, token-looking value, Bearer value, auth env assignment, raw stdout/stderr, workflow log, prompt, tarball shasum, or pack shasum. |
64+
| `agent-cli-runtime.stableSurfaceCheck.v1` | `schemaVersion`, `ok`, `packageRoot`, `publicTypes`, `schemaInventory`, `cliVocabularies`, `packagedDocs`, `diagnostics`, `boundary` | `ok`, `packageRoot.ok`, `publicTypes.ok`, `schemaInventory.ok`, `cliVocabularies.ok`, `packagedDocs.ok`, `diagnostics[].code`, `boundary.repoOnlyGate`, `boundary.stableSurfaceCheckIsRuntimePublicApi` | Repo-only stable surface gate summary; no raw stdout/stderr, workflow log, prompt, private path, token-looking value, Bearer value, or auth env assignment. |
6465

6566
## Failure Taxonomy
6667

@@ -117,3 +118,5 @@ Downloaded release-candidate artifacts are normalized with `npm run release:arti
117118
Main release-candidate evidence uses `agent-cli-runtime.mainReleaseCandidateEvidence.v1` and is generated with `npm run release:main-candidate:evidence -- --stage <stage> --release-target-sha <origin-main-sha> --local-release-dir <local-strict-dir> --remote-run-json <run.json> --artifacts-json <artifacts.json> --downloaded-dir <normalized-artifact-dir> --out .release-evidence/<stage-lower>-main-release-candidate.json`. The stage label accepts current phase labels such as `P9-2` and is not tied to P8. The summary binds `releaseTargetSha` to `origin/main`, records local strict compatibility and release verification, records a fresh `release-candidate.yml --ref main` run with matching `headSha`, and lists P8 main release-candidate files only under `historicalMainEvidence` with `historical: true` and `currentMainFreshEvidence: false`.
118119

119120
Package content equivalence uses `agent-cli-runtime.packageContentEquivalence.v1` and is run with `npm run release:package-content:verify -- --base-ref <release-target-sha> --head-ref <sha-or-ref>`. It compares npm package-visible files from temporary git worktrees by package-relative file list, mode, size, and file-content hashes. It does not treat gzip/tarball bytes as the only equivalence signal. Package-external evidence, tests, and repo-only scripts can produce `evidenceOnlyDrift: true` with `freshReleaseCandidateRequired: false`; package-visible README/docs/package.json/dist/types/bin/examples changes produce package content drift and require fresh release-candidate evidence before the head ref is treated as a release target.
121+
122+
Stable surface regression checking uses `agent-cli-runtime.stableSurfaceCheck.v1` and is run with `npm run stable:surface:check` after `npm run build`. It verifies the package-root runtime value export, root declaration boundary, schema inventory documentation, terminal/classification vocabularies, experimental adapter-surface classification, and npm package exclusions for repo-only paths and scripts. This is a beta-candidate repository/release gate, not a runtime public API schema for package consumers.

docs/release-checklist.md

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,8 @@ Run these before treating a future alpha version as a local release candidate:
2424
npm test
2525
npm run typecheck
2626
npm run lint
27+
npm run build
28+
npm run stable:surface:check
2729
npm run package:check
2830
npm run package:docs:check
2931
npm run compat:real:evidence:verify
@@ -41,6 +43,8 @@ Acceptance:
4143
- [x] `npm test` passes.
4244
- [x] `npm run typecheck` passes.
4345
- [x] `npm run lint` passes.
46+
- [x] `npm run build` passes before build-output gates.
47+
- [x] `npm run stable:surface:check` passes with `schemaVersion: "agent-cli-runtime.stableSurfaceCheck.v1"` and keeps package-root value exports limited to `createAgentRuntime`.
4448
- [x] `npm run package:check` passes and rejects `.release-evidence/` plus `.reference/` if they appear in pack metadata.
4549
- [x] `npm run package:docs:check` unpacks the local tarball and rejects stale publish-state claims for this version, dry-run stop wording, publish-ready candidate wording, and old current dist-tag claims.
4650
- [x] `npm run compat:real:evidence:verify` passes without launching authenticated real agent runs.
@@ -111,7 +115,7 @@ The package must not contain:
111115
- The package root value export remains `createAgentRuntime`.
112116
- The schema inventory and versioning policy live in [docs/api-schema-contract.md](./api-schema-contract.md).
113117
- The daemon/product shell embedding contract lives in [docs/daemon-ready-contract.md](./daemon-ready-contract.md).
114-
- `agent-cli-runtime.releaseVerification.v1`, `agent-cli-runtime.releaseGateEvidence.v1`, `agent-cli-runtime.mainReleaseCandidateEvidence.v1`, and `agent-cli-runtime.packagedDocsVerification.v1` are release artifact schemas.
118+
- `agent-cli-runtime.releaseVerification.v1`, `agent-cli-runtime.releaseGateEvidence.v1`, `agent-cli-runtime.mainReleaseCandidateEvidence.v1`, `agent-cli-runtime.packagedDocsVerification.v1`, and `agent-cli-runtime.stableSurfaceCheck.v1` are release or repository gate schemas.
115119

116120
## Schema Vocabulary
117121

docs/release-report.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -96,6 +96,8 @@ Main-scoped remote release-candidate closure uses `npm run release:main-candidat
9696

9797
Package-content drift review uses `npm run release:package-content:verify -- --base-ref <release-target-sha> --head-ref <sha-or-ref>`. The verifier emits `schemaVersion: "agent-cli-runtime.packageContentEquivalence.v1"` and compares the npm package file list plus file-content hashes for both refs from temporary git worktrees. `.release-evidence/`, tests, and repo-only scripts can change without changing package content; README, README.zh-CN, packaged docs, package.json, dist, type declarations, bin files, examples, and other package-visible files trigger `freshReleaseCandidateRequired: true` when their package content differs. The P8-8 evidence file is `.release-evidence/p8-8-package-content-equivalence.json`. It is a package-content decision, not a replacement for fresh main release-candidate workflow evidence.
9898

99+
Stable surface regression review uses `npm run stable:surface:check` after `npm run build`. The verifier emits `schemaVersion: "agent-cli-runtime.stableSurfaceCheck.v1"` and checks the package-root runtime value export, root declarations, schema inventory docs, terminal/classification vocabularies, experimental adapter-surface classification, and repo-only package exclusions. It is a local repository/release gate; it does not publish npm, create a GitHub Release, launch authenticated real agent runs, or make `dist/**` subpaths public API.
100+
99101
## Package Boundary
100102

101103
The npm package may include stable docs, examples, `dist/`, and the runtime entrypoints. It must not include:
@@ -123,6 +125,7 @@ The API and CLI schema inventory, versioning policy, root export boundary, and f
123125
- `agent-cli-runtime.releaseArtifactNormalization.v1`
124126
- `agent-cli-runtime.mainReleaseCandidateEvidence.v1`
125127
- `agent-cli-runtime.packageContentEquivalence.v1`
128+
- `agent-cli-runtime.stableSurfaceCheck.v1`
126129
- `agent-cli-runtime.realCompatibilityEvidenceVerification.v1`
127130
- `agent-cli-runtime.realCompatibilityMatrix.v1`
128131
- `agent-cli-runtime.realCompatibilityEvidence.v1`

docs/stable-readiness.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,7 @@ The project remains pre-alpha / developer preview. The labels below describe rea
3131
| Built-in adapter values, parser helpers, executable resolution, stores, schedulers, task-graph helpers, and storage modules | `internal` | These are implementation details and must not be documented as public API. |
3232
| Daemon embedding gate and runtime safety gate | `beta-candidate` | `daemon:verify` and `runtime:safety` prove installed-tarball local-kernel behavior with fake CLIs. They do not create a hosted daemon contract. |
3333
| Release verification, published verification, package-content, packaged-docs, and compatibility evidence schemas | `beta-candidate` | These are repository/release gate contracts. They are not runtime public API for package consumers. |
34+
| Stable surface regression gate | `beta-candidate` | `npm run stable:surface:check` emits `agent-cli-runtime.stableSurfaceCheck.v1` and verifies package-root exports, public declarations, schema inventory docs, frozen CLI vocabularies, experimental adapter-surface classification, and repo-only package exclusions. It is a repository/release gate, not runtime public API. |
3435
| Repo-only scripts for release evidence, real compatibility evidence, published verification, package-content equivalence, artifact normalization, and package checks | `internal` | Script outputs can be versioned, but the scripts themselves are repository workflow surfaces, not package-root public contract. |
3536
| `.release-evidence/`, downloaded verification material, raw workflow logs, and local machine observations | `internal` | These are evidence inputs or summaries outside the npm package boundary. |
3637

@@ -62,13 +63,15 @@ This list is synchronized with `src/core/schema-contract.ts`.
6263
| `agent-cli-runtime.releaseArtifactNormalization.v1` | `beta-candidate` |
6364
| `agent-cli-runtime.mainReleaseCandidateEvidence.v1` | `beta-candidate` |
6465
| `agent-cli-runtime.packageContentEquivalence.v1` | `beta-candidate` |
66+
| `agent-cli-runtime.stableSurfaceCheck.v1` | `beta-candidate` |
6567

6668
## Stable Gaps
6769

6870
- `getAdapter` and `RuntimeOptions.adapters` remain `experimental`; adapter authoring needs real external consumer dogfood before any stable promise.
6971
- Internal `dist/**` subpath imports remain unsupported, even when those files appear in the tarball.
7072
- CLI JSON schemas have a versioning policy, but command removal or flag semantic changes still need an explicit pre-alpha breaking-change note.
7173
- Release evidence schemas are repository and release-gate contracts; they do not become runtime public API for package consumers.
74+
- `agent-cli-runtime.stableSurfaceCheck.v1` is a repository/release gate schema; it does not become runtime public API for package consumers.
7275
- Codex `session` and `authProbe` remain in `needsVerification`.
7376
- Claude Code `session.id` and `reasoning` remain in `needsVerification`.
7477
- OpenCode `extraAllowedDirs`, `session`, and `permissionPolicy.read-only` remain in `needsVerification`.

package.json

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -47,14 +47,14 @@
4747
],
4848
"scripts": {
4949
"build": "tsc -p tsconfig.json",
50-
"ci": "npm run typecheck && npm run lint && npm test && npm run build && npm audit --omit=dev && npm run package:check && npm pack --dry-run",
50+
"ci": "npm run typecheck && npm run lint && npm test && npm run build && npm run stable:surface:check && npm audit --omit=dev && npm run package:check && npm pack --dry-run",
5151
"compat:real:evidence": "node ./scripts/create-real-compatibility-evidence.mjs",
5252
"compat:real:evidence:verify": "node ./scripts/verify-real-compatibility-evidence.mjs",
5353
"daemon:verify": "node ./scripts/verify-daemon-ready.mjs",
5454
"dogfood": "node ./scripts/dogfood.mjs",
5555
"package:check": "node ./scripts/check-package-boundary.mjs && node ./scripts/check-packaged-docs.mjs",
5656
"package:docs:check": "node ./scripts/check-packaged-docs.mjs",
57-
"prepublish:check": "npm run typecheck && npm run lint && npm test && npm run build && npm run daemon:verify && npm run runtime:safety && npm run compat:real:evidence:verify && npm run dogfood && npm audit --omit=dev && npm run package:check && npm pack --dry-run",
57+
"prepublish:check": "npm run typecheck && npm run lint && npm test && npm run build && npm run stable:surface:check && npm run daemon:verify && npm run runtime:safety && npm run compat:real:evidence:verify && npm run dogfood && npm audit --omit=dev && npm run package:check && npm pack --dry-run",
5858
"prepack": "npm run build",
5959
"published:adapters:verify": "node ./scripts/verify-published-adapters.mjs",
6060
"published:daemon:verify": "node ./scripts/verify-published-daemon-consumer.mjs",
@@ -70,6 +70,7 @@
7070
"release:verify": "node ./scripts/verify-release-artifacts.mjs",
7171
"runtime:safety": "node ./scripts/verify-runtime-safety.mjs",
7272
"smoke:published": "node ./scripts/smoke-published.mjs",
73+
"stable:surface:check": "node ./scripts/check-stable-surface.mjs",
7374
"typecheck": "tsc -p tsconfig.json --noEmit",
7475
"lint": "tsc -p tsconfig.json --noEmit",
7576
"test": "npm run build && vitest run --reporter=verbose --no-file-parallelism --testTimeout 30000"

0 commit comments

Comments
 (0)