Skip to content

feat(tools): make release qualification gates explicit and fail closed (#1216) - #1257

Merged
SisyphusZheng merged 1 commit into
devfrom
v044/a10.8-gate-verdicts
Sep 2, 2026
Merged

feat(tools): make release qualification gates explicit and fail closed (#1216)#1257
SisyphusZheng merged 1 commit into
devfrom
v044/a10.8-gate-verdicts

Conversation

@SisyphusZheng

Copy link
Copy Markdown
Member

Problem

Issue #1216 (umbrella #1155, Alpha.10, audit H6). tools/consumer-smoke.ts collapsed confirmed-absence / DNS failure / timeout / 5xx / malformed responses into one boolean (catch { return false }), then printed "not yet available; skipping" and exited 0 — wired into the release plan post-publish, so registry/CDN infra failure silently greened a release gate. No canonical PASS/FAIL/SKIP/UNKNOWN verdict type existed.

Owner

New canonical verdict contract tools/gate-verdict.ts (small module under tools/, no package, no public API): only PASS admits a release; SKIP_ALLOWED requires explicit policy; UNKNOWN/INFRA failure fails closed. Autoflow's runner layer stays boolean-justified (subprocess exit-code mapping is already fail-closed; spawn failure throws).

Evidence

  • Live RED at base SHA: deno run -A tools/consumer-smoke.ts --version 0.0.0-a10-8-red-probe printed "skipping" and exited 0. After: prints FAIL: confirmed absence… and exits 1.
  • 18 hostile decision-logic tests (injected fetchers): registry 200+version-confirming payload is the only PASS; 404 FAIL; 5xx/redirect/malformed/version-mismatch UNKNOWN; DNS/timeout UNKNOWN; CDN 404/empty FAIL, 5xx/network UNKNOWN; two aggregate sweeps prove no hostile input maps to PASS/SKIP. RED at base SHA (TS2305 ×5 — the contract did not exist), GREEN after.
  • Gates: tools/ suite 448 passed; fmt/lint/typecheck/release:state-machine/workflow:check/docs:truth all exit 0.
  • Fresh kimi-code/k3-256k HIGH implementer session; orchestrator re-ran the hostile tests independently.

Scope / non-goals

CLI contract (flags, exit codes) unchanged → release plan wiring and published-consumers workflow untouched. M15 breadth (other gate scripts' decision tests) deferred to B2.8/#1230 by design. No gate weakened.

Risk

Release runs now correctly fail on registry/CDN infra errors (explicit retries expected); pre-publish local iteration uses --local. Rollback: revert.

#1216)

Alpha.10 truth closure (H6): consumer-smoke collapsed confirmed-absence,
DNS failure, timeout, 5xx and malformed responses into one boolean via
catch→false, then printed 'skipping' and exited 0 — registry/CDN infra
failure silently greened a release gate (proven live at base SHA:
probe against an unpublished version exited 0).

- new canonical verdict contract tools/gate-verdict.ts:
  PASS / FAIL / SKIP_ALLOWED / UNKNOWN; only PASS admits a release by
  default; SKIP_ALLOWED only under explicit policy; UNKNOWN fails closed
- consumer-smoke probes refactored to pure injectable decision
  functions: registry 200 with version-confirming payload is the only
  PASS; 404 is FAIL; 5xx/redirect/malformed/timeout/DNS are UNKNOWN;
  CDN 404/empty is FAIL, CDN 5xx/network is UNKNOWN; every non-PASS
  exits 1
- 18 hostile decision-logic tests via injected fetchers; live probe now
  prints FAIL and exits 1 on confirmed absence

M15 breadth (other scripts' decision tests) is B2.8/#1230 scope.
Implemented by fresh kimi-code/k3-256k HIGH session
(.agents/v044-kimi-implementer.md), packet A10.8. No publish, no tag.
@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

APIError: Insufficient Balance

opencode session  |  github run

@SisyphusZheng
SisyphusZheng merged commit 0662b1d into dev Sep 2, 2026
8 of 9 checks passed
@SisyphusZheng
SisyphusZheng deleted the v044/a10.8-gate-verdicts branch September 3, 2026 09:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant