quad-cli gate: add a periodic live four-CLI dogfood check (smoke:quad) #15

Activity

1. drvoss commented on Aug 27, 2026

   

   drvoss

   on Aug 27, 2026

   OwnerAuthor

   More actions

   Update — 2026-08-27

   PR #16 merged (dac7dae) after independent cross-verification by three models (Claude Opus, Claude Sonnet, and local Codex CLI, all reached MERGE independently). This closes the npm run smoke:quad / smoke:quad:available item and the reviewer-drop-out observability item.

   Still open from this issue's remaining scope (per PR #16's own "Remaining scope" note — it explicitly does not close #15):

   • Transport-integrity nonce in the production protocol (needs a design decision first).
   • Revisit --min-reviewers (currently 2) and the supported-reviewer contract now that four reviewers actually run.

   New non-blocking findings from the cross-model review of #16, worth folding into this work:

   1. (Codex, P2) observed_failure in the reviewer-drop diagnostics persists the raw failure string, which can leak an internal filesystem path (e.g. Mock response not found: C:\Users\...\agy.json) even though resolved_path is redacted separately — scripts/quad-cli-orchestrate.mjs:1019.
   2. (Sonnet) reviewers/environment diagnostic fields are only populated when --artifact is passed, but orchestration/skills/quad-cli-consensus-gate.md describes them as carried "when relevant" alongside always-on fields like status — doc/behavior mismatch.
   3. (Sonnet) When --artifact is used, buildEnvironment re-probes CLI versions for all declared tools serially (up to 4x 10s-capped spawns), outside the --gate-timeout budget and undocumented.

   Keeping this issue open to track the nonce, the --min-reviewers revisit, and the three findings above.
2. drvoss

   added a commit that references this issue on Aug 27, 2026

   fix(quad-cli): add transport-integrity nonce, redact leaked paths, fi…

   ...

   This issue will close once commit 6cc8e00 is merged into the 'main' branch.

   6cc8e00
3. drvoss

   closed this as completedin #17on Aug 27, 2026
4. drvoss commented on Aug 27, 2026

   

   drvoss

   on Aug 27, 2026

   OwnerAuthor

   More actions

   Closed by #17 — 2026-08-27

   PR #17 merged (squash), implementing this issue's remaining scope:

   • Transport-integrity nonce: implemented. Per-run random token, echoed back by reviewers as top-level transport_nonce, placed after the <diff> block in the prompt (not before) so tail truncation of the diff — the realistic failure mode — cannot produce a false confirmed. States: confirmed / mismatch (dropped from consensus) / not-echoed (not penalized, ambiguous by design) / unavailable. Recorded in reviewers.nonce_status under --artifact.
   • --min-reviewers revisit: confirmed already resolved by PR feat(quad-cli): record declared vs effective reviewers and add a live smoke preflight #16's documented decision (stays at 2, with recorded rationale in orchestration/skills/quad-cli-consensus-gate.md) — no code change needed.
   • The 3 non-blocking findings from feat(quad-cli): record declared vs effective reviewers and add a live smoke preflight #16's cross-review are fixed: path-leak in observed_failure redacted, reviewers/environment artifact-gating documented, --artifact CLI-reprobe cost documented.

   Cross-verified by Claude Opus, Claude Sonnet, and (partially, before hitting a usage limit) the local Codex CLI across two review rounds. Sonnet's first-round review caught a real design gap (nonce placed before the diff, so tail truncation wouldn't be detected) — fixed by moving the nonce instruction after </diff>; both models confirmed MERGE on the corrected version.

   npm test: 1116/1116 passing. npm run validate / npm run lint:md: passing.

   Closing.