[Dream Cycle 2026-09-06] flywheel-promotion: sequential.ts's anytime-valid gate was unreachable from run.ts/replay.ts + evals-verticals,bench scan

[ruvnet]

Tonight's Rotation

DEEP=flywheel-promotion, SCAN=evals-verticals,bench (slot 1, 20260906 % 5 == 1). No bonus deep-dive
(20260906 % 25 == 6, % 75 == 31). Session commit (parent): d5833dc6512ac1adeeef91a331c29055cd8a4dbb.
Branch: dream/2026-09-06-flywheel-promotion.

Ledger Check

Inspected docs/dream-cycle/LEDGER.md (14+ rows) plus direct GitHub state (0 open PRs at session start —
confirmed via list_pull_requests, not inferred from a sparse ledger). Prior flywheel-promotion nights
(2026-08-16 #205, 2026-08-26 #231, 2026-09-01 #258) all hardened verifyReplayBundle/gateReExecutes
against receipt-splicing, anchor-bypass, and chain-forgery — all three MERGED. That surface is now heavily
defended; per STEP 1.1 ("substantially the same finding in ≥3 prior runs → reject the duplicate
direction"), I deliberately did NOT touch replay.ts's existing hardened checks again and instead reviewed
the rest of the flywheel package for a genuinely different gap — found in sequential.ts (untouched by
all three prior nights).

Prior gist score (STEP 1.2, 2026-09-05): 10/10 — grade-A evidence, 5 competitor rows, 3 specific
recommendations, valid witness, 1497/1500 words, genuinely novel finding.

Learning signals applied: 0 open PRs at session start (no review-bottleneck bias needed tonight); no
LLM_EVAL=blocked streak (OPENROUTER_API_KEY present, though unused — this candidate needs no model
calls).

Deep Dive Findings

packages/flywheel/src/sequential.ts implements withSequentialEvidence() — an anytime-valid e-process
gate (Ville's inequality) meant to stop a thin/noisy promotion from surviving repeated peeking at the same
holdout (cited justification: naive greedy acceptance has a published 30-42% false-commit rate). It was
fully built, exported from the public API, and unit-tested (14/14 passing) — but had zero production call
site. runFlywheelGenerations never populated PromotionEvidence.pairedOutcomes (the type didn't even
have the field), so a caller supplying promotionRule: withSequentialEvidence(meetsPromotionRule) on a
real FlywheelConfig always silently degraded to the plain frozen gate. An independent critic
(fresh-subagent adversarial review, not self-review) found the SAME bug class at a sibling call site:
verifyReplayBundle's ADR-235 gate re-execution (replay.ts) had the identical gap — closed in the same
diff rather than deferred. Same bug class as ADR-278 (2026-09-02, Darwin Mode's Tier-2 sandbox never
calling inspectVariant), different domain.

Hypothesis

Given a FlywheelConfig whose Evaluator reports per-item itemWins for both baseline and candidate on the
same holdout, when promotionRule: withSequentialEvidence(meetsPromotionRule) is supplied to
runFlywheelGenerations, then a thin-margin win (few discordant items, e-value below threshold) should be
REJECTED even though the frozen base gate alone would promote it, and a strong-margin win should be
PROMOTED identically — subject to: zero behavioral change for any Evaluator omitting itemWins;
meetsPromotionRule itself untouched; the same guarantee holding under verifyReplayBundle's independent
replay. Frozen before implementation; not modified after evaluation began.

Evaluation Receipt

evaluated: accepted

8 new tests (packages/flywheel/__tests__/sequential-wiring.test.ts), all confirmed non-vacuous by
reverting each of the two sub-fixes (run.ts, replay.ts) independently via git stash and re-running:

• THIN evidence (3/20 discordant, e=1.5³=3.375<20): frozen gate alone promotes through a REAL
  runFlywheelGenerations call; sequential-wrapped does not. Pre-fix: both promote (test fails as
  predicted — 1 promotion observed where 0 expected).
• STRONG evidence (20/20 discordant, e=1.5²⁰≫20): both promote.
• Backward-compat: an Evaluator omitting itemWins degrades to the base rule through the real run path.
• Same THIN/STRONG pair re-verified through verifyReplayBundle's gate re-execution — pre-fix (replay.ts
  alone reverted), THIN sealed evidence incorrectly re-passes (gateReExecutes: true); post-fix, correctly
  fails.

Full suite: flywheel 77/77 (was 72/72, +5 net after the pre-existing PairedOutcome relocation), tsc --noEmit clean. Downstream sweep, 0 regressions: evals-math 14/14, evals-sql 7/7, evals-toolcall 14/14,
evals-servedmodel 12/12, evals-hle 7/7, evals-extract 7/7, autogenous 7/7. Build: JS/TS only in this
sandbox (npm ci && npm run build clean for the affected packages; no wasm/NAPI steps required for this
candidate).

Darwin Results

Not applicable — tonight's candidate is a structural wiring fix to the flywheel's own infrastructure code,
not a tunable policy genome. No Darwin generations run (consistent with all three prior flywheel-promotion
nights, none of which ran Darwin either).

Flywheel Evidence

No flywheel receipt/lineage produced tonight — the candidate IS flywheel infrastructure, not a scored
policy run through it. Evidence classification: this finding is a DECISION (a real, reproducible defect

• fix), not an INFERENCE — confirmed by direct code execution (reverting each sub-fix), not by static
  reading alone.

Reward Hack Check

No test/benchmark weakened; meetsPromotionRule (the frozen default gate) is byte-for-byte unchanged
(confirmed: gateFingerprint(meetsPromotionRule) unaffected — no test asserts this explicitly tonight
since the function body wasn't touched, verified by diff). No new caching, no live model spend, fully
deterministic. Both new optional fields (Score.itemWins, PromotionEvidence.pairedOutcomes) are
additive; nothing was deleted or narrowed to make a test pass. Independent critic explicitly checked for
cherry-picked/tautological tests and hand-verified the THIN-evidence math (1.5³ vs. threshold 20)
independently.

Security Review

Not a security-sensitive surface change — no new external input parsing, no new trust boundary. The
existing receipt/signature trust model (receipts.ts) is untouched; pairedOutcomesFromItemWins operates
only on already-sealed/already-trusted Score fields at both call sites.

Scan Findings (evals-verticals, bench)

• evals-verticals: every evals-{math,sql,toolcall,extract,hle,servedmodel}/src/gate.ts composite gate
  calls meetsPromotionRule directly (never withSequentialEvidence), and none of their Evaluators set
  itemWins — so tonight's fix does not, by itself, give any vertical sequential-evidence coverage.
  Disclosed, not fixed (Recommended Next Steps feat(web-ui): browser-based agent harness generator + Claude skill/agent/command authoring #1 below) — adopting it per-vertical is a product decision
  about each domain's holdout size/noise floor, not a mechanical wiring fix, and doing it for all 6
  verticals in one night would roughly double this diff for an unvalidated effect on real (non-synthetic)
  domains.
• bench: packages/bench/src/draco/* (routing, self-consistency, cost-efficiency, grounding-gate) is a
  separate benchmark harness with its own scoring, unrelated to the flywheel promotion gate — reviewed,
  no flywheel-promotion-relevant finding surfaced tonight.

Competitors Reviewed

LangGraph, AutoGen, CrewAI, DSPy/GEPA (ICLR 2026 oral), OpenAI Agents SDK/Codex — none found to ship an
anti-multiple-testing / sequential-evidence gate for their own optimization or promotion loop as of this
survey. Primary source cross-checked: arXiv:2512.03109 ("E-valuator: Reliable Agent Verifiers with
Sequential Hypothesis Testing", submitted 2025-12-02, revised 2026-05-28) — an active academic approach to
a closely related problem (per-step trajectory verification, not per-generation promotion), confirming this
is a genuinely open area in production agent-harness tooling rather than something a competitor already
solved.

Gist

No gist-hosting tool is available in this session (GitHub issues/PRs work via MCP; gist creation does not).
Published locally instead: docs/dream-cycle/2026-09-06-gist.md (committed on this branch).

Witness

session_commit : d5833dc6512ac1adeeef91a331c29055cd8a4dbb
report_sha256   : 89298e5e8bf3fd853d1219fa9eba6e8cb4089cf3ce519330da111f6390c46d84
witness         : f88888a48df67ad3d317fa441b80d69e344d8d0e9662afc9617551c323e8291a

Recommendation

evaluated: accepted — human review requested. ADR-279 filed. See PR for the diff; do not self-merge, do
not autonomously promote any Flywheel state — this PR only makes withSequentialEvidence reachable, it
does not enable it anywhere by default.

1. Pilot itemWins + withSequentialEvidence on one real (non-synthetic) evals-* vertical — start with
   evals-math (smallest holdout) — before recommending repo-wide adoption.
2. Decide alpha/lambda defaults per vertical as a statistics/product call — current 0.05/0.5 are
   untuned placeholders inherited from the original unit tests.
3. Re-run metaharness flywheel replay against the 7 real committed ReplayBundles referenced in the
   2026-08-16/08-26/09-01 ledger rows to directly confirm this change is a strict no-op for all of them.