Skip to content

fix(CrossVendorAudit): raise codex timeout to 600s and pin reasoning_effort=medium#1383

Open
ryan-baum wants to merge 1 commit into
danielmiessler:mainfrom
ryan-baum:fix/crossvendoraudit-codex-timeout
Open

fix(CrossVendorAudit): raise codex timeout to 600s and pin reasoning_effort=medium#1383
ryan-baum wants to merge 1 commit into
danielmiessler:mainfrom
ryan-baum:fix/crossvendoraudit-codex-timeout

Conversation

@ryan-baum

Copy link
Copy Markdown

Problem. Cato's cross-vendor audit (Rule 2a, E4/E5) silently returns verdict: "skipped" on real ISAs. CrossVendorAudit.ts caps the codex call at CODEX_TIMEOUT_MS = 120_000 (2 min), but codex exec --model gpt-5.4 is a reasoning model: on a real ISA + artifact bundle it routinely needs ~3 min. The 120s cap fires first → SIGTERM[TIMEOUT after 120s] → the audit is skipped before GPT-5.4 ever returns a verdict. The cross-vendor gate looks like it ran but produced nothing.

Two compounding factors:

  1. The cap is too tight for gpt-5.4's reasoning latency on a non-trivial bundle.
  2. No reasoning_effort is pinned, so the call inherits ~/.codex/config.toml's default. If that default is high/xhigh, the call blows past even a generous timeout. Empirically, medium returns a full audit in ~3 min; xhigh exceeds 600s on a real bundle.

Fix. Raise the in-process cap to 600s and pin model_reasoning_effort=medium so latency is bounded by an explicit, known-good effort tier rather than an inherited config default.

-const CODEX_TIMEOUT_MS = 120_000;
+const CODEX_TIMEOUT_MS = 600_000;
-      ["exec", "--sandbox", "read-only", "--model", "gpt-5.4", "-"],
+      ["exec", "-c", "model_reasoning_effort=medium", "--sandbox", "read-only", "--model", "gpt-5.4", "-"],
-      resolvePromise({ stdout, stderr: stderr + "\n[TIMEOUT after 120s]", code: 124 });
+      resolvePromise({ stdout, stderr: stderr + "\n[TIMEOUT after 600s]", code: 124 });

Why medium and not just a bigger timeout. A bigger timeout alone doesn't help if the inherited effort default is xhigh (it can exceed 600s on a real ISA). Pinning medium makes the audit's latency predictable and keeps it inside the cap. This is the load-bearing half of the fix.

Scope / non-overlap. Narrowly scoped to the timeout constant, the spawn args, and the timeout message string. Does not touch the --skip-git-repo-check / OPENAI_API_KEY env-inheritance work already in flight in #1346 — these are independent and should merge cleanly alongside it.

Testing. bun CrossVendorAudit.ts --doctor still passes; a full E4/E5 audit that previously skipped at 120s now returns a parsed verdict in ~3 min.


…effort=medium

The cross-vendor audit (Cato, Rule 2a) silently returns verdict:skipped on
real ISAs: codex exec --model gpt-5.4 is a reasoning model needing ~3min on a
real bundle, but CODEX_TIMEOUT_MS caps at 120s, so SIGTERM fires before a
verdict returns. Also pin model_reasoning_effort=medium so latency is bounded
by an explicit known-good tier rather than an inherited config default
(high/xhigh can exceed even 600s). Narrowly scoped; independent of danielmiessler#1346.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant