Skip to content

Add the codex-agent provider: keyless completions through the Codex CLI - #205

Merged
ethanj merged 1 commit into
mainfrom
feat/codex-agent-provider
Sep 3, 2026
Merged

Add the codex-agent provider: keyless completions through the Codex CLI#205
ethanj merged 1 commit into
mainfrom
feat/codex-agent-provider

Conversation

@ethanj

@ethanj ethanj commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Closes #56.

What this adds

--provider codex-agent: completions run through the locally installed Codex CLI, so the CLI's own login is the only credential. No API key is read or forwarded.

  • Process boundary. Every call is codex exec --ephemeral --sandbox read-only --skip-git-repo-check --ignore-user-config --ignore-rules --color never, prompt on stdin, in a per-call throwaway cwd that is removed on every exit path. The child gets a 14-key environment allowlist and runs in its own process group; termination is SIGTERM then SIGKILL, and it also fires on parent SIGINT, SIGTERM, and exit, so an interrupted llmwiki never orphans a Codex process tree.
  • Executable lookup fails closed. codex is resolved only from absolute PATH entries pointing at regular executable files; when none matches the call fails with an actionable error instead of launching a binary that happens to sit beside the Node runtime.
  • Output handling. The answer is read from --output-last-message with a bounded O_NOFOLLOW read; stdout is drained uncounted; only stderr counts against the 1 MiB cap. Tool calls are validated against the requested tool's input_schema with Ajv (exactly one tool; malformed JSON and schema failures are rejected).
  • Credential hygiene. Child output never enters an error message. Stderr is classified into actionable failures: not installed, not logged in, CLI too old for these flags (0.152.1 or newer), timed out. Selecting codex-agent on argv or in the shell environment skips loading the project .env.
  • Honesty. maxTokens is documented as not forwarded. An unset model passes no --model and reports the codex-cli-default sentinel. Embeddings must name an explicit provider; the startup guard refuses an implicit one before Codex is spawned.
  • CLI. A uniform --provider option on the seven commands that take a provider, each pinned by a --help test, with preservation tests for every existing provider.
  • Docs. providers, environment variables, installation, FAQ, compile reference, README, CHANGELOG.

Verification

  • 51 new vitest cases across six files. The process-boundary tests drive the real built CLI: exact env allowlist, launcher-wrapper injection bypass, PATH fail-closed with a copied Node binary beside a trap codex, .env isolation observed at the readFileSync call, and interrupt custody for SIGINT, repeated SIGINT, and SIGTERM.
  • A live smoke test runs against the installed Codex binary and skips only when codex is absent (ENOENT). Every argv flag was checked against codex exec --help of codex-cli 0.152.1.
  • Timeout and interrupt tests wait on the fake Codex's readiness marker and drive the provider's timers with fake timers; nothing depends on process start-up speed.
  • Full suite on this branch: 677 files passed, 5087 tests passed, 3 skipped.

Known limits

  • The version floor (0.152.1) is enforced by classifying the CLI's rejection of unknown flags, not by a codex --version check.
  • Child diagnostics are withheld from error messages by design; there is no scrubbed debug channel yet.
  • Windows executable lookup is untested.

Closes #56.

`--provider codex-agent` runs every completion as `codex exec --ephemeral --sandbox
read-only --skip-git-repo-check --ignore-user-config --ignore-rules`, so the CLI's own
login (CODEX_HOME) is the only credential and no API key is read. Each call gets a
throwaway cwd and a 14-key environment allowlist; the child runs in its own process
group with SIGTERM-then-SIGKILL termination that also fires on parent SIGINT, SIGTERM,
and exit. The `codex` executable is resolved only from absolute PATH entries and fails
closed when none matches, so a binary beside the Node runtime is never launched. Tool
calls are validated against the requested tool's input_schema with Ajv; the answer is
read from `--output-last-message` with a bounded O_NOFOLLOW read, stdout is drained
uncounted, and only stderr counts against the 1 MiB cap. Child output never enters an
error message; stderr is classified into actionable failures (not installed, not logged
in, CLI too old for these flags, timed out). Selecting codex-agent on argv or in the
shell environment skips loading the project .env. maxTokens is documented as not
forwarded; an unset model passes no --model and reports the codex-cli-default sentinel;
embeddings must name an explicit provider. The --provider flag is uniform across the
seven commands that take a provider.

Tests drive the real built CLI for the process boundary (env allowlist, launcher-wrapper
bypass, PATH fail-closed with a copied Node binary, .env isolation at the read, interrupt
custody) and a gated live smoke test exercises the installed Codex binary. The timeout
tests wait on the fake's readiness marker and drive the provider's timers with fake
timers, so nothing depends on process start-up speed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@ethanj
ethanj merged commit cbd09c6 into main Sep 3, 2026
3 checks passed
@ethanj
ethanj deleted the feat/codex-agent-provider branch September 3, 2026 09:39
@itoqa

itoqa Bot commented Sep 3, 2026

Copy link
Copy Markdown

Ito QA test results
Commit: d8844b1: 19 test cases ran, 19 passed ✅.

Summary

Coverage spans provider selection and overrides, successful compilation and structured output, embedding setup, command isolation, secret handling, safe error messages, and process cleanup. It includes normal flows plus edge and adversarial cases such as invalid configuration, malformed data, unsafe executable locations, interruptions, timeouts, and retries, with no application regressions identified.

Safe to merge — the run found no failures or PR-attributable regressions across provider behavior, compilation, security boundaries, error handling, and cleanup paths. No merge-blocking risk is indicated.

Tests run by Ito

View full run

Result Severity Type Description
Anthropic Anthropic stays selected when no provider is specified. Explicit OpenAI selection, existing provider behavior, credential checks, and non-Codex embedding rules also continue to work.
General A timed-out Codex request was stopped, its child process and temporary files were removed, and a new request completed normally with fresh resources.
General An incompatible local command produced clear upgrade guidance without showing its bearer-token diagnostic.
General Selecting Codex explicitly avoids reading the project's dotenv file, while ordinary startup still loads .env and DOTENV_CONFIG_PATH settings.
General Sending one or repeated interrupt signals stops the active Codex request and removes its temporary work folder.
Codex Missing, incompatible, and unauthenticated Codex setups fail with clear next steps. Child diagnostics and secret-shaped text stay out of the error message, and the command does not silently switch to another provider.
Codex The Codex provider ignored unsafe program locations and used the eligible executable when one was available. When no eligible executable remained, it stopped with installation guidance instead of launching an unsafe file.
Command All seven tested CLI commands accept a provider override. Codex selection works from the environment or command flag, keeps API credentials out of the child process, and requires an explicit embedding provider.
Embedding The compile command clearly asks for an embedding backend when none is set, without starting the Codex work. After an Ollama backend is named, the full compile finishes and writes its output.
Output Text completion returned exactly "compiled page", and buffered streaming returned the same text in one final callback chunk.
Output When the local command fails, the user sees a useful authentication error without any secret values from the command output.
Provider Selecting Codex completes the compile and returns the expected page. With no model override, the provider reports its default and sends no model or token limit to the Codex command.
Rev Quickstart completed successfully with the Codex provider even though the parent environment selected another provider. It generated a wiki page, kept the alternate provider unused, restored the parent setting, and withheld the parent's secret and runtime settings from Codex.
Rev The compile command stopped with clear guidance when the selected embedding backend could not provide embeddings. Codex was not started and no partial wiki was created.
Sandbox The completion returned the expected page text while running with read-only, non-interactive limits in a temporary folder. Unrelated parent variables did not reach the child process, and the folder was removed afterward.
Sandbox Explicit Codex selection kept project .env secrets and unrelated launcher variables out of the child process while preserving the required allowlisted environment.
Structured Structured extraction returned the requested value in valid JSON, and the compiler accepted the result.
Structured Malformed JSON and JSON with the wrong fields were both rejected before the compiler could use them.
Timeout Hanging requests stop within the allowed time, and their temporary working folders are removed.

Tip

Reply with @itoqa to send us feedback on this test run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support of OpenAI Codex OAuth

1 participant