Skip to content

fix: cap max_tokens on scan requests, and read the API key from the env - #31

Merged
higagan merged 1 commit into
fix/urlallowlist-nestedfrom
feat/scan-env-and-max-tokens
Jul 30, 2026
Merged

fix: cap max_tokens on scan requests, and read the API key from the env#31
higagan merged 1 commit into
fix/urlallowlist-nestedfrom
feat/scan-env-and-max-tokens

Conversation

@higagan

@higagan higagan commented Jul 30, 2026

Copy link
Copy Markdown
Owner

Stacked on #30 β€” merge that first and this diff shrinks to just the two changes below. Version β†’ 0.3.5.

Both problems were found by actually running modelfuzz scan against hosted models through OpenRouter for the first time. Neither is theoretical.

1. scan was unusable against hosted models

_probe and _mutate never set max_tokens, so a gateway reserves the target model's full context window up front. Against anthropic/claude-sonnet-5, every single request failed:

402 - This request requires more credits, or fewer max_tokens.
      You requested up to 65536 tokens, but can only afford 3937.

Proven to be the cause, not a credit problem β€” same model, same prompt, same key:

WITHOUT max_tokens (what shipped):  402 error
WITH max_tokens=1024:               works
  VERDICT: SAFE (refused)

This made hosted scanning fail outright on credit-limited accounts β€” which is exactly what a first-time user has. Now capped at 1024 by default, overridable with --max-tokens. A probe needs room for one tool call and a mutation for one prompt, so 1024 is generous.

2. The API key could only reach the CLI on the command line

--api-key was a plain typer.Option with no envvar, so a real key had to be typed as an argument β€” landing in shell history and visible in ps for the duration of the run. It now reads MODELFUZZ_API_KEY, and Typer documents it in --help automatically:

--api-key  <str>  API key for the endpoint. Read from MODELFUZZ_API_KEY when
                  not passed, which keeps the key out of your shell history
                  and process list.

Verified end to end

Real scans against OpenRouter, no key on any command line:

Model Result
openai/gpt-4o-mini 3/3 seeds broke through at generation 1
anthropic/claude-sonnet-5 0/3 β€” refused through generation 3
$ history | grep -c 'sk-or-v1'
0

Tests

71 β†’ 76. StubClient now records max_tokens per request, so both call sites are asserted rather than just the probe. New coverage: env var is read, an explicit --api-key still wins over it, the dummy-key fallback still works for local models, and --max-tokens is honoured.

One trap worth noting for future tests: the new cases originally used the default non-empty mutation, which keeps every lineage alive until the wall-clock budget expires β€” four tests Γ— 30s hung the suite. They use mutation="" to end each lineage after one mutate call, which still exercises both call sites. Suite is back to 0.03s.

Found but NOT fixed here

Running this surfaced a separate, more serious problem: against an aligned model the mutation loop degenerates. The mutation step asks the target to rewrite the attack; Claude refuses, and its refusal text becomes the next generation's "attack prompt". So generations 2 and 3 probe with refusal prose, not attacks. The run reported 7 attack attempts when only 3 were genuine.

That needs a design decision (detect refusals and kill the lineage, or use a separate model as the mutation engine), so it is recorded in the local audit backlog rather than bolted on here.

Two problems found while running `modelfuzz scan` against hosted models
through OpenRouter for the first time.

`scan` never set max_tokens, so a gateway reserves the target model's full
context window up front. Against anthropic/claude-sonnet-5 every request
failed:

    402 - This request requires more credits, or fewer max_tokens.
          You requested up to 65536 tokens, but can only afford 3937.

The same scan with max_tokens=1024 completes and returns a real verdict. A
probe only needs room for one tool call and a mutation for one prompt, so
1024 is generous; --max-tokens raises it. This made scanning a hosted model
fail outright on exactly the credit-limited accounts most first-time users
have, which is the worst possible audience for it.

Separately, --api-key was a plain option with no envvar, so a real key could
only reach it as a command-line argument -- landing in shell history and
visible in `ps` for the duration of the run. It now reads MODELFUZZ_API_KEY.

Verified end to end against OpenRouter with no key on the command line:
  openai/gpt-4o-mini        3/3 seeds broke through at generation 1
  anthropic/claude-sonnet-5 0/3, refused through generation 3

Tests 71 -> 76. The stub client now records max_tokens per request, so both
call sites are asserted rather than just the probe.
@higagan
higagan merged commit a46387d into fix/urlallowlist-nested Jul 30, 2026
@higagan
higagan deleted the feat/scan-env-and-max-tokens branch August 3, 2026 15:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant