llm reads OPENAI_BASE_URL but not OPENAI_API_BASE — and replaying a whole script of llm calls offline

[xizhuomengcontin]

Two small things, one of which is useful on its own.

llm honours OPENAI_BASE_URL, not OPENAI_API_BASE

If you want to point llm at a local endpoint or a gateway without editing extra-openai-models.yaml, only one of the two common spellings works. Local HTTP server on 127.0.0.1 returning a canned chat-completion and logging the path it is asked for, llm 0.35:

OPENAI_BASE_URL=http://127.0.0.1:9984/v1  llm -m gpt-4o-mini hi --no-stream
  -> request arrived at /v1/chat/completions, printed "ok"

OPENAI_API_BASE=http://127.0.0.1:9985/v1  llm -m gpt-4o-mini hi --no-stream
  -> nothing arrived; Error code: 401 from the real API

Both names turn up across the ecosystem — I have probed libraries that honour both, and one that honours neither because it pins the origin in its own config — so it is worth knowing which one llm reads before you assume your traffic went where you meant it to.

Replaying a session, which llm logs deliberately does not do

llm logs already gives you every prompt and response in SQLite, and for most purposes that is the right tool — it is searchable, it is yours, and it needs nothing extra.

What it does not do is let you run the thing again without calling the model. That is the gap I fill with a recording proxy:

npm i -g orcareplay

orca record generic-openai -- llm -m gpt-4o-mini 'summarise this' < notes.md   # real call, recorded
orca replay last                                                               # the same call again, no model

Because the recorder wraps the process rather than llm itself, a whole shell script full of llm invocations records as one run and replays as one run — which is the case I actually wanted it for: a pipeline of half a dozen llm calls that you want to iterate on without paying for the whole chain each time.

orca replay last --from N --model <other> replays up to step N from the recording and then continues on a different model, so comparing models over a fixed prefix is a single command.

Disclosure: OrcaReplay is mine — Apache-2.0, Node 20+, no account, replay needs no key. The llm measurement above stands on its own.

What it does not give you

• A matching replay is not a determinism result. It proves the recorded exchange reproduces, not that a fresh call would answer the same way.
• Replay blocks model-provider egress only. It is not a sandbox — everything else in your script still runs for real.
• Embedding calls are not captured by the default adapter, so llm embed work is not served from the recording.
• If all you need is "what did I ask and what came back", llm logs is simpler and you should use that.