perf: measure whether ort's load-time static init costs anything, and lazy-dlopen it if so

[kengio]

Context

Split out of #332, which turned out to be environmental (a VMware-on-Apple-Silicon artifact — bare-metal Pi 4 and Pi 5 are unaffected). While investigating it, one structural fact was established that stands on its own and is not specific to ARM or to virtualization:

onnxruntime is initialized at library load, on every invocation, regardless of the command.

Measured on the aarch64 Ubuntu guest against the default (semantic) build:

• the ort cpuid probe runs on --version, --help, and schedule register --remove
• it runs on onebrain nosuchverb, which fails at clap argument parsing — before any command logic executes
• no env var defers or silences it: ORT_LOGGING_LEVEL, ORT_DISABLE_ALL_LOGS, ORT_LOG_SEVERITY_LEVEL, OMP_NUM_THREADS
• a --no-default-features (lex-only) build shows none of it, confirming ort as the source

So every onebrain doctor, note read, schedule register, token check — the majority of invocations, including every hook and scheduled job — pays whatever ort's static initialization costs, to use a runtime it never touches.

What this issue is NOT

It is not a fix for #332. That warning only appears on virtualized ARM Linux whose hypervisor reports an invalid MIDR part number, and it is harmless. Closing #332 does not depend on this.

The honest state of the evidence

I have not shown that this costs anything user-visible. Timing --version on macOS (semantic build):

run 1: 1.10s     ← cold, dominated by page-in of a 400 MB debug binary
runs 2-5: 0.01s  ← warm

0.01 s warm is not a problem. So this is a measure-first issue, not a known regression. It may well turn out that ort's static init is nearly free once the pages are resident and there is nothing worth changing.

What makes it still worth measuring:

• the release binary is far smaller than the debug binary, so the cold-start profile differs
• the interesting case is cold start, which is what a launchd/systemd scheduled job and a Claude Code hook actually hit — a warm loop is the least representative case
• lower-powered targets (Pi Zero, ARM32, Raspberry Pi 4) will show any fixed cost more clearly than an M-series Mac
• onebrain is invoked as a hook on many editor/agent events, where a fixed per-invocation cost is paid constantly

Suggested work

1. Measure first. Cold-start --version (or doctor) on: macOS release build, Pi 4, Pi Zero/ARM32, comparing default vs --no-default-features. The lex-only build is the natural control — it has no ort at all. If the delta is negligible everywhere, close this and record the number.
2. Only if the delta is material: investigate linking ort so its runtime is dlopened lazily on first embedding use rather than initialized at load. ort exposes a load-dynamic feature; today the runtime arrives via fastembed with download-binaries, statically initialized.
3. Watch for the trade-off — lazy loading typically moves the cost to first search rather than removing it, and adds a runtime failure mode (runtime present at build, missing at run) that static linking does not have. That may be a bad trade for a search-first tool.

Acceptance

Either a recorded measurement showing the cost is negligible (and this closed as won't-fix), or a change that demonstrably reduces cold-start time for non-search commands on at least one real low-power target, with before/after numbers.

🤖 Generated with Claude Code