A modular framework for agentic Bayesian optimization.

Docs · Blog · Install · Plugins · Experiments · Contributing

Full documentation, including guides and a plugin reference, is at richardcsuwandi.github.io/plugbo.

Bayesian optimization (BO) uses a probabilistic surrogate to choose each evaluation, but its overall search strategy is usually fixed in advance. Agentic BO places an LLM agent at the center of the loop: it reviews trial data, surrogate diagnostics, and natural-language context, then decides how to search next. As evidence accumulates, the agent can query or override backend suggestions and revise the surrogate, acquisition function, search bounds, objectives, or constraints [1].

Meta's Agentic Bayesian Optimization through Surrogate-Augmented Autoresearch instantiates that idea as Sara, an LLM agent, calling lenz, a BoTorch backend that owns the trial log and posterior computation [1] [6]. PlugBO keeps that agent and backend, then turns the backend into a plugin surface: surrogate, region, prior, and sampler are slots. Existing and new BO modules wrap as lenz verbs the agent can enable, inspect, or override. The arrangement is analogous to MCP: Sara is the host, with only bash and read, and lenz is the shared tool surface. A BO method registers extra verbs there the way an MCP server registers tools, so the agent stays fixed while the surrogate, region, prior, or sampler can be swapped in as a plugin.

Note: PlugBO is not an official implementation of Meta's agentic BO paper [1]. What is new here is the plugin protocol, together with new experiments and results.

The technical blog post covers the implementation, experimental setup, and results in more detail.

Why PlugBO?

• Provides a modular and easily extensible interface for composing BO methods. The surrogate, region, prior, and sampler are slots on a shared BoTorch backend.
• Places an LLM agent in a live BO loop. The agent can inspect trials, query the posterior, and reconfigure slots mid-run, while BoTorch retains the trial log and posterior.
• Supports future BO development as tools on a shared interface analogous to MCP. A method registers extra lenz verbs the way an MCP server registers tools, so the agent stays fixed while new modules can be plugged in.

---

Install

Not on PyPI. Two ways to install from source (Python 3.10+):

Editable install

git clone https://github.com/richardcsuwandi/plugbo.git
cd plugbo
pip install -e .

That pulls torch, gpytorch, botorch, and LLM clients (anthropic, openai). Copy .env.example to .env if you will call an LLM.

Dev install (tests, plus the LoRA HPO emulator extra)

pip install -e ".[dev]"    # tests
pip install -e ".[bolt]"   # LoRA HPO emulator (BoLT)

---

Plugins

Sara, lenz, and the BO methods are the same kind of piece: modules on one control plane. Sara is the host (bash and read). lenz is the tool surface. Existing and new BO modules occupy slots on that surface. The current version of PlugBO supports the following BO modules:

Module	Role	Default	Occupant	Commands
sara	search agent	—	—	sara run
lenz	trial log, posterior, acquisition	BoTorch loop	—	create, suggest, submit, incumbent
Surrogate	GP	fixed Matérn	CAKE [2]	set-surrogate, evolve-kernels, kernel-population
Region	search bounds	box	TuRBO [3]	set-region, set-bounds, turbo status
Prior	belief	none	πBO [4]	set-belief
Sampler	candidates	BoTorch	LLAMBO [5]	set-sampler, llambo sample

Occupying a slot is a set-* verb. Plugins may add more verbs. Method state lives in state.json under plugins, not on the live shelf. Reconfiguring never discards trials. lenz plugins lists installed modules.

Vanilla loop (no LLM):

lenz create --state ./state.json \
  --space '{"x1":{"kind":"range","lower":-5,"upper":10},"x2":{"kind":"range","lower":0,"upper":15}}' \
  --objectives '{"y":"minimize"}' \
  --acqf noisy_logei

lenz suggest --state ./state.json
lenz submit --state ./state.json --config '{"x1":1.0,"x2":2.0}' --metrics '{"y":12.3}'
lenz incumbent --state ./state.json

Agent in the loop:

sara run \
  --provider anthropic --model claude-opus-5 \
  --context examples/branin/context.md \
  --eval "python3 examples/branin/eval.py" \
  --budget 30 \
  --workdir ./results/logs/branin-1

--context is problem markdown, --eval is a command that takes one config JSON. --no-lenz drops the backend: Sara proposes configs and calls ./oracle herself.

--provider	Notes
anthropic	ANTHROPIC_API_KEY
openai	OPENAI_API_KEY
openai-compatible	Requires --base-url
ollama	http://localhost:11434/v1 by default

Occupy a slot, with or without the agent:

lenz set-surrogate --state ./state.json --surrogate cake
lenz set-region --state ./state.json --policy turbo
lenz set-belief --state ./state.json --prior '{"x":{"dist":"normal","mu":0.3,"sigma":0.1}}'
lenz set-sampler --state ./state.json --sampler llambo

lenz create --state ./state.json \
  --space '{"x1":{"kind":"range","lower":-5,"upper":10},"x2":{"kind":"range","lower":0,"upper":15}}' \
  --objectives '{"y":"minimize"}' \
  --surrogate cake --budget 30

CAKE and LLAMBO inherit Sara's provider and model. Pass --kernel-llm-* or --sampler-llm-* only to use a different model. TuRBO and πBO do not call an LLM. CAKE evolves a GP kernel population inside lenz during observe/submit [2]; that trace is not part of Sara's conversation.

Command reference: sara/prompts/LENZ_REF.md. To add a method, implement LenzPlugin in lenz/plugins/, ship a {name}.md prompt beside it, and register it in registry.py. See CONTRIBUTING.md.

---

Experiments

Textbook functions such as Hartmann and Ackley have published optima that an LLM can recall. The synthetic harness therefore uses an anti-memorization sandbox: renamed parameters, a unit cube, and a shifted optimum. Scoring uses a hidden answer key outside the sandbox. List functions:

python3 -c "from benchmarks.functions import REGISTRY; print(sorted(REGISTRY))"

./scripts/run_synthetic.sh hartmann6
./scripts/run_synthetic.sh hartmann6 --disclosure revealed
./scripts/run_synthetic.sh hartmann6 --disclosure revealed-shift
./scripts/run_synthetic.sh hartmann6 --backend sara-lenz --disclosure all
./scripts/run_synthetic.sh hartmann6 --warmup 7

--disclosure controls how much identity the agent sees:

• blind (default): renamed parameters, unit cube, shifted optimum, generic problem text. This is the search comparison, not a retrieval test.
• revealed-shift: real name and bounds, same shift as the blind run. Search is still required.
• revealed: the textbook problem, unshifted. The headline metric is whether evaluation 1 is already the known optimum.
• all: run the three levels. Needs a single --backend.

--backend is a comma list (vanilla, cake, turbo, sara-lenz, sara-lenz-cake, sara-only), or all for vanilla + sara-lenz + sara-lenz-cake. Every backend uses the same seeded Sobol warm-start, defaulting to d+1 evaluations when a seed is set. Pass --warmup N to override it for all selected backends.

gp_sample<dim> is a no-prior control (a fresh GP sample path, not in REGISTRY). Run it the same way as hartmann6.

LoRA hyperparameter optimization

The mixed-type experiment is LoRA hyperparameter optimization on BoLT (Black-box Optimization for LLM Tasks) [7]. There is no textbook optimum to recall. The oracle is a deterministic emulator of expensive LLM fine-tuning runs. The search space is always revealed (seven mixed continuous, integer, and categorical variables). --context only changes the story the agent reads, not the space:

• domain (default): real LoRA/Qwen names and a short task description.
• generic: names, types, and bounds only. No domain prose.
• misleading: false LoRA folklore (dropout near 0.05, lora_target = 0, few layers) presented as known-good defaults.

pip install -e '.[bolt]'
./scripts/run_bolt.sh
./scripts/run_bolt.sh --backend vanilla,cake
./scripts/run_bolt.sh --context generic

Provider and model come from .env or PROVIDER / MODEL. Completed and in-flight legs are skipped. Plots land in compare.html next to the run, or open plugbo-viz.

---

Run viewer

plugbo-viz is a local UI over run logs. Open a run to read the trial table, the agent trace, tool-use over the run, and (when CAKE is on) the kernel population. Search the sidebar, tick several runs to overlay them, or switch to Experiments to put every condition in a group on one regret chart.

plugbo-viz
plugbo-viz --root ./results/logs --port 9000

You can also run python3 -m viz.merged_server to see a single merged regret chart for all conditions, or open plugbo-viz to compare multiple runs.

Development

pip install -e ".[dev]"
pytest -m "not slow"
./scripts/smoke_plugins.sh

Optional smoke flags and what they cover are in CONTRIBUTING.md. The script header is the source of truth for --live, --baseline, and any later flags.

---

Contributing

Bug reports, new BO plugins, and benchmarks are welcome.

• CONTRIBUTING.md: setup, plugin protocol, adding a function
• Issues: bugs and proposals
• Code of Conduct

Please open an issue before a new slot, a large dependency, or a new experiment backend.

---

Citation

If you use PlugBO in your research, please cite:

@software{plugbo,
  title = {PlugBO: A modular framework for agentic Bayesian optimization},
  author = {Richard Cornelius Suwandi},
  year = {2026},
  publisher = {GitHub},
  url = {https://github.com/richardcsuwandi/plugbo}
}

---

License

MIT. See LICENSE. Contributions are under the same license.

---

References

1. Brunzema, P., Tiao, L., Le, N., De Angeli, K., Xuan, Y., Gligorijevic, D. Agentic Bayesian Optimization through Surrogate-Augmented Autoresearch. arXiv:2608.00316, 2026. Paper No official code (this repo is an independent re-implementation).

2. Suwandi, R. C., Yin, F., Wang, J., Li, R., Chang, T.-H., Theodoridis, S. Adaptive Kernel Design for Bayesian Optimization Is a Piece of CAKE with LLMs. NeurIPS 2025. Paper Code

3. Eriksson, D., Pearce, M., Gardner, J., Turner, R. D., Poloczek, M. Scalable Global Optimization via Local Bayesian Optimization. NeurIPS 2019. Paper Code

4. Hvarfner, C., Stoll, D., Souza, A., Lindauer, M., Hutter, F., Nardi, L. πBO: Augmenting Acquisition Functions with User Beliefs for Bayesian Optimization. ICLR 2022. Paper Code

5. Liu, T., Astorga, N., Seedat, N., van der Schaar, M. Large Language Models to Enhance Bayesian Optimization. ICLR 2024. Paper Code

6. Balandat, M., Karrer, B., Jiang, D. R., Daulton, S., Letham, B., Wilson, A. G., Bakshy, E. BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization. NeurIPS 2020. Paper Code

7. Chew, R. W. T., Chen, Z., Hemachandra, A., Low, B. K. H. BoLT: A Benchmark to Democratize Black-box Optimization Research for Expensive LLM Tasks. arXiv:2605.17000, 2026. Paper Code