feat: add SIMBAUQSamplingStrategy

[psschwei]

Fixes generative-computing/mellea#718
Fixes #80

Adds SIMBAUQSamplingStrategy, a confidence-aware sample selector for Mellea, to the agent-utilities package.

Given a single instruction, it generates len(temperatures) * n_per_temp samples across a temperature schedule, computes an N×N pairwise similarity matrix between them, and returns the sample with the highest estimated confidence. Confidence metadata (per-sample scores, the similarity matrix, temperatures used) is attached to the selected ModelOutputThunk under _meta["simba_uq"].

Two confidence-estimation methods:

• aggregation (data-free): scores each sample by aggregating its similarity to the others (mean, geometric/harmonic mean, median, max, min). Samples that agree with the majority score higher.
• classifier: a probabilistic classifier (random forest, trained from labeled data or supplied pre-fitted) predicts P(correct) from the similarity features.

Five pluggable similarity metrics: rouge, jaccard, difflib, levenshtein (no extra deps), and sbert (Sentence-BERT embeddings). The sbert metric and the classifier method live behind a simbauq optional extra (scikit-learn, sentence-transformers); the rouge/jaccard/difflib/levenshtein + aggregation path works with the base install.

Ships with unit + integration tests, a four-variant runnable example (including live Hugging Face training-data generation), and docs.

Based on SIMBA-UQ (Bhattacharjya et al., 2025), arxiv.org/abs/2510.13836.

Port of generative-computing/mellea#785 to the contribs repo

cc @avinash2692 @radum2275

[planetf1]

A couple of things worth a second look before this merges — mostly carrying over gaps from the mellea#785 review that didn't quite make it through the port to here.

CI never actually runs the async path. agent-utilities/pyproject.toml has skip_ollama = false for this package, so CI does spin up Ollama and pull granite4:micro. But package-ci.yml runs pytest -m "not qualitative and not e2e", and TestSIMBAUQIntegration is marked ollama, e2e, qualitative — so it's excluded there, and in the twice-weekly smoke job too (same workflow, no different filter). Net result: the whole async path in sample() — generation-failure handling, parse-error handling, the two spots I've commented on below — never runs in any CI job, despite Ollama being sitting right there ready to go. Might be worth a mocked-backend unit test for sample() itself, or rethinking the marker combo so this gets exercised somewhere automated.

On ownership — OWNERS for agent-utilities is just @generative-computing/mellea-maintainers, nobody specific. Fine for the lightweight stuff like top_k, but SIMBAUQ drags in scikit-learn/sentence-transformers and has some real footguns we've already been through in review (feature-count mismatches, similarity clipping, etc), and radum2275 isn't a maintainer. Worth asking whether he or psschwei should get added as owner for that file specifically — otherwise whoever's on rotation inherits debugging something fairly niche with nobody flagged as the person to actually ask.

[radum2275]

@planetf1 @planetf1 you can add me as maintainer for this component (simbauq strategy).

[planetf1]

@planetf1 @planetf1 you can add me as maintainer for this component (simbauq strategy).

This is handled by adding an OWNERS file in the directory for your component. See the contributing docs

[psschwei]

There's no perfect solution here, at least as the repo is currently structured.
Owners covers the entire subpackage, not just the simbauq sampling strategy.
Creating a new subpackage for every contribution will very quickly get out of hand too.
In this imperfect world, this seemed like the right balance (having the mellea maintainers reach out to Radu seems better than pinging Radu about things unrelated to his code), but happy to switch and use the package OWNERS file instead of the more granular CODEOWNERS.

@avinash2692 any thoughts?