Releases: kelkalot/probelock
Releases · kelkalot/probelock
Release list
v1.0.0
v0.4.0
v0.3.1
v0.3.0
v0.2.0
v0.1.0
A capability lockfile for local models: derive deterministic probes from your tool
schemas, score them with no LLM judge, and gate CI on within-model regression.
Capabilities (11)
tool_selection, tool_discrimination, needle_in_tools, arity_robustness,
arg_validity, required_args, structured_output, format_adherence, and three
negative/safety probes — tool_restraint, tool_permission, no_hallucinated_tool.
Backends
- Any OpenAI-compatible endpoint via
--endpoint(Ollama, vLLM, llama.cpp,
LM Studio, HF TGI, OpenAI, OpenRouter, …) - Anthropic / Gemini / Mistral / etc. via
--via anyllmor--via litellm
(optional extras:probelock[anyllm],probelock[litellm])
Also in this release
--samples Nwith a statistically-aware gate (--confidence, one-sided
two-proportion test; sub-significant drops are flagged "noisy", not failed)diff --format table | markdown | json | htmlprobelock initscaffolding and a composite GitHub Action (action.yml)
Install
Requires Python 3.10+ (the anyllm extra needs 3.11+).
uvx probelock --help
pip install probelock