hotato 1.15.0: the loop, unified around hotato prove#34
Merged
Conversation
"The local-first AI engineering platform" named a category, not what the tool does. Every surface now leads with the concrete line: "Local-first testing and observability for AI agents", and the README opens on the hair-on-fire hook (your evals are green; your agent still ships bugs they can't see) instead of an abstract platform claim. - README hero + intro rewritten; PyPI Summary, llms.txt, server.json (<=100 chars), CITATION, CONTRIBUTING, the TS SDK, docs index, the CLI description, and the first-run screen all carry the new line. - tests/test_positioning_lockstep.py now ENFORCES the new positioning and retires "AI engineering platform" alongside the other abandoned framings, so the vague noun cannot creep back. - Derived surfaces regenerated; copy_lint + positioning/capture/llms suites green. GitHub repo About description updated to match. Claude-Session: https://claude.ai/code/session_01BqJW5Dey1DzqBeBCAJXrH9
Composes the evidence lanes you already ran into a single verdict with a portable receipt: --contracts re-scores contracts through contract verify's own logic, --suite runs a conversation-test suite, --before/--after scores the battery rollup through hotato verify (with its min-n refusal), --gauntlet runs the bundled stress suite. Pure composition: every number is one existing command's measurement; prove adds no scoring engine. Fail-closed: overall pass only when every activated lane passed; any failed or regressed lane fails the proof (exit 1); a refused or inconclusive lane with none failed is inconclusive (exit 2), so CI never reads "could not tell" as green. Zero lanes is a usage error: a proof of nothing is refused. Writes proof.json (schema hotato.proof.v1) + proof.md: verdicts, counts, relative names, and sha256 digests only (no transcript text, audio bytes, or absolute paths), created_at honors SOURCE_DATE_EPOCH, and content_id is the sha256 of the canonical JSON without the content_id field, so the same inputs regenerate the same address byte for byte. Adversarially verified in a fresh context: 12/12 dedicated tests, byte-identical re-runs under a pinned epoch, composition honesty confirmed by call-site inspection, neighboring contract/start suites green. Claude-Session: https://claude.ai/code/session_01BqJW5Dey1DzqBeBCAJXrH9
Every production failure becomes a portable test, every candidate runs against it, and every release carries evidence. New docs/LIFECYCLE.md maps every command onto the loop (observe, catch, pin, test, prove, and the production edge that feeds the next pass). docs/COMPARE.md drops the complementary framing for the lifecycle side by side with hosted platforms and the three structural differences. README restructured: the loop table replaces the four planes, hotato prove is the capstone with its verbatim output, and the quickstart ends in a release proof. llms.txt lead carries the loop. Claude-Session: https://claude.ai/code/session_01BqJW5Dey1DzqBeBCAJXrH9
hotato prove is new functionality: minor bump. Version in lockstep across pyproject / __init__ / server.json / CITATION / README pin / llms.txt / docs; atlas records, PyPI readme, and llms-full regenerated; CHANGELOG 1.15.0 entry. Claude-Session: https://claude.ai/code/session_01BqJW5Dey1DzqBeBCAJXrH9
…-full prove's _file_sha256 read a CLI-supplied path with a raw open(), which the FIFO-guard structural lint (test_no_unguarded_opens) rightly refuses; it now routes through errors.open_regular. Adds the dedicated gauntlet-lane proof test the adversarial verifier noted as missing (10/10, verdict pass, counts pinned). llms-full regenerated with the now-tracked docs/LIFECYCLE.md. Claude-Session: https://claude.ai/code/session_01BqJW5Dey1DzqBeBCAJXrH9
hotato turn-taking eval8 of 8 scenarios pass. 0 fail. No regression.
RegressionsNone. Reproducible timing measured locally from call audio. Swap the bundled self-test step for your own captured recordings to gate on your agent. github.com/attenlabs/hotato |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
hotato 1.15.0 — the loop, unified
Unifies the product around one sentence: every production failure becomes a
portable test, every candidate runs against it, and every release carries
evidence.
hotato prove(new)One command composes the evidence lanes you already ran (contracts, suites,
before/after batteries, the bundled stress suite) into one fail-closed verdict
with a portable, content-addressed receipt (
hotato.proof.v1). Purecomposition over the existing lane functions; no new scoring engine.
Adversarially verified: byte-identical re-runs under a pinned epoch,
fail-closed on every path, no path leaks, 13 dedicated tests.
The story is the loop
docs/LIFECYCLE.md: Observe -> Catch -> Pin -> Test -> Prove, every commandmapped, plus the production edge that feeds the next pass.
docs/COMPARE.md: the lifecycle side by side with hosted platforms and thethree structural differences; the complementary framing is retired.
AI agents"; the lockstep test retires the vague platform noun.
Full gate: 4715+ green (the 4 tail failures were the not-yet-cut tag, the
llms-full regen, and one FIFO-guard finding in prove, all fixed in follow-ups).
Version 1.15.0 in lockstep across every surface.
https://claude.ai/code/session_01BqJW5Dey1DzqBeBCAJXrH9