Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/banner.html

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion .github/social-card.html

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -102,3 +102,6 @@ docs/SAA-BEHAVIOR-CARD.md
docs/SAA-FIX-POINTER.md
docs/saa_behavior_card.py
tests/test_saa_card.py

# hotato start writes its demo saydo/ output to CWD; ignore it at repo root
/saydo/
7 changes: 7 additions & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
Expand Up @@ -107,3 +107,10 @@ exclude docs/saa_behavior_card.py
exclude tests/test_saa_card.py
recursive-exclude docs SAA-*.md saa_*.py
recursive-exclude tests test_saa_*.py

# Rendered showcase reports with base64-embedded audio (~12 MB combined) are
# documentation artifacts, not runtime dependencies: no code or test reads them.
# Keep them in git (linked from the corpus READMEs) but exclude them from the
# shipped sdist so a plain `pip install hotato` is ~0.6 MB instead of ~12 MB.
exclude corpus/vapi-defaults/sample-report.html
exclude corpus/real/sample-report.html
6 changes: 3 additions & 3 deletions docs/CONTRACTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ reports pass/fail.
| **Re-scores** | the SAME `audio/event.wav` the contract was created from | a new recording of the SAME stimulus against your CURRENT agent |
| **A pass proves** | the evidence, policy, and scorer are still intact and still agree with the human label | the CURRENT agent's behavior on that stimulus still matches the label |
| **Leaves open** | whether the deployed agent's behavior has changed since | nothing extra -- this lane speaks to the live agent |
| **Runs** | every push, in the shipped `ci/github-action.yml` (`contract verify contracts/`) | only when you recapture by hand or on a schedule -- see [`docs/RECAPTURE.md`](RECAPTURE.md) |
| **Runs** | every push, in the shipped `ci/github_action.yml` (`contract verify contracts/`) | only when you recapture by hand or on a schedule -- see [`docs/RECAPTURE.md`](RECAPTURE.md) |

A frozen-recording pass is necessary but not sufficient: the recording
never changes, so it can only fail if someone edits the bundle's audio
Expand Down Expand Up @@ -149,7 +149,7 @@ Exit codes are the CI contract:
| `2` | usage error, empty directory, or corrupt `contract.json` |

`--junit` writes one `<testcase>` per contract; the shipped
`ci/github-action.yml` scaffold runs this on push, on PR, and weekly,
`ci/github_action.yml` scaffold runs this on push, on PR, and weekly,
and publishes the JUnit file as an artifact.

Every text and HTML render of `verify` also prints, verbatim: *"This
Expand Down Expand Up @@ -238,7 +238,7 @@ matching the packed manifest.

## CI

The shipped `ci/github-action.yml` is the minimal wiring:
The shipped `ci/github_action.yml` is the minimal wiring:

```bash
uvx hotato contract verify contracts/ --junit contracts-junit.xml \
Expand Down
10 changes: 7 additions & 3 deletions docs/LIFECYCLE.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,9 +91,13 @@ deterministic: the same scenario and seed render the same bytes on every run.

## 5. Prove

One command composes every evidence lane you have into one fail-closed release
proof: contracts re-verified, suites re-run, before/after movement measured,
the stress suite cleared. The proof is a content-addressed receipt; CI gates on
One command composes every evidence lane you have into one fail-closed,
content-addressed proof: contracts re-verified, suites re-run, before/after
movement measured, the stress suite cleared. The proof headlines its claim
scope, exactly what the evidence establishes: contracts alone re-measure stored
evidence (Captured Evidence), a suite or the stress suite establishes a Test
Suite ran, and a before/after run reaches Candidate Revision only when you bind
the candidate identity (`--candidate-config-hash`, `--provider`). CI gates on
the exit code, and the receipt stays verifiable anywhere.

```bash
Expand Down
207 changes: 0 additions & 207 deletions docs/assets/hotato-cast.cast

This file was deleted.

Binary file removed docs/assets/hotato-cast.gif
Binary file not shown.
16 changes: 10 additions & 6 deletions llms-full.txt

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 3 additions & 1 deletion llms.txt
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@
> finds what text evals miss), pin (a labeled failure becomes a portable
> content-addressed contract), test (simulate, drive, and the bundled stress
> suite), and prove (hotato prove composes every evidence lane into one
> fail-closed release proof). The wedge leads the demo: give it a two-channel recording or a
> fail-closed, content-addressed proof headlining its claim scope: contracts
> alone are Captured Evidence, a suite is a Test Suite, and a bound before/after
> run is a Candidate Revision, never over-claiming). The wedge leads the demo: give it a two-channel recording or a
> timestamped transcript and it measures turn timing and say-do evidence -- did
> the agent yield when the caller took the floor, how fast, how many seconds
> both talked at once, and did what the agent said match what the backend did.
Expand Down
8 changes: 0 additions & 8 deletions saydo/state.json

This file was deleted.

Loading
Loading