feat: score-on-arrival — the console engine (serve --score-production)#42
Merged
Conversation
…ebuild-scores) A supervisor-pattern worker inside hotato serve polls the production evidence db (mode=ro + query_only + write-denying authorizer) for sessions reaching COMPLETE/QUIESCENT and scores each one -- one at a time, keyset queries only -- with the deterministic scan scorer over the session's recorded two-channel audio, writing durable records to a versioned console.sqlite3 sidecar beside the evidence db: per-dimension observations (never blended), ranked candidate moments, one measured failure-reason sentence, and timing derived purely from evidence event timestamps (per-hop latency keeps the reporting event's declared authority; turn spans and end-to-end are labeled derived:event_timestamps). Refusal is first-class: absent/unavailable/mono/unreadable audio is a NOT_SCORABLE record with its reason; a scorer crash on one session becomes an ERROR record and the loop continues; a persist failure surfaces through a minimal ERROR write path plus a counter and is retried -- a score is claimed only after its sidecar write commits. The sidecar is derived data, never authority: --rebuild-scores regenerates it entirely from the evidence db and exits, and the same evidence db always rebuilds to a byte-identical canonical dump (the one wall-clock column, created_at, is excluded from the comparison). Bind, auth, and the server's read-only route surface are unchanged. Claude-Session: https://claude.ai/code/session_015njkxtoAKioqTPT2k14xFr
…arrival entry; ruff import-sort on console_worker Claude-Session: https://claude.ai/code/session_015njkxtoAKioqTPT2k14xFr
hotato turn-taking eval8 of 8 scenarios pass. 0 fail. No regression.
RegressionsNone. Reproducible timing measured locally from call audio. Swap the bundled self-test step for your own captured recordings to gate on your agent. github.com/attenlabs/hotato |
This was referenced Jul 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Wave A PR-1 of the console program: every completed production session gets scored deterministically as it lands.
hotato serve --score-production: a supervisor-pattern background worker polls the production evidence db (strictly read-only: mode=ro + query_only + write-denying authorizer) for COMPLETE/QUIESCENT sessions and scores each — one at a time, bounded memory — with the existing deterministic scorer over the session's two-channel recording.console.sqlite3sidecar (derived, versioned, rebuildable): per-dimension observations (never blended), ranked candidate moments, one measured failure-reason sentence, timing derived from evidence event timestamps with provenance (per-hop latency keeps the reporting event's declared authority; derived spans labeledderived:event_timestamps), scorer version + config hash on every record.--rebuild-scores: regenerates the sidecar entirely from evidence, deterministically — same evidence db, byte-identical content (schema migration = rebuild).Independently verified by a fresh reviewer (48 checks green, own harness): read-only discipline, determinism incl. the incremental path, error-visibility injections, bounded-memory query shape, no new outbound calls. Two verifier blockers resolved in follow-up commits: restored a CHANGELOG heading the new entry had consumed, import-sort. Full suite green locally (4795 passed); CI is the clean-environment confirmation.
Spec: hotato-growth/research/CONSOLE-WAVE-A-SPEC.md (R1–R4, invariants I1–I5, I7)
https://claude.ai/code/session_015njkxtoAKioqTPT2k14xFr