feat(keywords): Custom Keywords backend — glossary-driven transcript + summary correction (draft) - #367
Draft
Optic00 wants to merge 1 commit into
Draft
Conversation
…+ summary correction Backend-only slice of Custom Keywords (RFC-adjacent, issue stenolabs#365): a user glossary of preferred spellings + aliases that (a) persists in config.json, (b) heals the transcript at transcription time and on reprocess, and (c) injects a reference block into the summariser prompts. Empty glossary is a no-op (zero behavior change). - src/keywords.py: parse/normalize (defensive against malformed config), boundary-aware + diarised-safe alias replacement, reference-block builder. - src/config.py: get/set_custom_keywords (validated, replace-not-merge). - src/summarizer.py: reference block injected into permissive/markdown/ template-report/title/query prompts. - simple_recorder.py: guarded transcript heal in the pipeline + live-fallback + reprocess; get/set-custom-keywords CLI. Never heals the silence sentinel. - Unit tests incl. silence-sentinel safety, diarised timestamped labels, and malformed-config hardening. Test fixtures use fictional placeholders only. UI/IPC bridge + e2e spec land in a follow-up PR (re-integrated against stenolabs#344). Refs stenolabs#365.
Optic00
force-pushed
the
feat/custom-keywords-backend
branch
from
July 22, 2026 08:49
b9fb5da to
cbebb93
Compare
21 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft — backend slice of Custom Keywords. Needs a hardening pass before it is production-ready (see checklist). Part of #365.
What this is
A user-maintained Custom Keywords glossary (preferred spellings + aliases / common mis-hearings). Backend-only slice:
src/keywords.py(new) — parse/normalize the glossary (defensive against malformed config), boundary-aware alias replacement (diarised-safe), and the summary reference-block builder.src/config.py—get_custom_keywords/set_custom_keywords(persist inconfig.json, replace-not-merge, validated).src/summarizer.py— inject a reference block into the permissive, markdown, template-report, title and query prompts. Empty glossary = byte-for-byte unchanged prompts (zero behavior change when unused).simple_recorder.py— apply the glossary to the transcript at transcription time and onreprocess(retroactive heal, no re-ASR), plusget-custom-keywords/set-custom-keywordsCLI.test_keywords,test_keywords_cli,test_keywords_pipeline,test_reprocess_keywords,test_summarizer_keywords,test_configadditions.Out of scope here (follow-up PR): the Settings UI (re-integrated against the #344 nav-rail redesign), the IPC bridge, and the
custom-keywords.t2e2e spec. This slice is independently verifiable viaruff+unittest.Reconstruction note
The original prototype branch was ~82 commits behind
main; this is a clean re-application onto currentmain(semantic merge into the divergedsummarizer.py/config.py/simple_recorder.py), not a cherry-pick.Review round 1 — real bugs fixed (with regression tests)
An independent cross-family review found and this branch fixes:
speech -> Speechlycould mutate the"No speech detected in audio"sentinel, defeat the silence/live-rescue check, and get a fake-empty transcript summarised/saved (audio possibly deleted). Fixed: the sentinel is never healed.[timestamp], leaving[You]/[Others]exposed (real format is[MM:SS] [You] text). Fixed to protect every leading[...]label.normalize_keywordsdidn't validate persisted shape — a hand-edited"aliases": "foo"(string) was char-iterated into aliasesf,o. Fixed with defensive coercion + control/newline stripping.Hardening checklist before un-drafting (why this stays a draft)
A second adversarial review surfaced a bigger design issue than the individual deferrals — ordered by severity:
reprocessruns on already-healed text (and audio may later be deleted), so a wrong glossary rule is irreversible — the original wording cannot be recovered. Before production: keep an immutable raw transcript and derive the healed one from it (or persist replacement provenance and reprocess from the raw source). The most likely silent corruption is a short common-word alias, e.g.United States: USturning "let us know" into "let United States know" — word boundaries do not resolve semantic ambiguity, and today that mistake is permanent._needs_chunkingdoes not budget for. Filter to terms actually present in the transcript, or hard-cap the injected set.openai->OpenAI) is currently discarded, but capitalization enforcement is a central "preferred spelling" use case. Replacement is single-pass, so it is safe and idempotent — keep it, don't drop it.custom-keywords.t2).Verification
ruff check .— zero new errors vsorigin/mainbaseline.python -m unittest discover tests— 515 passed (skipped 1), incl. the round-1 regression tests.Refs #365.