Skip to content

feat: prefer two-pass finals and add clean output mode - #23

Merged
SidUParis merged 3 commits into
mainfrom
feature/definite-clean-output-final
Sep 1, 2026
Merged

feat: prefer two-pass finals and add clean output mode#23
SidUParis merged 3 commits into
mainfrom
feature/definite-clean-output-final

Conversation

@SidUParis

Copy link
Copy Markdown
Owner

Summary

  • assemble Volcengine terminal text from official timestamped definite=true two-pass utterances, with safe full/single malformed-frame handling and bounded state
  • add a conservative local delete-only clean-expression processor and faithful/clean per-utterance output style
  • keep partials and provider finals raw, store replayable schema-v3 delivery separately, and skip adaptive ASR learning when local cleaning changed the delivered text
  • wire output style through GTK, CLI, systemd, source/Debian packaging, privacy gates and migration docs

Safety boundaries

  • missing configuration remains faithful; no silent behavior change for existing users
  • no LLM, extra network request, startup delay, gain change, term/case/number rewrite or global punctuation normalization
  • malformed/unsafe cleaner results fall back to raw final
  • review-last continues to compare raw provider text with human-confirmed verbatim

Validation

  • Voice 710/710; Engine 38/38; scripts 140/140
  • isolated real IBus smoke; Ruff/format/compile/diff/shell/docs deterministic checks
  • clean history secret scan
  • preview and Debian double builds byte-identical and verified

No real microphone recording, provider request, key access or installation was used for acceptance.

Copilot AI lite review requested due to automatic review settings September 1, 2026 18:47

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

It introduces at least one confirmed logic bug in clean-expression edit coalescing and makes the review socket client unnecessarily incompatible with older daemons by requiring delivered_text.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR enhances the voice daemon’s final-text pipeline by (1) assembling Volcengine finals from official timestamped two-pass utterances (preferring definite=true) and (2) adding a private, hot-loaded “output style” that can deliver either faithful provider finals or a bounded local deletion-only “clean” delivery, with schema-v3 replay/audit support and updated UX/packaging/docs.

Changes:

  • Implement a bounded Volcengine two-pass utterance assembler keyed by time intervals and validate/lock supported result_type values.
  • Add output-style.json (faithful/clean) with deterministic clean-expression delivery, freeze-per-utterance semantics, and integrate through session, UI, CLI, systemd, and dataset retention (schema v3 + usage schema v2).
  • Expand tests and documentation to cover new delivery semantics, privacy boundaries, and artifact/packaging exclusions.
File summaries
File Description
voice/tests/test_volcengine.py Adds extensive coverage for two-pass utterance assembly, malformed-frame handling, and result_type validation.
voice/tests/test_settings_controller.py Adds output-style persistence/load tests and usage-summary schema v2 validation tests.
voice/tests/test_settings_app.py Adds GTK UI tests for output-style controls and delivered-text display in review UX.
voice/tests/test_session.py Adds session-level tests for clean delivery, freezing mode at start, and learning-skip behavior.
voice/tests/test_output_style.py New tests for output-style config I/O, delivery behavior, and delivery validation.
voice/tests/test_data_collection.py Updates retention tests for schema v3 delivery + usage schema v2 counting delivered text.
voice/tests/test_control.py Updates review socket tests to round-trip delivered text and larger JSON bounds.
voice/tests/test_cli.py Wires --output-style into CLI and verifies hot-reload behavior.
voice/tests/test_clean_expression.py New unit tests for deterministic deletion-only clean-expression behavior and bounds.
voice/README.md Documents faithful vs clean output and schema v3 + usage v2 semantics.
voice/pyproject.toml Bumps voice package version to 0.1.0a7.
voice/murmur_voice/volcengine.py Implements _VolcengineResultAssembler, utterance parsing/validation, transcript bounds, and strict result_type.
voice/murmur_voice/settings_controller.py Adds load/save APIs for output style and validates usage-summary schema v1/v2 strictly.
voice/murmur_voice/settings_app.py Adds GTK output-style controls, status messaging, and delivered-text read-only field for review UI.
voice/murmur_voice/session.py Freezes output style per utterance, applies terminal delivery, persists delivery metadata, and skips auto-learning when delivery changed.
voice/murmur_voice/output_style.py New module: private config schema v1, delivery model, deliver/validate logic with safe fallback.
voice/murmur_voice/data_collection.py Upgrades record to schema v3 with delivery and usage to schema v2 counting delivered text.
voice/murmur_voice/control.py Extends LastReview to include delivered text and increases bounded review response size.
voice/murmur_voice/cli.py Adds --output-style path wiring into runtime session creation.
voice/murmur_voice/clean_expression.py New module: bounded deterministic deletion-only cleaner with replayable edit audit.
voice/murmur_voice/init.py Bumps module __version__ to 0.1.0a7.
scripts/verify_preview_bundle.py Updates preview bundle shape checks to include output-style config exclusion.
scripts/uninstall-user.sh Updates retained-settings messaging to include output-style.
scripts/tests/test_user_install.py Updates wheel/version expectations and verifies systemd units include output-style path.
scripts/tests/test_systemd_units.py Ensures units include --output-style and env var wiring.
scripts/tests/test_preview_sbom.py Updates SBOM fixture versions/paths to 0.1.0a7.
scripts/tests/test_preview_bundle.py Updates preview bundle forbidden-config list and wheel version expectations.
scripts/tests/test_deb_packaging.py Updates Debian version mapping and asserts unit includes explicit output-style config path.
scripts/install-user.sh Adds output-style config path and wires it into rendered systemd units.
scripts/build-deb.sh Extends packaging “must not own private config names” check to include output-style + interaction.
README.md Updates release link/version and describes faithful vs clean final delivery behavior.
README.en.md Documents faithful vs clean final delivery and updated settings list.
packaging/systemd/murmur-ime-voice.service.in Adds --output-style to ExecStart template.
packaging/debian/README.md Notes output-style is not owned/altered by packaging.
packaging/debian/murmur-ime-voice.service Adds explicit --output-style %E/.../output-style.json to ExecStart.
packaging/debian/io.github.SidUParis.OpenVoiceInputLinux.metainfo.xml Adds release note entry for 0.1.0-alpha.7.
engine/murmur_ime_engine/constants.py Bumps engine VERSION to 0.1.0-alpha.7.
docs/user-service.md Documents output-style config location, semantics, and updated retention schema.
docs/threat-model.md Extends threat model to cover clean delivery boundaries and new private setting.
docs/remote-dataset-storage.md Updates retention language to include delivery and its unreviewed semantics.
docs/release-process.md Adds release checklist items for clean output, schema v3, usage v2, and output-style privacy/exclusion.
docs/recognition-accuracy.md Documents Volcengine assembly behavior and faithful vs clean delivery semantics and learning-skip rule.
docs/README.zh-CN.md Adds Chinese docs for faithful/clean final delivery and schema v3/v2 changes.
docs/product-launch-plan.zh-CN.md Updates messaging to include delivery retention alongside provider text.
docs/privacy.md Documents output-style privacy guarantees and clean delivery boundaries.
docs/press-kit.zh-CN.md Updates press-kit language for retention semantics (raw + delivery).
docs/personal-asr-data-plan.md Updates plan for schema v3 including delivery and usage v2 meaning.
docs/launch-positioning.md Updates positioning to show raw provider + delivery retention semantics.
docs/architecture.md Updates architecture to include frozen terminal delivery layer and schema v3 retention semantics.
CHANGELOG.md Adds 0.1.0-alpha.7 release notes reflecting the new assembly + output style + schema changes.
.gitignore Adds interaction/output-style config files to ignore list.
.github/workflows/ci.yml Extends CI artifact/config exclusion list to include interaction and output-style.
Review details
  • Files reviewed: 51/52 changed files
  • Comments generated: 2
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread voice/murmur_voice/clean_expression.py
Comment thread voice/murmur_voice/control.py Outdated
@SidUParis
SidUParis merged commit 6cb5dfc into main Sep 1, 2026
4 checks passed
@SidUParis
SidUParis deleted the feature/definite-clean-output-final branch September 1, 2026 19:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants