fix(observer): merge split user utterances into same turn - #101
Merged
Conversation
STT/VAD can split a single user utterance into two finals with a short/backchannel-like agent reply interjected in between (e.g. a premature partial answer). The turn-boundary heuristic in Observer.on_transcript only merged consecutive user finals when NO agent final had landed yet, so the second half was counted as a brand new turn, corrupting turn_taking_ms (issue #98's root cause) and making transcript.turn_metrics() drop the first half of the utterance entirely (event_writer overwrote user_text instead of appending). - observer.py: extend the same_turn merge to also cover a short agent final (<=6 words) landing within a short grace window (700ms) of the prior user final — treat it as an interjection, not the turn-ending answer, and keep waiting for the real reply. - event_writer.py / lks-core event.rs: turn_metrics() now concatenates user_text across same_turn merges instead of overwriting it, in both the Python and Rust implementations. - asserts.rs: drop-in fix for a pre-existing duplicate `negate` field in OutcomeExpect (introduced by the #97 merge) that broke the lks-core build entirely; unrelated to this bug but blocked verifying the Rust changes above. Tests: added test_split_utterance_with_short_agent_interjection_stays_same_turn and test_new_turn_starts_after_a_real_agent_answer to tests/test_observer.py. Full python suite (682 tests) and `cargo build -p lks-core` pass.
The #97 merge (negate/OutcomeExpect) left a duplicated test-function block and duplicated struct-literal fields in crates/lks-core/tests/asserts.rs (mirrors the duplicate-field bug in asserts.rs itself, fixed in the prior commit) — this broke `cargo test -p lks-core` with E0428/E0062. Removed the duplicate outcome_transcript_contains_negate_{pass,fail}_* functions and the duplicate negate: false lines.
qdang46
added a commit
that referenced
this pull request
Sep 4, 2026
…) (#102) * fix(evals): ground judge verdicts to reduce hallucination false-pass (#99) The LLM-judge path (llm_bool/goals_met/constraint_respected assert types + judge_run) let a criterion's met=true stand even when the judge cited no transcript evidence for it, and unconditionally promoted a fail→pass whenever the model self-labeled the failing criteria as "irrelevant" — neither self-label is independently verified. Together these let a topically-related-but-wrong agent reply (e.g. "I'm ready to hear more about your company..." instead of re-asking for the missing company name) slip through as a pass. - evals/types.py::parse_judgment_payload: when a criterion is reported met=True with blank/missing evidence, flip it to met=False and mark the whole judgment needs_human_review — an ungrounded "met" claim is not trustworthy. - evals/relevancy.py::apply_relevancy: keep the fail→pass promotion when all relevant criteria are met (usually correct), but flag needs_human_review instead of trusting the model's own relevant=False self-label blindly. Tests: added grounding/promotion coverage to tests/test_evals_judge.py. Full suite (uv run pytest -q) → 683 passed. * fix(rust): drop-in fix for duplicate negate field breaking lks-core build Same pre-existing main-branch build break as in #101 (introduced by the #97 merge); needed here too since this branch was cut from main before that fix landed. * fix(rust): dedupe botched merge-conflict remnants in asserts.rs tests Same pre-existing #97 merge damage as on fix/turn-split-merge — this branch was cut from main before that fix landed. --------- Co-authored-by: quangdang46 <tranquangdang21@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #100. Root-causes issue #98 (latency false readings caused by turn misattribution).
Problem
STT/VAD can split a single user utterance into two
transcript.user.finalevents with a short/backchannel-like agent reply interjected in between.Observer.on_transcript's turn-boundary heuristic only merged consecutive user finals into the same turn when no agent final had landed in between — so the second half was always counted as a new turn.Side effects:
turn_taking_ms(used by the latency asserts in Latency detector misses ~9s gap before follow-up question (ebroad scenario) #98) is measured against the spurious interjection instead of the real answer, producing artificially fast/slow/missing latency samples.EventWriter.turn_metrics()overwroteuser_textper turn instead of appending, so the first half of the split utterance was silently dropped from the transcript row the judge/asserts see.Fix
observer.py: extend thesame_turnmerge to also cover a short agent final (≤6 words) that lands within a 700ms grace window of the prior user final — treat it as an interjection rather than the turn-ending answer, and keep waiting for the real reply. The existing echo/dedupe check runs first so it doesn't change behavior for legitimate short agent acks.event_writer.py(and the Rust mirror inlks-core/src/logging/event.rs):turn_metrics()now concatenatesuser_textacrosssame_turnmerges instead of overwriting it.asserts.rs: unrelated drive-by fix —cargo build -p lks-corewas broken onmainby a duplicatenegatefield inOutcomeExpectintroduced by the Feat/negate transcript assert #97 merge; fixed so the Rust changes above could actually be verified to compile.Tests
test_split_utterance_with_short_agent_interjection_stays_same_turnandtest_new_turn_starts_after_a_real_agent_answertotests/test_observer.py.uv run pytest -q→ 682 passed.cargo build -p lks-coreandcargo test -p lks-core --lib→ pass.