Feat/negate transcript assert - #97
Merged
Merged
Conversation
Port the Python negate feature to the Rust asserts module: - asserts.rs: add negate: bool field to OutcomeExpect; invert match in transcript_contains evaluation when negate=true; emit negate in JSON output. - tests/asserts.rs: add negate:false to existing literals + 2 new tests (negate_pass_when_absent, negate_fail_when_present). - Pre-existing clippy warning in scenario_ops.rs left untouched.
Resolve conflicts in asserts.rs / tests/asserts.rs: keep both the negate field (this branch) and the prompt field (main) on OutcomeExpect. Wire negate into the JSON->OutcomeExpect parser (main added parsing that this branch didn't have).
feat(rust): add negate to OutcomeExpect (transcript_contains assert)
qdang46
pushed a commit
that referenced
this pull request
Sep 4, 2026
The #97 merge (negate/OutcomeExpect) left a duplicated test-function block and duplicated struct-literal fields in crates/lks-core/tests/asserts.rs (mirrors the duplicate-field bug in asserts.rs itself, fixed in the prior commit) — this broke `cargo test -p lks-core` with E0428/E0062. Removed the duplicate outcome_transcript_contains_negate_{pass,fail}_* functions and the duplicate negate: false lines.
qdang46
pushed a commit
that referenced
this pull request
Sep 4, 2026
Same pre-existing #97 merge damage as on fix/turn-split-merge — this branch was cut from main before that fix landed.
qdang46
added a commit
that referenced
this pull request
Sep 4, 2026
* fix(observer): treat split user utterances as same turn (#100) STT/VAD can split a single user utterance into two finals with a short/backchannel-like agent reply interjected in between (e.g. a premature partial answer). The turn-boundary heuristic in Observer.on_transcript only merged consecutive user finals when NO agent final had landed yet, so the second half was counted as a brand new turn, corrupting turn_taking_ms (issue #98's root cause) and making transcript.turn_metrics() drop the first half of the utterance entirely (event_writer overwrote user_text instead of appending). - observer.py: extend the same_turn merge to also cover a short agent final (<=6 words) landing within a short grace window (700ms) of the prior user final — treat it as an interjection, not the turn-ending answer, and keep waiting for the real reply. - event_writer.py / lks-core event.rs: turn_metrics() now concatenates user_text across same_turn merges instead of overwriting it, in both the Python and Rust implementations. - asserts.rs: drop-in fix for a pre-existing duplicate `negate` field in OutcomeExpect (introduced by the #97 merge) that broke the lks-core build entirely; unrelated to this bug but blocked verifying the Rust changes above. Tests: added test_split_utterance_with_short_agent_interjection_stays_same_turn and test_new_turn_starts_after_a_real_agent_answer to tests/test_observer.py. Full python suite (682 tests) and `cargo build -p lks-core` pass. * style: cargo fmt * fix(rust): dedupe botched merge-conflict remnants in asserts.rs tests The #97 merge (negate/OutcomeExpect) left a duplicated test-function block and duplicated struct-literal fields in crates/lks-core/tests/asserts.rs (mirrors the duplicate-field bug in asserts.rs itself, fixed in the prior commit) — this broke `cargo test -p lks-core` with E0428/E0062. Removed the duplicate outcome_transcript_contains_negate_{pass,fail}_* functions and the duplicate negate: false lines. --------- Co-authored-by: quangdang46 <tranquangdang21@gmail.com>
qdang46
added a commit
that referenced
this pull request
Sep 4, 2026
…) (#102) * fix(evals): ground judge verdicts to reduce hallucination false-pass (#99) The LLM-judge path (llm_bool/goals_met/constraint_respected assert types + judge_run) let a criterion's met=true stand even when the judge cited no transcript evidence for it, and unconditionally promoted a fail→pass whenever the model self-labeled the failing criteria as "irrelevant" — neither self-label is independently verified. Together these let a topically-related-but-wrong agent reply (e.g. "I'm ready to hear more about your company..." instead of re-asking for the missing company name) slip through as a pass. - evals/types.py::parse_judgment_payload: when a criterion is reported met=True with blank/missing evidence, flip it to met=False and mark the whole judgment needs_human_review — an ungrounded "met" claim is not trustworthy. - evals/relevancy.py::apply_relevancy: keep the fail→pass promotion when all relevant criteria are met (usually correct), but flag needs_human_review instead of trusting the model's own relevant=False self-label blindly. Tests: added grounding/promotion coverage to tests/test_evals_judge.py. Full suite (uv run pytest -q) → 683 passed. * fix(rust): drop-in fix for duplicate negate field breaking lks-core build Same pre-existing main-branch build break as in #101 (introduced by the #97 merge); needed here too since this branch was cut from main before that fix landed. * fix(rust): dedupe botched merge-conflict remnants in asserts.rs tests Same pre-existing #97 merge damage as on fix/turn-split-merge — this branch was cut from main before that fix landed. --------- Co-authored-by: quangdang46 <tranquangdang21@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.