fix(sight): show non-streaming LLM calls in audit#681
Merged
Conversation
jfeng18
force-pushed
the
fix/sight-per-call-token
branch
from
74832bd to
d2ac3a6
Compare
Collaborator
Author
|
Gentle ping — this fixes #699 (per-call token always null in audit records). 3 discriminating unit tests + ECS E2E verified. Ready for review. |
jfeng18
force-pushed
the
fix/sight-per-call-token
branch
2 times, most recently
from
9c2caf8 to
3558624
Compare
Collaborator
Author
|
Per-call token audit (123 lines). Adds per-invocation token tracking to the audit path. |
jfeng18
force-pushed
the
fix/sight-per-call-token
branch
2 times, most recently
from
e79ff6e to
7bd8552
Compare
jfeng18
changed the title
jfeng18
force-pushed
the
fix/sight-per-call-token
branch
from
7bd8552 to
cae6cdb
Compare
audit --type llm only showed SSE (streaming) calls: analyze_http returned None for non-SSE requests, so non-streaming completions (stream:false) were invisible. Create an llm_call audit record for non-streaming LLM API calls too (matched by path: /chat/completions, /v1/messages, /v1/completions, /api/v1/copilot/generate_copilot), and set is_sse from the record instead of hardcoding true. Non-streaming token extraction is handled separately by alibaba#789; merge that first so these records carry token counts.
jfeng18
force-pushed
the
fix/sight-per-call-token
branch
from
cae6cdb to
baf9f7f
Compare
chengshuyi
approved these changes
Jun 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Make non-streaming LLM calls visible in
audit --type llm. Previouslyanalyze_httpreturnedNonefor non-SSE requests, so non-streaming completions (stream: false) never produced an audit record.What changed
analyzer/audit/analyzer.rsanalyze_httpalso emits anllm_callfor non-streaming LLM API calls (matched by path:/chat/completions,/v1/messages,/v1/completions,/api/v1/copilot/generate_copilot);is_ssenow reflects the record instead of being hardcodedtrue; +3 unit testsScope (narrowed)
This PR was narrowed to
analyzer/audit/analyzer.rsonly. The earlier version also touchedanalyzer/unified.rsto extract non-streaming token usage, which overlaps #789 (the comprehensive non-streaming + H2 token extraction). That part was dropped so the two PRs do not conflict:Merge order: #789 first. Until it lands, non-streaming calls appear in audit with 0 tokens; they get real counts once #789 is merged.
Verification (ECS, rustc 1.89 = CI toolchain)
cargo build --release,cargo fmt --all --check,cargo clippy --all-targets -- -D warnings: PASSorigin/main: 95%Note: issue link TBD — previously referenced #782, which is unrelated (that is the config-watcher
process::exitbug, fixed by #783).