fix: preserve the auto-capture cursor after successful history-flow extractions#932
Open
gorkem2020 wants to merge 1 commit into
Open
fix: preserve the auto-capture cursor after successful history-flow extractions#932gorkem2020 wants to merge 1 commit into
gorkem2020 wants to merge 1 commit into
Conversation
|
After #919 was merged, this PR now has merge conflicts with the latest |
…ction Two auto-capture flows share the autoCaptureSeenTextCount counter. In the ingress flow (message_received feeds pendingIngressTexts and each agent_end carries only the newest message) the counter is a pure accumulator of new texts toward extractMinMessages, and resetting it to 0 after a successful extraction is the intended windowing from issue message history each turn) the same counter is also the slice cursor into that history; resetting it to 0 made the very next capture re-read and re-extract the entire session: repeated LLM cost and repeated admission and dedup evaluations over already-extracted content. Make the post-success reset flow-aware: ingress-fed turns keep the reset to 0 (the existing counter-reset scenario in test/smart-extractor-branches.mjs still passes unchanged), while history-carrying turns record the consumed history length so the next capture only sees the delta. Red-proved: the new regression test fails against the previous code with turn 1 content re-read in turn 2, and passes after the change. Wire the suite into the local npm test chain and the core-regression CI group.
gorkem2020
force-pushed
the
fix/autocapture-watermark-reset
branch
from
67ef600 to
137ff6e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
After every successful auto-capture write, the per-session cursor is reset to zero (
autoCaptureSeenTextCount.set(sessionKey, 0)). In history-carrying sessions, whereagent_enddelivers the whole session each turn, the cursor doubles as the slice offset, so the turn after any successful extraction re-reads the entire session history as new. Observed live: a later extraction's input re-contained turns already extracted minutes earlier. Costs grow with session length, and re-rolling the same content through extraction and admission multiplies the chances of a low-quality item eventually slipping through.The reset is not simply a bug: the commit trail and a pinning test show it is intentional for ingress-fed sessions (per-message channels feeding
pendingIngressTexts), where the counter acts as a pure new-text accumulator forextractMinMessageswindowing and a length-based reset would wedge the window closed.Fix
Flow-aware reset:
pendingIngressTexts.length > 0 ? 0 : eligibleTexts.length. Ingress sessions keep the accumulator semantics; history sessions record the consumed length so subsequent turns slice only the delta.Tests
Red-proven regression (turn 2 re-read turn 1's content before the fix; only the delta after) plus the pre-existing ingress counter-reset scenario passing unchanged.
Verification on a live gateway
A four-turn session produced extraction inputs containing each turn exactly once; the previous whole-history re-read signature is gone.