fix(tool_parsing): Qwen bare-marker regex strips remove \nend\n #5548
Open
shuwudow wants to merge 1 commit into
Open
fix(tool_parsing): Qwen bare-marker regex strips remove \nend\n #5548shuwudow wants to merge 1 commit into
shuwudow wants to merge 1 commit into
Conversation
…\nend\n` disappear Chat messages that have end statements, for example in the context of coding languages, cut off the "\nend\n" part which is not seen in front-end. Changed similar function parsing lines in both tool_parsing.py and in chatRenderer.js. Fix: require the pipes (`|end|`) so only Qwen's actual turn marker matches. The `|end|` and `/|end|` cases still strip correctly, so Qwen artifact cleanup SHOULD be left unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Chat messages that have end statements, for example in the context of coding languages, cut off the
\nend\npart.The QWEN_BARE_MARKER_RE regex used optional pipe delimiters around
end(\|?end\|?), which matched the bare keywordendon its own line. Chat messages containing Ruby code that closes blocks with a loneend— had those lines stripped from the rendered message in the front-end. The\nend\npart disappeared both from sent and received messages.Fix: require the pipes (
|end|) so the pattern only matches Qwen's actual turn marker. The|end|and/|end|cases still strip correctly, so Qwen artifact cleanup is unchanged.Applied to the duplicate regex in both
src/tool_parsing.py(backend) andstatic/js/chatRenderer.js(frontend).Target branch
dev, notmain.Linked Issue
Fixes #5547
Linked to #5546 because both commits modify the same file (tool_parsing.py), however changed areas do not match.
Type of Change
Checklist
devdocker compose uporuvicorn app:app) and verified the change works end-to-end. Type-checks and unit tests are not enough.How to Test
docker compose up -d --buildendfragment shouldn't disappear anymore.Since this change affects qwen models, I must mention that I verified its originally intended stripping functionality only by AI agent, I rebuilt Docker image, agent ran runtime regex substitution tests (Python + JS) showing bare end is preserved while |end|, /|end|>, </|end|>, and bare assistant are still stripped identically to pre-fix.
I did not test live Qwen model end-to-end. Should be Qwen-compatible — real Qwen markers are wrapped in
|characters (like|end|), and those are still stripped correctly, unless I'm missing something.Visual / UI changes — REQUIRED if you touched anything that renders
Anything that changes what the UI looks like — buttons, icons, padding, colors, fonts, spacing, layout, CSS, HTML, SVG, or any
static/js/module that draws to the DOM — needs all of the following. PRs that change rendering without these WILL be closed.--red,--fg,--bg,--card,--border, etc.) — do not introduce new color values, font sizes, or spacing units.static/index.html) or plain text.Fira Code) for primary UI text. Don't override.Screenshots / clips
before:

after:
