Skip to content

feat(parsing): add support for Qwen tool call tokens and AST parsing#5546

Open
CodeRulerNo1 wants to merge 1 commit into
odysseus-dev:devfrom
CodeRulerNo1:feat/qwen-tool-call-parser
Open

feat(parsing): add support for Qwen tool call tokens and AST parsing#5546
CodeRulerNo1 wants to merge 1 commit into
odysseus-dev:devfrom
CodeRulerNo1:feat/qwen-tool-call-parser

Conversation

@CodeRulerNo1

Copy link
Copy Markdown

Summary

When Qwen models emit tool calls in their native token format:
<|tool_call_start|>[tool_name(args)]<|tool_call_end|>

They were not parsed or executed by src/tool_parsing.py, causing them to fail and leak into user-facing assistant messages as raw text.

This PR fixes the issue by:

  1. Adding a new _QWEN_TOOL_CALL_RE regex pattern matching the start/end tags and parsing the function name + arguments.
  2. Implementing _parse_qwen_tool_call which extracts argument keys/values using safe Python AST parsing (wrapping arguments in a dummy function call to evaluate keyword arguments) and falling back to a regex parser on syntax errors.
  3. Adding the Qwen pattern matchers into parse_tool_blocks and strip_tool_blocks so the call executes correctly and the raw tags/tokens are cleaned from user-facing text.
  4. Adding a suite of unit tests (tests/test_qwen_tool_call.py) covering empty arguments, multiple arguments, type coercion, and syntax recovery.

Target branch

  • This PR targets dev, not main. All PRs land in dev; main is curated by the maintainer at each release. If your PR is on main by accident, click "Edit" on this PR and change the base.

Linked Issue

Fixes #5545

Type of Change

  • Bug fix (non-breaking — fixes a confirmed issue)
  • New feature (non-breaking — adds new behaviour)
  • Breaking change (changes or removes existing behaviour)
  • Refactor / cleanup (behaviour unchanged)
  • Documentation only
  • CI / tooling / configuration

Checklist

  • I searched open issues and open PRs — this is not a duplicate.
  • This PR targets dev
  • My changes are limited to the scope described above — no unrelated refactors or whitespace changes mixed in.
  • I actually ran the app (docker compose up or uvicorn app:app) and verified the change works end-to-end. Type-checks and unit tests are not enough.

How to Test

  1. Run the new unit tests in the Odysseus environment:
    venv/bin/pytest tests/test_qwen_tool_call.py
  2. Ensure both tests pass, demonstrating correct parsing of list_email_accounts() and list_emails(account="Gmail", ...) as well as clean stripping of the tokens.

Visual / UI changes — REQUIRED if you touched anything that renders

Anything that changes what the UI looks like — buttons, icons, padding, colors, fonts, spacing, layout, CSS, HTML, SVG, or any static/js/ module that draws to the DOM — needs all of the following. PRs that change rendering without these WILL be closed.

  • Screenshot or short clip of the change in the running app, attached below. Mobile screenshot too if the change affects mobile.
  • Style match: the change uses Odysseus's existing visual language. Specifically:
    • Reuse existing CSS variables (--red, --fg, --bg, --card, --border, etc.) — do not introduce new color values, font sizes, or spacing units.
    • Reuse existing button/input/card/border classes. Don't invent parallel styling.
    • No Unicode emoji in UI or code. Use inline SVG (matching the monochrome icon style already in static/index.html) or plain text.
    • Monospaced font (Fira Code) for primary UI text. Don't override.
    • Dark theme is the default; any light-mode work must be wired through the existing theme system, not hard-coded.
  • No new component patterns. If a similar widget already exists in the app, extend it instead of writing a parallel one.
  • I am not an LLM agent submitting a bulk PR. If you are, please open an issue describing the problem first — bulk auto-generated PRs that don't match the project's visual style are closed on sight, even when the underlying fix is correct.

Screenshots / clips

None (pure token parsing backend utility, UI layout remains unchanged).

@github-actions github-actions Bot added the ready for review Description complete — ready for maintainer review label Jul 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready for review Description complete — ready for maintainer review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: Qwen-style token tool calls are not parsed or executed

1 participant