Skip to content

[BUG] thinking section not parsed for gpt-oss 120b #27

Description

@avidwriter

OS

Linux

GPU Library

CUDA

YALS commit sha

34e09fb

Describe the bug

here is a sample output for gpt-oss 120b, it looks like it's not parsing the thinking section.

<|channel|>analysis<|message|>The user just says "a joke". They want a joke. We should respond with a joke. Could be any style. Probably a short, witty joke. Could ask what kind, but they just said "a joke". So deliver a joke. Maybe a classic. Also be mindful of being appropriate. Provide a light-hearted joke. Let's give a short joke.<|end|><|start|>assistant<|channel|>final<|message|>Why don’t scientists trust atoms anymore?

Because they make up everything! 😄

Reproduction steps

using openwebui frontend
YALS backend
prompt: tell a joke

Expected behavior

should have a reasoning block before the answer

Logs

No response

Additional context

I think llama.cpp fixed this, might just need to update the deps.

Thanks!

Acknowledgements

  • I have looked for similar issues before submitting this one.
  • I have read the disclaimer, and this issue is related to a code bug. If I have a question, I will use the Discord server.
  • I understand that the developers have lives and my issue will be answered when possible.
  • I understand the developers of this program are human, and I will ask my questions politely.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions