Skip to content

feat: detect decision models via <arch>.decision.type - #63

Merged
thxCode merged 1 commit into
gpustack:mainfrom
orangedeng:add-decision-model-support
Oct 23, 2026
Merged

thxCode merged 1 commit into
gpustack:mainfrom
orangedeng:add-decision-model-support

Conversation

@orangedeng

Copy link
Copy Markdown
Contributor

Refer to issue:

Parse the ".decision.type" metadata the llama.cpp conversion tools write, mirroring common_get_decision_type: the presence of the key alone marks a decision model serving "POST /v1/systemone", whatever its value says.

Expose it as GGUFArchitecture.DecisionType and a Decision flag on the llama.cpp estimates, plus a DECISION column in the CLI table. The declared marker outranks the reranking heuristics: general backbones such as modern-bert carry classifier tensors and rank pooling that would otherwise read as reranking.

Cover both with offline fixture tests and a first-party Laya GGUF from ModelScope.

Copilot AI balanced review requested due to automatic review settings October 23, 2026 07:43
@gpustack-code-review

gpustack-code-review Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

✅ OpenCodeReview: Review complete: 0 finding(s) across 5 selected item(s).

Comment thread file_architecture.go Resolved
Comment thread README.md Resolved

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Detection misses valid markers, malformed metadata can panic, and the live-model test bypasses the required opt-in behavior.

3 open issues
What changed in this PR

Adds decision-model identification to the GGUF library and CLI, supporting GPUStack’s decision-model serving work in #6436.

Changes:

  • Parses <architecture>.decision.type and exposes decision flags in llama.cpp estimates and summaries.
  • Gives decision markers precedence over reranking heuristics and adds a CLI Decision column.
  • Adds synthetic fixtures and a Laya model check, and documents decision-model screening.
File Description
README.md Adds Decision to the supported model-screening types.
file_estimate__llamacpp.go Adds decision classification and propagates it to summaries.
file_decision_test.go Tests metadata, serialization, classification, and Laya detection.
file_architecture.go Reads and exposes decision-type metadata.
cmd/​gguf-parser/​main.go Displays decision support in estimate tables.

Review effort: Balanced

Comment thread file_architecture.go Resolved
Comment thread file_decision_test.go Resolved
Comment thread file_estimate__llamacpp.go Resolved
@orangedeng
orangedeng force-pushed the add-decision-model-support branch from d1aed73 to 3c76557 Compare October 23, 2026 08:01
Parse the "<architecture>.decision.type" metadata the llama.cpp
conversion tools write, mirroring common_get_decision_type: the
presence of the key alone marks a decision model serving
"POST /v1/systemone", whatever its value says.

Expose it as GGUFArchitecture.DecisionType and a presence-based
Decision flag on the llama.cpp estimates, plus a DECISION column in
the CLI table. The detection is independent of the attention
direction and outranks the reranking heuristics: general backbones
such as modern-bert carry classifier tensors and rank pooling that
would otherwise read as reranking.

Also fix reading a zero-length string value: the pooled buffer
grows a zero length to the default size, so the read swallowed the
rest of the file and a legal empty value broke the parse.

Cover both with offline fixture tests and the first-party Laya GGUF
via TEST_MODEL_PATH.

Signed-off-by: Yuxing Deng <15376837+orangedeng@users.noreply.github.com>
@orangedeng
orangedeng force-pushed the add-decision-model-support branch from 3c76557 to fb11a48 Compare October 23, 2026 08:22
@orangedeng
orangedeng requested a review from thxCode October 23, 2026 08:57

@thxCode thxCode left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@thxCode
thxCode merged commit 9257743 into gpustack:main Oct 23, 2026
3 checks passed
@orangedeng
orangedeng deleted the add-decision-model-support branch October 23, 2026 09:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants