Repository navigation
feat: detect decision models via <arch>.decision.type - #63
Merged
Merged
Conversation
|
✅ OpenCodeReview: Review complete: 0 finding(s) across 5 selected item(s). |
There was a problem hiding this comment.
🟡 Changes recommended
Detection misses valid markers, malformed metadata can panic, and the live-model test bypasses the required opt-in behavior.
3 open issues
What changed in this PR
Adds decision-model identification to the GGUF library and CLI, supporting GPUStack’s decision-model serving work in #6436.
Changes:
- Parses
<architecture>.decision.typeand exposes decision flags in llama.cpp estimates and summaries. - Gives decision markers precedence over reranking heuristics and adds a CLI Decision column.
- Adds synthetic fixtures and a Laya model check, and documents decision-model screening.
| File | Description |
|---|---|
| README.md | Adds Decision to the supported model-screening types. |
| file_estimate__llamacpp.go | Adds decision classification and propagates it to summaries. |
| file_decision_test.go | Tests metadata, serialization, classification, and Laya detection. |
| file_architecture.go | Reads and exposes decision-type metadata. |
| cmd/gguf-parser/main.go | Displays decision support in estimate tables. |
Review effort: Balanced
orangedeng
force-pushed
the
add-decision-model-support
branch
from
October 23, 2026 08:01
d1aed73 to
3c76557
Compare
Parse the "<architecture>.decision.type" metadata the llama.cpp conversion tools write, mirroring common_get_decision_type: the presence of the key alone marks a decision model serving "POST /v1/systemone", whatever its value says. Expose it as GGUFArchitecture.DecisionType and a presence-based Decision flag on the llama.cpp estimates, plus a DECISION column in the CLI table. The detection is independent of the attention direction and outranks the reranking heuristics: general backbones such as modern-bert carry classifier tensors and rank pooling that would otherwise read as reranking. Also fix reading a zero-length string value: the pooled buffer grows a zero length to the default size, so the read swallowed the rest of the file and a legal empty value broke the parse. Cover both with offline fixture tests and the first-party Laya GGUF via TEST_MODEL_PATH. Signed-off-by: Yuxing Deng <15376837+orangedeng@users.noreply.github.com>
orangedeng
force-pushed
the
add-decision-model-support
branch
from
October 23, 2026 08:22
3c76557 to
fb11a48
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Refer to issue:
Parse the ".decision.type" metadata the llama.cpp conversion tools write, mirroring common_get_decision_type: the presence of the key alone marks a decision model serving "POST /v1/systemone", whatever its value says.
Expose it as GGUFArchitecture.DecisionType and a Decision flag on the llama.cpp estimates, plus a DECISION column in the CLI table. The declared marker outranks the reranking heuristics: general backbones such as modern-bert carry classifier tensors and rank pooling that would otherwise read as reranking.
Cover both with offline fixture tests and a first-party Laya GGUF from ModelScope.