Skip to content

[Transformers] Declare DSV4 IndexCache config fields explicitly - #4997

Open
chenbong wants to merge 1 commit into
PaddlePaddle:developfrom
chenbong:feature/dsv4-indexcache-config-fields
Open

[Transformers] Declare DSV4 IndexCache config fields explicitly#4997
chenbong wants to merge 1 commit into
PaddlePaddle:developfrom
chenbong:feature/dsv4-indexcache-config-fields

Conversation

@chenbong

@chenbong chenbong commented Sep 9, 2026

Copy link
Copy Markdown

PR Category

Transformers / Model Config

PR Types

New features

Description

index_topk_pattern and indexcache_multi_layer_distill configure DSV4
IndexCache. The F/S pattern spans the ratio=4 CSA layers, one character per C4
layer: F runs that layer's own learned Indexer, S reuses the nearest
preceding F layer's top-k. The distill flag additionally builds a
served-layer distillation target against the producer's top-k probabilities.

Both already reach the model today, but only via PretrainedConfig's generic
**kwargs passthrough. That has three costs:

  • DeepseekV4Config() carries no such attribute, so every consumer has to use
    getattr(config, "index_topk_pattern", None) rather than reading a field.
  • Neither field is documented, so the accepted values (only F/S, must start
    with F, length equal to the C4 layer count) are not discoverable from the
    config class.
  • Nothing pins the defaults, so "IndexCache disabled" is an absence rather than
    a declared None / False.

This PR declares both as real fields with documented defaults, next to the
existing CSA group.

Compatibility

No behaviour change. Configs that set either field explicitly resolve to exactly
the same values as before; the only difference is that a config which omits them
now reports the documented defaults instead of raising AttributeError on
attribute access.

Validation

  • A bare DeepseekV4Config() exposes index_topk_pattern=None and
    indexcache_multi_layer_distill=False, and both appear in __dict__.
  • Explicit values (index_topk_pattern="FSSF",
    indexcache_multi_layer_distill=True) arrive unchanged.
  • Both survive a to_dict() / from_dict() round trip and are present in
    to_dict() output.
  • black, isort, flake8 and copyright_checker pass on the touched file.

`index_topk_pattern` and `indexcache_multi_layer_distill` configure DSV4
IndexCache: an F/S pattern over the ratio=4 CSA layers picks which layers run
their own learned Indexer (`F`) and which reuse the nearest preceding producer's
top-k (`S`), and the distill flag adds a served-layer distillation target.

They already reach the model today, but only through `PretrainedConfig`'s
generic `**kwargs` passthrough, which means:

* `DeepseekV4Config()` has no such attribute at all, so every consumer must go
  through `getattr(config, "index_topk_pattern", None)` instead of reading the
  field;
* neither field is documented, so the accepted values (F/S only, must start with
  `F`, one character per C4 layer) are not discoverable from the config;
* nothing pins the defaults, so "IndexCache off" is an absence rather than a
  declared `None` / `False`.

Declare both as real fields with documented defaults. Behaviour is unchanged for
configs that set them explicitly.

Verified against this change: a bare `DeepseekV4Config()` now exposes
`index_topk_pattern=None` and `indexcache_multi_layer_distill=False`; explicit
values still arrive unchanged; and both survive a `to_dict()` / `from_dict()`
round trip and appear in `to_dict()` output. black, isort, flake8 and
copyright_checker pass on the touched file.

@Paddle-Bot Paddle-Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Paddle-Bot Review Board (review完成)

序号 位置 优先级 规则来源 状态
1 构造函数参数顺序 P1 仓库规则:基础评审规则 🚧
Powered by Nyanpasu with gpt-5.6-sol 默认推理级别, please check the suggestions carefully.

csa_compress_rotary_base=160000.0,
csa_dense_mode=False,
# === IndexCache (CSA F/S top-k reuse) ===
index_topk_pattern=None,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 此处把两个新参数插入已有 dsa_index_n_heads 之前,会改变 DeepseekV4Config 的 positional 参数顺序:旧代码的第 29、30 个实参原本配置 DSA heads/head_dim,现在会被解释为 pattern/distill,并使后续配置整体错位,静默产生错误模型配置。请将新参数追加到原签名末尾(**kwargs 前)或采用不改变旧参数位置的兼容方式,并补充 positional 兼容测试。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants