Skip to content

Clarification on dataset-specific hyperparameters for KgCoOp and ProGrad #12

Description

@lamlethanh777

Hello authors,
Thank you for this interesting and motivating work. I really appreciate your contribution to this field.

While reproducing your results, I noticed that some dataset-specific hyperparameters differ across trainer configurations, and I would like to better understand the intuition behind these choices.

In particular:

Temperatures used in the few-shot ProGrad trainer configs:

Dataset Temperature
BTMRI 2.0
BUSI 0.25
DermaMNIST 4.0
Other datasets 1.0

MSE loss weights used in the few-shot KgCoOp trainer configs:

Dataset MSE weight
BTMRI 2.0
BUSI 10.0
CTKidney 8.0
DermaMNIST 8.0
KneeXray 4.0
Kvasir 6.0
LungColon 8.0
Other datasets 1.0

In addition, I noticed that the MSE loss weight used for the base-to-novel task in KgCoOp is set to 8.0, which differs from the value 1.0 reported in the KgCoOp configuration from the original paper. I was wondering whether this change was made intentionally for this codebase or task setting.

More generally, I would like to ask:

  • What was the underlying intuition or criterion for choosing these dataset-specific temperature and loss-weight values?
  • Were they tuned empirically (e.g., via validation performance), or do they reflect dataset characteristics such as modality, scale, or class imbalance?
  • Would you expect these values to generalize to new datasets, or should they be re-tuned during reproduction?

Clarifying this would greatly help with faithful reproduction and extension of your work.

Thanks again for your great paper, it has been very motivating to study and reproduce. I look forward to your response.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions