Hello authors,
Thank you for this interesting and motivating work. I really appreciate your contribution to this field.
While reproducing your results, I noticed that some dataset-specific hyperparameters differ across trainer configurations, and I would like to better understand the intuition behind these choices.
In particular:
Temperatures used in the few-shot ProGrad trainer configs:
| Dataset |
Temperature |
| BTMRI |
2.0 |
| BUSI |
0.25 |
| DermaMNIST |
4.0 |
| Other datasets |
1.0 |
MSE loss weights used in the few-shot KgCoOp trainer configs:
| Dataset |
MSE weight |
| BTMRI |
2.0 |
| BUSI |
10.0 |
| CTKidney |
8.0 |
| DermaMNIST |
8.0 |
| KneeXray |
4.0 |
| Kvasir |
6.0 |
| LungColon |
8.0 |
| Other datasets |
1.0 |
In addition, I noticed that the MSE loss weight used for the base-to-novel task in KgCoOp is set to 8.0, which differs from the value 1.0 reported in the KgCoOp configuration from the original paper. I was wondering whether this change was made intentionally for this codebase or task setting.
More generally, I would like to ask:
- What was the underlying intuition or criterion for choosing these dataset-specific temperature and loss-weight values?
- Were they tuned empirically (e.g., via validation performance), or do they reflect dataset characteristics such as modality, scale, or class imbalance?
- Would you expect these values to generalize to new datasets, or should they be re-tuned during reproduction?
Clarifying this would greatly help with faithful reproduction and extension of your work.
Thanks again for your great paper, it has been very motivating to study and reproduce. I look forward to your response.
Hello authors,
Thank you for this interesting and motivating work. I really appreciate your contribution to this field.
While reproducing your results, I noticed that some dataset-specific hyperparameters differ across trainer configurations, and I would like to better understand the intuition behind these choices.
In particular:
Temperatures used in the few-shot ProGrad trainer configs:
MSE loss weights used in the few-shot KgCoOp trainer configs:
In addition, I noticed that the MSE loss weight used for the base-to-novel task in KgCoOp is set to 8.0, which differs from the value 1.0 reported in the KgCoOp configuration from the original paper. I was wondering whether this change was made intentionally for this codebase or task setting.
More generally, I would like to ask:
Clarifying this would greatly help with faithful reproduction and extension of your work.
Thanks again for your great paper, it has been very motivating to study and reproduce. I look forward to your response.