Clarification on LSDA ckpt_best.pth selection during cross-dataset evaluation

[haibaraemily-G]

Hi, thank you for releasing DeepfakeBench.

I am reproducing the LSDA baseline using the official DeepfakeBench code.

I noticed that during training, if I set:
--train_dataset FaceForensics++
--test_dataset Celeb-DF-v2
metric_scoring: auc
and do not use --no-save_ckpt,

the framework saves ckpt_best.pth according to the best AUC on Celeb-DF-v2 during training.

I would like to confirm the intended evaluation protocol:

1. For cross-dataset evaluation, is ckpt_best.pth selected according to the monitored target test dataset, e.g., Celeb-DF-v2?

2. Or should the checkpoint be selected only using the validation split of the source training dataset, e.g., FF++ validation, and then evaluated once on Celeb-DF-v2 / DFDC / DFDCP?

3. For the LSDA results reported in the paper / DeepfakeBench benchmark, which checkpoint selection strategy was used?

This clarification would help me ensure that my reproduction follows the official protocol.

Thank you!