Hi,
congratulations on your paper.
I was able to reproduce your results on the 4 folds of COCO-20i.
However I am encountering issues in reproducing the results from Tab4 of your manuscript, i.e. the COCO->Pascal domain shift. The table is meant to test on Pascal the model trained on COCO-20i.
I implemented the Pascal splits as described in the paper that first proposed the setting ( https://arxiv.org/pdf/2012.06166), where the ratio is that there should be no overlap between the training and testing classes (as in standard k-fold protocols). For this reason, in the COCO->Pascal setting the folds of the Pascal dataset need to be repurposed to avoid overlap with COCO20i folds.
Despite reproducing your results on COCO, on COCO->Pascal I am getting much lower numbers (average 68 vs 75.9 in the paper)
Can I ask you to clarify which Pascal split did you use? I am able to get a number close to 75.9 by testing on the standard Pascal folds, which however would not be fair for the reasons outlined above, and since the other methods in the table have been tested with the proper setting
thanks
Hi,
congratulations on your paper.
I was able to reproduce your results on the 4 folds of COCO-20i.
However I am encountering issues in reproducing the results from Tab4 of your manuscript, i.e. the COCO->Pascal domain shift. The table is meant to test on Pascal the model trained on COCO-20i.
I implemented the Pascal splits as described in the paper that first proposed the setting ( https://arxiv.org/pdf/2012.06166), where the ratio is that there should be no overlap between the training and testing classes (as in standard k-fold protocols). For this reason, in the COCO->Pascal setting the folds of the Pascal dataset need to be repurposed to avoid overlap with COCO20i folds.
Despite reproducing your results on COCO, on COCO->Pascal I am getting much lower numbers (average 68 vs 75.9 in the paper)
Can I ask you to clarify which Pascal split did you use? I am able to get a number close to 75.9 by testing on the standard Pascal folds, which however would not be fair for the reasons outlined above, and since the other methods in the table have been tested with the proper setting
thanks