About sampling strategies of test input patch

[dituu]

Hi, thanks for releasing the code. I have a question about the test setting on PU-GAN dataset.

In original PU-GAN dataset, the test input sparse patches are generated by random downsampling form the ground truth, while this work uses Poisson downsampling in prepare_pugan.py. The former generates a realistic and non-uniform distribution, while the Poisson downsampling produces a uniform distribution.

I directly test the released model on PU-GAN test set under random downsampling setting, and find the performance degenerates greatly (CD 0.245->0.495). It seems that the method is sensitive to the input distribution pattern.

So, do you have some ideas about this phenomenon? Many thanks!

[yunhe20]

1. As already stated in the "4.1 Experiment Setup" section, we follow PU-GCN to prepare the test set of PU-GAN dataset.
2. In genearal, the randomly sampled test set is more difficult than possion sampled test set, so it is normal that the metrics may drop (include other methods). Have you tested other methods on your test set? And how did they perform on your test set?

[dituu]

Thanks for your reply! I have tested the public PU-GAN and PU-GCN model on my prepared test set (random downsampling). The results are as follows,
CD || HD || P2F
PU-GAN 0.28 4.60 3.18
PU-GCN 0.30 4.31 2.78
It seems the CD errors are consistent with the results of possion sampling, while HD and P2F are degenerated.