Inconsistent results with respect to publication results when trying to re-train the GoMatch BVs model from scratch

[mooncos]

Hello dear authors,

As part of a master thesis work I am exploring the capabilities of your work on geometric matching and in the process of results replication I have encountered discrepancies among the results published on Is Geometry Enough for Matching in Visual Localization? (by using the published pre-trained weights GoMatchBVs_best.ckpt) and the ones I obtain from benchmarking the results from the retraining of the architecture by using the training command provided in the README. I would like to know if the published and exported models differ in any way in parameters, or training configurations from the ones generated via the provided training command.

For instance the results, evaluated in the 7Scenes dataset as per the publications results (Table 4) are as follows:
Median Pose Error (m, ∘) (↓)

Method	Chess	Fire	Heads	Office	Pumpkin	Kitchen	Stairs
BPnPNet [11](SIFT [45])	1.29/43.82	1.48/51.82	0.93/55.13	2.61/59.06	2.15/39.85	2.15/43.00	2.98/60.27
BPnPNet (SP [22])	1.25/43.9	1.42/45.09	0.8/50.05	2.33/14.54	1.71/31.81	1.68/33.91	2.1/55.78
GoMatch (SIFT)	0.04/1.65	0.13/3.86	0.09/5.17	0.11/2.48	0.16/3.32	0.13/2.84	0.89/21.12
GoMatch (SP)	0.04/1.56	0.12/3.71	0.05/3.43	0.07/1.76	0.28/5.65	0.14/3.03	0.58/13.12

However after training with the following command (specified as the command used to obtain the results):

python -m gomatch_train.train_matcher --gpus 0 --batch 16 -lr 0.001 \
    --max_epochs 50 --matcher_class 'OTMatcherCls' --share_kp2d_enc \
    --dataset 'megadepth' --train_split 'train' --val_split 'val' \
    --outlier_rate 0.5 0.5  --topk 1 --npts 100 1024 \
    --p2d_type 'sift' --p3d_type 'bvs' \
    --inls2d_thres 0.001 --rpthres 0.01 --prefix 'gomatchbvs' \
    -o 'outputs/eccv22' --num_workers 4

I obtain the following results:
Median Pose Error (m, ∘) (↓), Method: Original GoMatch BVs (SIFT)

Scene	chess	fire	heads	pumpkin	redkitchen	stairs
t error median (m)	0.576249	0.933911	0.855726	1.740582	1.724768	2.920737
R error median (deg)	19.437645	26.878000	44.161449	28.971685	31.764105	49.433578

I attach the resulting best model from training and the output folder from training. I have replicated the environment in Conda and considered the same library versions that were used at the time of development of the published research.

Attachments:
GoMatchBVs_replication_best.ckpt.tar.gz
benchmark_cache_results.tar.gz
eccv22_train_outputs.tar.gz

Best regards /Marcos