Question about reproducing the project results

[kangzicheng11]

Hello, thank you very much for open-sourcing the CAMELTrack code. I have followed the instructions in the repository and completed the full reproduction pipeline, but my results still show a noticeable gap compared to those reported in the paper. I would like to ask for your guidance on possible reasons.

I am using the DanceTrack dataset and followed the README with the following steps:

1. Generate states and run tracking on train / val / test:

uv run tracklab -cn cameltrack dataset=dancetrack dataset.eval_set=train
uv run tracklab -cn cameltrack dataset=dancetrack dataset.eval_set=val
uv run tracklab -cn cameltrack dataset=dancetrack dataset.eval_set=test

2. Train the association model using the generated states:

uv run tracklab -cn cameltrack_train dataset=dancetrack max_epoch=10

The training converges normally (train loss ~0.11, val sim_auroc ~0.99), and the validation threshold search is successfully completed (~0.558).

3. Evaluation on the validation set (tracking + TrackEval):

HOTA: 61.9

AssA: 48.6

DetA: 79.3

IDF1: 64.2

MOTA: 88.6

Compared to the reported CAMELTrack results on the DanceTrack test set:

HOTA: 66.1

AssA: 54.0

IDF1: 71.1

MOTA: 91.4

there is still a noticeable gap, especially in association-related metrics (AssA and IDF1), with a drop of about 5–7 points.

I would like to ask a few questions:

Were the reported results trained with more epochs (e.g., >10) or a specific training schedule?

Are there any additional hyperparameters (e.g., sim_threshold, track buffer, matching strategy) that are not reflected in the default configuration?

Is it necessary to use the keypoints version (w/ keypoints) to reach the best performance reported in the paper?

Any suggestions or recommended configurations for reproducing the paper results would be greatly appreciated.

Thank you again for your work and for making it available!

[VlSomers]

Hi @kangzicheng11 , did you train with the keypoints model or only the bbox model? You need keypoints to reach the best performance, as you can see in the paper ablation studies. Moreover, validation and test sets on DanceTrack have different performance, you should rather look at the validation set performance reported in the Table 5 of the paper (Ablation Studies) to find the right numbers to compare to.

[VlSomers]

@kangzicheng11 did you solve your problem?