Reproducibility issues

[carlavintila]

Hello! I have been trying to benchmark RIPOR on another dataset, however I have encountered multiple issues. First of all, the pre-trained model cannot be used, to my knowledge, on other datasets than MSMARCO due to the fact that many of the model's scripts are hardcoded to use the MSMARCO document collection and queries. Training the model from scratch on another dataset has also proved infeasible, due to many of the bash scripts having hardcoded file paths that were not provided in the RIPOR_data Google drive. The README.md file does not contain anything other than surface-level instructions, and without a proper understanding of the internal code/file structure, it appears impossible to reproduce the experiment.

[HansiZeng]

Thanks for reporting this! To help me understand the issue more clearly, could you provide a bit more detail?

Seems like you have two issues:

1. Cannot run the pre-trained models.
2. Cannot train the models from scratch.

Let’s focus on the first issue first. Could you share the script you’re using, the data (including its structure), and the exact error messages you encounter? With those, I can help you debug the problem.

Specifically, it would be very helpful if you could share:

1. What did you try

Please include the exact command or script you used when the issue occurred.

2. The data format

I don’t need your actual dataset — just the structure.
For example:

Is it a .tsv, .json, or .jsonl file?

For .jsonl, what does one example look like (e.g., what fields/keys does it contain)?

3. The full error message

Please paste the complete error log or stack trace so I can reproduce the issue on my side.