Hello, Thanks for your released codes of your experiments done. I enjoyed the idea.
However, with the code given, it was hard to reproduce the number reported in your paper(ColBERT & MonoT5-Base[Appendix Table 6] Results, especially for "arguana")
Is it possible to get your trained data generated for the experiments?
Also, I'm curious is it possible to release other few-shot instructions for query generation of datasets other than FiQA).
Also, I have additional question about the train. Since the train data is very small(4k ~ 20k datapoints, depends on K,N parameter of document sampling), It seems that scores fluctuates in a range even if I retrain with the same train data. What was your policy to tackle this(or was it not the case you've been through?
Thanks in advance :D
Hello, Thanks for your released codes of your experiments done. I enjoyed the idea.
However, with the code given, it was hard to reproduce the number reported in your paper(ColBERT & MonoT5-Base[Appendix Table 6] Results, especially for "arguana")
Is it possible to get your trained data generated for the experiments?
Also, I'm curious is it possible to release other few-shot instructions for query generation of datasets other than FiQA).
Also, I have additional question about the train. Since the train data is very small(4k ~ 20k datapoints, depends on K,N parameter of document sampling), It seems that scores fluctuates in a range even if I retrain with the same train data. What was your policy to tackle this(or was it not the case you've been through?
Thanks in advance :D