General Questions

[kleenkanteen]

1. At this point in time, what is the current goal of the project?

2. For each topic, where are we pulling data from?

3. I think we should first focus on financial questions and pull in stock data and research articles. It will be easier to work towards a specific topic and gather a dataset just on that than the 20 different ones currently listed on https://huggingface.co/spaces/Tonic/e5. What do you guys think?

[Josephrp]

none of the "20 different ones" are datasets.

the rest is up to us !

[kleenkanteen]

I mean datasets about the 20 different prompts. What I am undestanding is that we have to gather a corpus of text on each topic that each prompt asks about. We finetune that embedding on that corpus of text. So that means we have to gather 20 copuses of text on 20 different topics. This sounds cumbersome. What do you guys think abut just gathering a corpus on one topic? Quality over quantity.

[Josephrp]

that is incorrect as that is handled by document loaders and agents

[kleenkanteen]

Can you elaborate? I see the source of data related to each prompt is a matter of discussion: #10

[kleenkanteen]

@Josephrp