What
For more sophisticated benchmark experiments, there is a need to understand how Hydra actually operates multi-runs.
(Whether it finishes each trial and run the next one in a row or run several trials (through an override method or something) and then return all the results at once, etc. )
Why
Currently, there are several problems arising form the lack of understanding of the multi-run operation of Hydra.
For instance,
- Custom logging is being recorded repeatedly as many times as the number of hydra multi-runs.
- It is unclear how to weave end-to-end pipelines which executes train->inference at once, in the multi-run experiment.
How
What
For more sophisticated benchmark experiments, there is a need to understand how Hydra actually operates multi-runs.
(Whether it finishes each trial and run the next one in a row or run several trials (through an override method or something) and then return all the results at once, etc. )
Why
Currently, there are several problems arising form the lack of understanding of the multi-run operation of Hydra.
For instance,
How