Parallelize frame processing on whole-video analysis

[shssoichiro]

On long videos, analysis of some of the metrics may take a while. Since each frame does not depend on any previous or future frames, we can analyze frames in parallel to speed up whole-video analysis.

[shssoichiro]

I've done some work on this but none of it has been usable, but I can report the roadblocks. Where I've run into issues is that a decoder can't be sent across threads, so the simple way of throwing it in a rayon iterator didn't work. In other words, the decoding has to be done serially, but we should be able to create a threadpool and queue decoded frames into it for processing.

[sathwikmatsa]

Hey there! I would like to take this one.

I'd like to have some clarity on few things :
I've gone through the code and I assume you want to parallelize this section of code
https://github.com/rust-av/av-metrics/blob/460b1c4ce7a65bf027fbdeda31d4c8842bc4a6c1/av_metrics/src/video/mod.rs#L215-#L221

More specifically, you'd like to execute self.process_frame(..,..) on different pairs of frames concurrently.

Since, process_frame takes mutable reference of self, is it feasible to share that unique reference among threads?

[shssoichiro]

Yes, you are correct about what I'd like to parallelize. And yes, the mutable reference to self has been an issue when I've attempted to implement it. Ideally, I'd hope this can be implemented without having to change the public API, but if it needs to be changed, then that's okay as long as the changes are user-friendly.

[lu-zero]

The easiest way is to spawn the actual process in a controller thread and just have the two ends of the channels in the outer context.