[DISCUSSION] Parallelism in Batch Writing

[andyafter]

I am creating this issue because I noticed that on the default settings:

• Batch Size as 1000

When I was training the MCCFR for the headsup game I found:

My CPU utilizations hangs at a relatively low percentage.

After some investigations, I figured that during the calculation, all of my 192 vcpus are actually spending most of the times waiting. Waiting on what you would ask.

So after some investigation, I figured that when the batch task is calculated, after the batch is completed, 1000 cfr tree updates are written into the strategy profile sequencially, and the CPUs for calculating batch tasks are simply waiting for the writing to finish so that it can start the next task.

So here there is a massive optimization spot where we can ignore what data structure to use for the profile, but simply think about "Can we make this writing process parallel on the side".

In the current set up, for each bacth i, we have tasks [t_1, t_2 ...t_1000] which is calculating sampled CFR trees based on profile P_i

Now if we write it parallelly, some of the tasks might be using profile P_i, and some of the tasks in batch i will be using profile P_(i+1).

After I went through the current code base, I figured that each task is only calculating the strategy of ONE Node and it's edges pointing to its children. Which means that even if in a batch "some of the tasks might be using profile P_i, and some of the tasks in batch i will be using profile P_(i+1). ", we do not need to worry about whether it affects the training results, because:

1. Throttle for calculation of tasks based on P_i and P_(i+1) will be the same
2. There will not be any case where "updated profile in a batch will make CFR sampling and computation less accurate".

If we optimize this, at least on AWS HPC instances, we can achieve a somewhat 3~5 times up for training.