Switching to a multi-node setup

[ashok-arora]

Hey,
Thank you for the code. I'm trying to run it on my university's cluster, but the GPUs have memory limits, so I was thinking of splitting the workload across multiple GPUs. Would it take a lot of effort to convert this code to a multi-node setup?

[kevslinger]

Hey Ashok, what are the limits? If you are trying to test in domains where the observations are images, you may need a lot of RAM. But otherwise, for simpler domains, I think very simple (4GB or 8GB) GPUs should be fine

[ashok-arora]

Even though the cluster has V100 GPUs, training currently takes 15–16 hours. Since I need to run multiple seeds and there's a limit on concurrent jobs, I'm wondering if it's possible to parallelize the training across multiple GPUs to reduce the time to 7–8 hours, then I can run more jobs on the cluster.

[kevslinger]

Ah, I understand. Here's a couple things I tried in uni to try to be more efficient with our cluster:

• Run multiple Python processes in parallel within a single cluster "job". This works if you aren't hitting memory constraints of the GPUs/overall RAM.
• If you're hyperparameter tuning, try reducing the total training steps, save the checkpoints, and then analyze the performance of the agent so far. Proceed with the promising (or top X most promising) hyperparameters.

At my uni, we were allowed to request up to X GPUs, so running 1 job on 4 GPUs or 4 jobs each on 1 GPU would both hit the same limit. That's why I never tried a multi-GPU setup. I'm sure it's possible to do, but I think it might alter performance, depending on how you set it up

[ashok-arora]

Thank you for the suggestions. They were helpful. I tried running processes in parallel, but that was hitting the memory limit and caused the jobs to take more than 24 hours instead of the earlier 16. Another approach I am trying is to run multiple processes across multiple GPUs within the same job itself because there is a limit only on the total number of jobs