Regarding the operation issue of spaceinvaders_dqn_config_multi_gpu_ddp.py

[zonewenke]

Hello, community：

• The running environment is capable of running the file spaceinvaders_dqn_config_multi_gpu_dp.py, and I am using the single-machine multi-GPU mode.

  • When running the ddp file, I also wish to use the single-machine multi-GPU mode, but encountered an error. The command I used for running is: ding -m serial -c spaceinvaders_dqn_config_multi_gpu_ddp.py -s 0.

  • Traceback (most recent call last):
      File "/root/miniconda3/envs/di-engine/bin/ding", line 8, in <module>
        sys.exit(cli())
      File "/root/miniconda3/envs/di-engine/lib/python3.8/site-packages/click/core.py", line 1157, in __call__
        return self.main(*args, **kwargs)
      File "/root/miniconda3/envs/di-engine/lib/python3.8/site-packages/click/core.py", line 1078, in main
        rv = self.invoke(ctx)
      File "/root/miniconda3/envs/di-engine/lib/python3.8/site-packages/click/core.py", line 1434, in invoke
        return ctx.invoke(self.callback, **ctx.params)
      File "/root/miniconda3/envs/di-engine/lib/python3.8/site-packages/click/core.py", line 783, in invoke
        return __callback(*args, **kwargs)
      File "/root/miniconda3/envs/di-engine/lib/python3.8/site-packages/ding/entry/cli.py", line 272, in cli
        run_single_pipeline(seed[0], config)
      File "/root/miniconda3/envs/di-engine/lib/python3.8/site-packages/ding/entry/cli.py", line 193, in run_single_pipeline
        serial_pipeline(config, seed, max_train_iter=train_iter, max_env_step=env_step)
      File "/root/miniconda3/envs/di-engine/lib/python3.8/site-packages/ding/entry/serial_entry.py", line 61, in serial_pipeline
        policy = create_policy(cfg.policy, model=model, enable_field=['learn', 'collect', 'eval', 'command'])
      File "/root/miniconda3/envs/di-engine/lib/python3.8/site-packages/ding/policy/base_policy.py", line 844, in create_policy
        return POLICY_REGISTRY.build(cfg.type, cfg=cfg, **kwargs)
      File "/root/miniconda3/envs/di-engine/lib/python3.8/site-packages/ding/utils/registry.py", line 144, in build
        raise e
      File "/root/miniconda3/envs/di-engine/lib/python3.8/site-packages/ding/utils/registry.py", line 129, in build
        return build_fn(*obj_args, **obj_kwargs)
      File "/root/miniconda3/envs/di-engine/lib/python3.8/site-packages/ding/policy/base_policy.py", line 146, in __init__
        self._init_multi_gpu_setting(model, bp_update_sync)
      File "/root/miniconda3/envs/di-engine/lib/python3.8/site-packages/ding/policy/base_policy.py", line 176, in _init_multi_gpu_setting
        broadcast(param.data, 0)
      File "/root/miniconda3/envs/di-engine/lib/python3.8/site-packages/torch/distributed/c10d_logger.py", line 75, in wrapper
        return func(*args, **kwargs)
      File "/root/miniconda3/envs/di-engine/lib/python3.8/site-packages/torch/distributed/distributed_c10d.py", line 2135, in broadcast
        default_pg = _get_default_group()
      File "/root/miniconda3/envs/di-engine/lib/python3.8/site-packages/torch/distributed/distributed_c10d.py", line 1008, in _get_default_group
        raise ValueError(
    ValueError: Default process group has not been initialized, please make sure to call init_process_group.

• Generally, to set up a single machine with multiple cards, you only need to add the following line of code with DDP:

  torch.distributed.init_process_group()
  local_rank = int(os.environ.get("LOCAL_RANK")) 

• However, in the relevant source code of DDP: init_process_group() is placed within this DDPContext, and the rank is indeed set, but it seems to have no effect.

• Therefore, I hope to seek assistance in order to smoothly execute the contents of the file named 'spaceinvaders_dqn_config_multi_gpu_ddp.py'. If any friend has relevant experience with this, please do share with me. I am very grateful!

[PaParaZz1]

You should use the torchrun command to enable multiple worker, here is the related doc in PyTorch.