Support for Off-Policy Actor-Critic Algorithms like ACER

[Riashat]

Hi,

Is there any support for the off-policy counterpart of A2C (ACER algorithm) that can be made based on this repo?

This is a very useful repo that we mostly use, and also nice to have its compatibility with gym minigrid tasks. However, there is no open source implementation of ACER that can be made compatible with gym minigrid yet.

Would be nice to have ACER support coming along with this repo?

There is a useful one I found, which works for ALE tasks, but no support for gym minigrid.
https://github.com/belepi93/pytorch-acer

[maximecb]

If it supports OpenAI Gym environments, then making it work with MiniGrid shouldn't be very difficult. You might just need to change the size of convolution layers and create a gym environment wrapper to have the input in the format you want.

[lcswillems]

Hi,

I don't know ACER. How does it differ from A2C?

It is unlikely I will have the it will not have time to implement this algorithm. But, maybe it can be easy for you. Do you want I give you more details on the code?

As Maxime has mentioned, it might also be easy to use for Minigrid by just doing in the repo you mention.