Is there an effective method to prompt the fuuro(副露) rate of AI?

[Pragmatism0220]

I want to try to train an AI like ASAPIN who is more inclined to fuuro. I tried to select players who are more inclined to fuuro on the training data set, but it didn't seem to have any obvious impact. So do you think there is any better way? Thanks a lot!

[Cryolite]

It depends on whether you are using kanachan.training.bert.train or kanachan.training.iql.train. Which are you using?

[Pragmatism0220]

iql

[Cryolite]

IQL is based on offline reinforcement learning, which generally does not reflect trends in the training data. The ultimate goal of IQL is to maximize the reward of reinforcement learning. For example, if the training data contains poor quality samples, i.e., bad moves, IQL uses them as good examples of "what not to do". Therefore, even if you have training data with a high fuuro (副露) rate, IQL still aims to maximize the reward, and if it determines that a fuuro does not contribute to maximizing the reward, IQL will learn the fuuro as a "what not to do".

The question is what exactly is the reward in IQL. The current implementation of IQL is hard-coded to use the increase or decrease in grading points of a half-length game (半荘戦) as the reward. I plan to change this part of the implementation in the near future to allow users to set arbitrary rewards. When this change is implemented, you can create a model with a high fuuro rate by setting some kind of reward for fuuro. I hope you will wait for a while until then.

[Pragmatism0220]

Thank you for your detailed answer! My understanding of it has deepened. I will wait, and I am looking forward to it!

[Cryolite]

I have implemented a new feature called reward plug-in. Users can freely design rewards in reinforcement learning by implementing a reward plug-in (Python source code). From now on, please try adjusting the direction of training with reinforcement learning using this feature.