Reinforcement learning is an area of machine learning where an agent observes its state in the environment and acts upon it to receive rewards and consequently, to learn.
In this implementation we use a Feed-forward neural network (FFNN) together with a genetic algorithm (GA) to train an agent to play the 'ATARI' arcade game asteroids. The genetic algorithm belong to the class of evolutionary algorithms and is inspired by natural selection. It thereby relies on biologically inspired operators like selection, crossover and mutation. The algorithm works as follows:
- Initialize a random population
- Evaluate its fitness
- Based on the agents' fitness, select parents to consequently create a next population
- Use biologically inspired operators like crossover and mutation on the new population
- Return to step 1
This process is repeated until a stop condition is met, e.g. the agent successfully learned to achieve its goal and reach a high fitness value.
The inputs and outputs of the single scripts flow as follows:
- Initially, a population with a certain amount of agents is instantiated. The agents' brains are initialized with random weights.
-
- In the environment, observations of the agent are collected and sent to the main() function (the genetic algorithm).
-
- Observations are sent to the agent's brain, which performs a forward pass in a feed-forward neural network.
-
- The output of the FFNN are the actions to be performed by the agent. They are sent to the GA.
-
- The received actions are performed in the agents' respective environments.
-
- The spaceship player object (rotation, thrust and shooting) is updated accordingly.
After a given number of generations is done with training, the average performance of the AI across all generations is plotted.
The fitness is the survival time of each agent. The selection of the parents for the next population takes place as fitness proportinate roulette wheel selection. Crossover is done by using each gene (NN weight) by either parent to 50% for the new child. Finally, each gene of a new child has 90% probability of being mutated by adding/subtracting 10-50% of its original value.
In the helper.py script you can find adjustable hyperparameters for the genetic training loop as well as the FFNN, like number of neurons in the net, number of training generations (epochs) or number of agents per population.