Reinforcement Learning

Why RL

There are three categories of ML algorithms : supervised, unsupervised and reinforcement. While supervised and unsupervised learning can be useful for big data stuff, I believe that IoT applications on embedded systems will solely use RL, because it's the only category that adapts itself to the environment, considering that the future will be different from the past.

Generic RL

There are several algorithms in RL, and lots of differences between those, but probably the way to start is with a kind of "generic MDP" (Markov Decision Process), that follows and optimises a policy to control a state machine by performing some actions in function of the expected reward.

Inverse RL

In generic RL, the reward function is considered as a short description of the task. But often, is it not obvious to define the reward function. Inverse RL, that also has variants, aims at optimising the reward function based on several policies and/or trajectories or episodes provided by an expert a.k.a. teacher.

Interesting RL Links

• Reinforcement learning strategies
• Recursive Least Squares for Linear Contextual Bandits, Parfenov, 2020
• Multi-Step Reinforcement Learning: A Unifying Algorithm, Asis et al., 2018
• Contextual Multi-Armed Bandit Problems in Reinforcement Learning, Shkulov, 2023
• GenRL easy implementations in Python : Tutorials
• Bayesian Reinforcement Learning: A Survey, Mohammad Ghavamzadeh et al., 2016
• Fast Approximations of Activation Functions in Deep Neural Networks when using Posit Arithmetic, Marco Cococcioni, Federico Rossi, Emanuele Ruffaldi and Sergio Saponara, in Sensors 2020, 20, 1515; doi:10.3390/s20051515
• A survey of inverse reinforcement learning, Stephen Adams, Tyler Cody and Peter A. Beling, 2022, Springer, or the similar article published by Elsevier, Saurabh Arora and Prashant Doshi, 2021, consider the task of predicting the rewards instead of finding the policy that maximise these.
• RL upside down, Matthia Sabatelli, 2021
  This presentation focuses on a new way to define rewards. It proposes to map the rewards to the actions instead of predicting them.