Explainable Reinforcement Learning with Adversarial LLM Feedback

Introduction

This repository contains the code and resources for the course project, of the course CS787 : Generative AI, project titled "Explainable Reinforcement Learning with Adversarial LLM Feedback". The project investigates the use of large language models (LLMs) as adversarial agents to improve the explainability and performance of reinforcement learning (RL) systems. It compares the outcomes of using RL alone versus RL assisted with LLM feedback.

We have used LLM using Gemini API, we used Gemini's 2.5-flash model for generating feedback as adversarial and defining the reward while fine tuning RL model.

Installation

1. Clone the repository:

   git clone https://github.com/Gaurav142005/XRL_CS787.git
   cd XRL_CS787

2. (Optional) Set up a virtual environment:

   python -m venv venv
   source venv/bin/activate

3. Install dependencies:

   pip install -r requirements.txt

4. Setup Environment Variables (Linux):

   echo 'GOOGLE_API_KEY = "your_google_api_key_here"\' > .env

---

Usage

Visualising Our Results

Although we have uploaded our results in the results folder, if you want to visualise the model for 6x6 grid, then just run:

python ./codes/visualise.py

Note: Initialise the API key to any dummy string for this.

Training on device

If you want to train on your device you have to run:

python ./codes/training.py

And if you want to evaluate and plot comparing graphs then run:

python ./codes/eval.py
python ./codes/plot_results.py