Neurosymbolic Reasoning with Incremental Knowledge (InK)

Installation

conda env create -f environment.yml
conda activate ink_bwts

Ensure current pwd is added to PYTHONPATH:

conda env config vars set PYTHONPATH="$PWD:$PYTHONPATH"

Set PROJECT_PATH in configs/config.py to the absolute path of this repo on your machine. Imports and file paths (logs, outputs, saved models) throughout phrl, train_scripts, and eval_scripts are resolved from this single constant, so it's the only place you need to update if the repo is moved or cloned elsewhere:

PROJECT_PATH = "/absolute/path/to/ink_bwts"

Install mujoco. Ensure .mujoco installation added to LD_LIBRARY_PATH

wget https://mujoco.org/download/mujoco210-linux-x86_64.tar.gz
tar -xf mujoco210-linux-x86_64.tar.gz
mkdir /.mujoco
mv mujoco210 /.mujoco/.
conda env config vars set LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/root/.mujoco/mujoco210/bin:/usr/lib/nvidia"

Ensure:

• mujoco_py is importing (check mujoco installation and path, this may raise error and require some apt-get installation depending upon your system)

Overview

Pre-Training

To pretrain NNs in clean maze for use in high level planning. Change env_tag and map_tag for the environment (point maze or ant maze) and map (Four Rooms, Medium, Hard)

python train_scripts/train_pretrain_controller.py

Model will be stores in outputs/pre_training/point_maze//SAC + TILO/simulation_

For ant-maze pretraining try: export LD_LIBRARY_PATH="$(python -c 'import sys,os;print(os.path.join(sys.prefix,"lib"))'):${LD_LIBRARY_PATH}"

RGL Training

Train RGL

python train_scripts/train_rgl.py

This will store the trained model and graph at /outputs/full_training/point_maze//RGL. Evaluation is done with files under /eval_scripts.

For evaluating RGL for when the first goal reached:

python train_scripts/train_rgl_first_goal.py

This will create a first_goal.txt having the number of steps to reach the goal for the first time.

---

Evaluation

Real Maze (InK vs Non-InK)

Table 1

For single goal:

python eval_scripts/real_maze/single_goal/point_maze.py 

For getting summary report of average steps to RGL first goal. logs/submission/rgl_first_goal/summary.txt is used to report results in publication.

python eval_scripts/real_maze/single_goal/rgl_first_goal.py --env point_maze --map four_rooms

For getting average RGL simulation time (which excludes low-level pretraining time). logs/submission/rgl_full/summary.txt is used to report results in publication.

python eval_scripts/real_maze/single_goal/rgl_full.py --env point_maze --map four_rooms

Table 2

For InK-BWTS, InK-BWTS-Prior, InK-D*:

python eval_scripts/real_maze/single_goal/bwts_vs_dstar.py

Figure 1 (in experiments)

For sequence of goals:

python eval_scripts/real_maze/seq_goal/point_maze.py

For InK vs InK (prior)

For bwts with custom prior, mcts.py, line 956, should be used to detect the wall direction. It is by default for horizontal and vertical wall belief set.

python eval_scripts/real_maze/single_goal/prior_bwts.py

---

Abstract Maze

(BWTS and D*)

This allows ablation study on performance when prior knowledge is available. The belief world index is used to process horizontal or vertical walls.

For getting results of BWTS and D*. In the file edit:

• grid_axis: horizontal or vertical belief worlds
• need_obstacle: Whether obstacle is required or not

Result will be stored to /logs

nohup python eval_scripts/abstract_maze/mcts_planner.py > /dev/null 2>&1 &

(BAMCP)

For results with bamcp:

Store results to /logs/submission

nohup python eval_scripts/abstract_maze/bamcp_planner.py > /dev/null 2>&1 &

To obtain the tree for analysis provide the grid index with ix:

python -m pybamcp.run --ix 0