Offline Policy Evaluation: Counterfactual Estimator

This repo is to reproduce the result of Doubly Robust Estimator.

Dataset: ./data/*

Dataset	Samples	Features	Labels
ecoli	336	7	8
glass	214	9	6
letter	20000	16	26
optdigits	5620	64	10
page_blocks	5473	10	5
pendigits	10992	16	10
satimage	6435	36	6
vehicle	846	18	4
yeast	1484	8	10

**All data can be found under UCI's great repositories

Results

• Bias: ./results/result_bias.txt
• RMSE: ./results/result_rmse.txt

Usage

• If Anaconda, please create your env: conda create -n <ENV_NAME> python=3.6.8
• Get dependencies: pip install -r requirements.txt
• Run the main file: python main.py

Dependencies

• Python: 3.6.8
• Packages: See ./requirements.txt
• OS: Windows10, MacOS, Ubuntu(18.04/16.04 LTS)

TODO

• Fuse the evaluation metrics of OPE by Mix and Rank!!

Reference

• IPS/IS
  • Capped IPS/IS
  • Self Normalised IPS/IS
  • Pointwise IPS
• Doubly Robust
  • SWITCH
  • Continuous Adaptive Blending
• Mix and Rank: A Framework for Benchmarking Recommender Systems