Describe the bug
Train the model using UpliftRandomForestClassifier and using default configuration, When the amount of training data reaches 4.5G(about 2.9 million rows of data, 440 features), the required memory exceeds 80G. I'm not sure if this is a bug,
To Reproduce
Can be reproduced the situation using random data。
from causalml.inference.tree import UpliftRandomForestClassifier
from causalml.metrics import plot_gain
from sklearn.model_selection import train_test_split
import causalml
causalml.__version__
from causalml.inference.tree import UpliftTreeClassifier
import numpy as np
from sys import getsizeof
import pandas as pd
treatment = np.random.randint(0, 2, (2900000)).astype(np.float32)
y = np.random.randint(0, 2, (2900000)).astype(np.float32)
feats = np.random.randn(2900000, 440).astype(np.float32)
treatment_bak = ['treat' if v==1 else 'control' for v in treatment.tolist()]
uplift_model = UpliftRandomForestClassifier(control_name='control')
uplift_model.fit(X=feats,
treatment=np.array(treatment_bak),
y=y)
Expected behavior
Is there any way to reduce memory usage besides reducing the amount of data?
Screenshots
If applicable, add screenshots to help explain your problem.
Environment (please complete the following information):
- OS: ubuntu
- Python Version: 3.6,
- Versions of Major Dependencies (
pandas, scikit-learn, cython): pandas== 1.0.4 , scikit-learn==0.24.2, cython==0 0.29.24, causalml=0.12.3
Additional context
Add any other context about the problem here.
Describe the bug
Train the model using UpliftRandomForestClassifier and using default configuration, When the amount of training data reaches 4.5G(about 2.9 million rows of data, 440 features), the required memory exceeds 80G. I'm not sure if this is a bug,
To Reproduce
Can be reproduced the situation using random data。
Expected behavior
Is there any way to reduce memory usage besides reducing the amount of data?
Screenshots
If applicable, add screenshots to help explain your problem.
Environment (please complete the following information):
pandas,scikit-learn,cython): pandas== 1.0.4 , scikit-learn==0.24.2, cython==0 0.29.24, causalml=0.12.3Additional context
Add any other context about the problem here.