Skip to content

Why does using UpliftRandomForestClassifier consume so much memory? #571

Description

@longweiwei

Describe the bug

Train the model using UpliftRandomForestClassifier and using default configuration, When the amount of training data reaches 4.5G(about 2.9 million rows of data, 440 features), the required memory exceeds 80G. I'm not sure if this is a bug,

To Reproduce
Can be reproduced the situation using random data。

from causalml.inference.tree import UpliftRandomForestClassifier
from causalml.metrics import plot_gain
from sklearn.model_selection import train_test_split
import causalml
causalml.__version__

from causalml.inference.tree import UpliftTreeClassifier
import numpy as np
from sys import getsizeof
import pandas as pd

treatment = np.random.randint(0, 2, (2900000)).astype(np.float32)
y =  np.random.randint(0, 2, (2900000)).astype(np.float32)
feats = np.random.randn(2900000, 440).astype(np.float32)
treatment_bak = ['treat' if v==1 else 'control' for v in treatment.tolist()]

uplift_model = UpliftRandomForestClassifier(control_name='control')
uplift_model.fit(X=feats,
                 treatment=np.array(treatment_bak),
                 y=y)

Expected behavior
Is there any way to reduce memory usage besides reducing the amount of data?

Screenshots
If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

  • OS: ubuntu
  • Python Version: 3.6,
  • Versions of Major Dependencies (pandas, scikit-learn, cython): pandas== 1.0.4 , scikit-learn==0.24.2, cython==0 0.29.24, causalml=0.12.3

Additional context
Add any other context about the problem here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions