RuntimeError in domain_updater: torch.cat expected a non-empty list of Tensors

[itamar108]

Hello, I’m new to alpha-beta-CROWN.

When running the branch-and-bound (bab) nonlinear-splitting heuristic, for a certain sample and specific x_range, I encounter the following RuntimeError: torch.cat(): expected a non-empty list issue:

BaB round 1
batch: 1
Start filtering...
Traceback (most recent call last):
  File ".../alpha-beta-CROWN/complete_verifier/abcrown.py", line 823, in <module>
    abcrown.main()
  File ".../alpha-beta-CROWN/complete_verifier/abcrown.py", line 797, in main
    verified_status = self.complete_verifier(
                      ^^^^^^^^^^^^^^^^^^^^^^^
  File ".../alpha-beta-CROWN/complete_verifier/abcrown.py", line 501, in complete_verifier
    l, nodes, ret = self.bab(
                    ^^^^^^^^^
  File ".../alpha-beta-CROWN/complete_verifier/abcrown.py", line 308, in bab
    result = general_bab(
             ^^^^^^^^^^^^
  File ".../alpha-beta-CROWN/complete_verifier/bab.py", line 462, in general_bab
    global_lb = act_split_round(
                ^^^^^^^^^^^^^^^^
  File ".../alpha-beta-CROWN/complete_verifier/bab.py", line 189, in act_split_round
    split_domain(net, domains, d, batch, impl_params=impl_params,
  File ".../alpha-beta-CROWN/complete_verifier/bab.py", line 74, in split_domain
    branching_heuristic.get_branching_decisions(
  File ".../alpha-beta-CROWN/complete_verifier/heuristics/nonlinear/bbps.py", line 139, in get_branching_decisions
    layers, indices, points = self._filter(
                              ^^^^^^^^^^^^^
  File ".../alpha-beta-CROWN/complete_verifier/heuristics/nonlinear/bbps.py", line 214, in _filter
    ret_lbs = self._compute_actual_bounds(domains, decisions)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../alpha-beta-CROWN/complete_verifier/heuristics/nonlinear/bbps.py", line 165, in _compute_actual_bounds
    self.net.build_history_and_set_bounds(
  File ".../alpha-beta-CROWN/complete_verifier/beta_CROWN_solver.py", line 814, in build_history_and_set_bounds
    domain_updater.set_branched_bounds(d, split, mode)
  File ".../alpha-beta-CROWN/complete_verifier/domain_updater.py", line 142, in set_branched_bounds
    new_alphas[k] = {kk: torch.cat([vv] * self.num_copy, dim=2)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../alpha-beta-CROWN/complete_verifier/domain_updater.py", line 142, in <dictcomp>
    new_alphas[k] = {kk: torch.cat([vv] * self.num_copy, dim=2)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: torch.cat(): expected a non-empty list of Tensors

Other stages (PGD and ACROWN) complete successfully on those inputs. Furthermore, this error only appears for very small x_range values: specifically, neighborhoods smaller than 0.001 (e.g., (x – 0.001, x + 0.001)) when verifying MNIST samples.

My model is composed of a fully connected mnist model:

class Net(nn.Module):
    def __init__(self, input_size, hidden_size_1, hidden_size_2, num_classes):
        super(Net, self).__init__()
        self.fc1 = nn.Linear(input_size, hidden_size_1)
        self.relu = nn.ReLU()
        self.fc2 = nn.Linear(hidden_size_1, hidden_size_2)
        self.fc3 = nn.Linear(hidden_size_2, num_classes)

    def forward(self, x):
        out = self.fc1(x)
        out = self.relu(out)
        out = self.fc2(out)
        out = self.relu(out)
        out = self.fc3(out)
        return out

Repository & branch: latest main of alpha-beta-CROWN.
Configuration (in config.yaml):

bab:
  pruning_in_iteration: False
  sort_domain_interval: 1
  branching:
    method: nonlinear
    candidates: 3
    nonlinear_split:
      num_branches: 2
      method: shortcut
      filter: true

Relevant code (in domain_updater.py):

if 'alphas' in d:
    new_alphas = defaultdict(dict)
    for k, v in d['alphas'].items():
        new_alphas[k] = {
            kk: torch.cat([vv] * self.num_copy, dim=2)
            for kk, vv in v.items()
        }
    d['alphas'] = new_alphas

System configuration
OS: macOS 12.7
Python: 3.11.7
PyTorch: 2.2.2
Hardware: MacBook Air, intel i7
Clean environment: yes, tested in a fresh virtualenv with latest main.

Thanks in advance