Invalid memory access in `Zoltan_PartDist_MPIOp`

[manodeep]

I have tracked down for out-of-range memory access in this MPI operation. In these lines, 6 memory addresses are unconditionally updated without checking the len parameter. I have verified that with OpenMPI v5.0.8, the default collective han algorithm results in a call to the function with len=3, and that causes an invalid memory write (because the 6 values are written unconditionally). The crash always occurs on the first rank on the node.

A potential fix is to not do anything when len<6 and always perform the (current) operation in units of 6 integers. I have tested this patch and fixes the crash with OpenMPI v5.0.8. Please let me know if the patch seems reasonable - happy to put through a PR with the patch

Note: The issue was debugged and patched with Claude Opus 5 (at Max effort). I have looked through the patch and seems sensible (fusing the 6 integers into a single custom MPI datatype) and the patch itself works for our simulations and produces bitwise identical results.

[anntsay]

It looks like current ESMF is using Zoltan 2.1. Team will assess whether to take this opportunity to update to a newer version of Zoltan (current Zoltan is 3.6) or take the user's patch to fix this immediate issue.

[manodeep]

Thank you for the quick response! Yes, please let me know what you decide.

I took a look at the latest Zoltan release (3.901) and that still has the same bug (unconditional writes into 6 integers without checking len)

void Zoltan_PartDist_MPIOp(
  void *in, 
  void *inout, 
  int *len, 
  MPI_Datatype *dptr)
{
int *int_in = (int *) in;
int *int_inout = (int *) inout;

  int_inout[0] += int_in[0];
  int_inout[1] += int_in[1];
  int_inout[2] = ((int_in[2] > int_inout[2]) ? int_in[2] : int_inout[2]);
  int_inout[3] += int_in[3];
  int_inout[4] += int_in[4];
  int_inout[5] += int_in[5];
}

Ultimately, the patch probably needs to go into the Zoltan library