Skip to content

BUG: boolean-mask operations on a null-dtype ArrowDtype column abort the process (SIGABRT) #66703

Description

@jbrockmendel

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import numpy as np
import pandas as pd
import pyarrow as pa

ser = pd.Series(pa.array([None, None, None], type=pa.null()), dtype=pd.ArrowDtype(pa.null()))

ser[np.array([True, False, False])] = None   # <- process aborts (SIGABRT)

Every one of these aborts the interpreter, all through ArrowExtensionArray._replace_with_mask:

ser[np.array([True, False, False])] = None      # boolean-mask setitem
ser.where(np.array([True, False, False]))
ser.mask(np.array([True, False, False]))
ser.combine_first(ser)

df = ser.to_frame("a")
df.loc[np.array([True, False, False]), "a"] = None

Issue Description

For a null-typed ArrowDtype column, any operation routed through
ArrowExtensionArray._replace_with_mask kills the process rather than raising:

/arrow/cpp/src/arrow/result.cc:27: Constructed with a non-error status: OK
[1]    79074 abort      python

This is a hard abort, not a Python exception, so it cannot be caught and takes down the
interpreter (including a pytest run). It is specific to the null type — I swept
string, dictionary, int64, list, struct and map through the same
boolean-mask setitem and all behave correctly.

The root cause is the pyarrow kernel, reduced to:

import pyarrow as pa, pyarrow.compute as pc
pc.replace_with_mask(pa.array([None], pa.null()), [True], pa.scalar(None, type=pa.null()))

That was reported upstream as apache/arrow#47447 and is fixed,
but the fix is milestoned for pyarrow 25.0.0
. pandas supports pyarrow >= 13.0.0, so with
every released pyarrow this is currently reachable from public pandas APIs.

_replace_with_mask already carries a version-independent workaround for the sibling
replace_with_mask defect on chunked booleans, immediately above the failing call:

if isinstance(values, pa.ChunkedArray) and pa.types.is_boolean(values.type):
    # GH#52059 replace_with_mask segfaults for chunked array
    # https://github.com/apache/arrow/issues/34634
    values = values.combine_chunks()
try:
    return pc.replace_with_mask(values, mask, replacements)

so a guard for the null type would sit naturally next to it — an all-null array has nothing
to mask out, so it can simply be returned unchanged. Alternatively it could be gated on the
existing pa_version_under25p0 flag, which array.py already uses for a different pyarrow 25
behavior change.

Noticed while working on GH-64320; the same helper is called from the fix there.

Expected Behavior

The operations above should return a null-dtype result rather than aborting:

>>> ser[np.array([True, False, False])] = None
>>> ser.tolist()
[<NA>, <NA>, <NA>]

At minimum pandas should never abort the interpreter for a supported dtype and a documented
API call.

Installed Versions

Details
commit                : ebd40365c94b2d34ce56ff5bfe1a527ed1bb7f49
python                : 3.13.11
python-bits           : 64
OS                    : Darwin
pandas                : 3.1.0.dev0+1520.gebd40365c94
numpy                 : 2.4.4
dateutil              : 2.9.0.post0
pyarrow               : 23.0.1
pytz                  : 2026.1.post1

Metadata

Metadata

Assignees

No one assigned

    Labels

    Arrowpyarrow functionalityBugSegfaultNon-Recoverable Error

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions