Merge pull request #2056 from graphistry/fix/gfql-2050-binding-alias-validation

fix(gfql): chain result contract: binding-column aliases decline, duplicate ids answer once, no internal columns (#2050, #2051, #2067)

Author: lmeyerov

CHANGELOG.md

@@ -18,8 +18,12 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
 
 ### Fixed
 
+* GFQL: a native chain whose edge alias is named like the source, destination or edge-ID binding column silently overwrote that binding on pandas and cuDF (the seed's edges vanished, or the edge ids became `True`) and raised a raw polars `SchemaError`; it is now the same typed decline (E108) the node-ID collision already gets, on every engine, before execution — the rule the Cypher route already applied (#2050).
+* GFQL polars: a colliding alias no longer leaves the internal `__gfql_shadow_restore__<alias>__` column on op-list results; the chain keeps the shadowed values under that name only while a compiled Cypher pipeline runs (its row pipeline reads them back), and on `gfql([...])` / `chain([...])` the marker simply shadows the column as on pandas and cuDF. Nothing is stripped from results: user-defined columns and pipelines composed from successive `gfql` calls are untouched.
+
 * GFQL: a multi-hop step (`hops=2`, `to_fixed_point`) whose edge alias shares its name with the column its own filter uses raised `incompatible-column-type` on pandas and cuDF; the backward re-execution now reads the graph's edge columns for that step, so the eager engines serve it like the single-hop form and return the same rows as polars (#2049).
 * GFQL: one alias-shadowing contract on every engine for op-list chains: an alias named like a column of the frame it marks becomes the boolean marker under that name (pandas, cuDF and now polars, which used to keep the user values and leak a `_right` join suffix). The shadowed values ride under the internal restore column the Cypher row pipeline already resolves, so Cypher keeps reading the user value through the variable (`MATCH (a)-[type:K]->(b) RETURN type.type`) on both engines; the polars residual decline for that shape is gone.
+* GFQL polars: an unnamed, untyped single-hop chain kept duplicate node rows (a node table carrying the same id twice) where pandas, cuDF and every other polars shape collapse them; the plain single-hop branch now returns one row per node id (#2051).
 - **cuDF 26.2 compatibility: `cudf.from_pandas` replaces the removed `cudf.DataFrame.from_pandas` at the five cuDF-only product sites (`ai_utils`, `umap_utils`, `feature_utils`) and in the test fixtures (#2043)**; the three cuDF chain differential cases that disagree with the full path on cuDF 26.2 are marked expected-failure on that line with the tracking issue, so the GPU lane reports them instead of crashing before them.
 - **GFQL pandas/cuDF: several single-alias `IN` (and other pushed-down) predicates across a hop no longer raise `Unalignable boolean Series` (#2020)**: the predicate pushdown filtered an alias frame by label after an earlier pushdown had already narrowed it, while the mask it evaluated carried a fresh positional index. Rows are now kept by position, which is the contract of a mask computed on the same rows; results equal the polars engine and the scalar `=` form.
 - **GFQL polars: a native chain whose edge alias shares its name with the column that step filters on is served instead of raising `incompatible-column-type` (#2039)**: the backward pass and the pruned re-execution now run each step on the graph's original edge columns, and a node step whose alias names its own filtered column filters the graph's node values rather than the marker of an earlier pass, so a stamped alias marker is never re-filtered as that column; pandas and polars return the same rows. A cross-engine collision matrix pins the single-hop shapes and records the multi-hop (#2049) and binding-column (#2050) forms as expected failures. The Cypher rows-route projection of such an alias on polars still declines with a typed error (pinned) and stays tracked.

bin/test-polars.sh

@@ -122,6 +122,7 @@ POLARS_TEST_FILES=(
     graphistry/tests/compute/gfql/lazy/engine/polars/test_chain_duplicate_node_rows_2051.py
     graphistry/tests/compute/gfql/cypher/test_variable_column_collision.py
     graphistry/tests/compute/test_chain_alias_column_collision.py
+    graphistry/tests/compute/test_gfql_op_list_hides_internal_columns.py
     graphistry/tests/compute/gfql/test_engine_polars_semi_key_dedup.py
     graphistry/tests/compute/gfql/test_engine_polars_call_modality.py
     graphistry/tests/compute/gfql/test_engine_polars_gpu.py

docs/source/gfql/spec/language.md

@@ -615,7 +615,7 @@ For Python accessor details (including row-pipeline result materialization), see
 
 ### Named Results
 
-Operations with `name` parameter add boolean columns to mark matched entities:
+Operations with `name` parameter add boolean columns to mark matched entities. The marker takes the name: an existing column of that name on the same frame is replaced (later definitions win, as with dataframe assignment). A `name` equal to that frame's binding column (the node id, or the edge source, destination or id) is rejected with a validation error, since the binding is structural and cannot be shadowed.
 
 ```python
 result = g.gfql([

graphistry/compute/chain.py

@@ -1027,6 +1027,54 @@ def _materialize_fast_path_graph() -> Plottable:
         g.nodes(nodes).edges(edges), alias_n0, alias_e1, alias_n2, src, dst, node, direction)
 
 
+def reject_alias_named_like_binding(
+    g: Plottable, chain_obj: "Chain", *, include_edge_endpoint_aliases: bool = False
+) -> None:
+    """Typed decline for an alias named after a binding column: a node alias equal to the
+    node-ID binding, and (native chains and Cypher alike) an edge alias equal to the source,
+    destination or edge-ID binding.
+
+    The alias marker is stamped as ``<alias> = True``, so an alias equal to the node-id
+    column overwrites the ids themselves: pandas then died with a raw
+    ``ValueError: The column label 'id' is not unique`` from the chain's own merge while
+    polars answered ``True``. Neither is a usable result; decline the same way on both.
+    """
+    from graphistry.compute.exceptions import ErrorCode, GFQLValidationError
+    node_id = getattr(g, "_node", None)
+    endpoint_cols = {
+        col for col in (getattr(g, "_source", None), getattr(g, "_destination", None), getattr(g, "_edge", None))
+        if isinstance(col, str)
+    }
+    for op in chain_obj.chain:
+        if isinstance(node_id, str) and isinstance(op, ASTNode) and getattr(op, "_name", None) == node_id:
+            raise GFQLValidationError(
+                ErrorCode.E108,
+                "A node alias cannot be named after the node-ID binding column",
+                field="chain.name",
+                value=node_id,
+                suggestion=(
+                    f"The alias flag is materialized as a column named '{node_id}', which would "
+                    f"overwrite the node-ID binding. Rename the alias."
+                ),
+            )
+        if (
+            include_edge_endpoint_aliases
+            and isinstance(op, ASTEdge)
+            and getattr(op, "_name", None) in endpoint_cols
+        ):
+            raise GFQLValidationError(
+                ErrorCode.E108,
+                "An edge alias cannot be named after an edge endpoint binding column",
+                field="chain.name",
+                value=getattr(op, "_name", None),
+                suggestion=(
+                    "The alias flag is materialized as a column named like the edge "
+                    "source, destination or edge-ID binding, which would overwrite it. "
+                    "Rename the alias."
+                ),
+            )
+
+
 @otel_traced("gfql.chain", attrs_fn=_chain_otel_attrs)
 def chain(
     self: Plottable,
@@ -1088,6 +1136,7 @@ def _chain_with_strictness(
     # _coerce_input_formats then converts input formats (polars, arrow, spark, dask) to that engine.
     if isinstance(engine, str):
         engine = EngineAbstract(engine)
+    reject_alias_named_like_binding(self, ops if isinstance(ops, Chain) else Chain(ops), include_edge_endpoint_aliases=True)
     from graphistry.compute.ComputeMixin import _coerce_input_formats  # lazy — avoids circular import
     engine_concrete_early = resolve_engine(engine, self)
     if engine_concrete_early in (Engine.POLARS, Engine.POLARS_GPU):

graphistry/compute/gfql/identifiers.py

@@ -1,3 +1,6 @@
+from contextlib import contextmanager
+from contextvars import ContextVar
+from typing import Iterator
 """GFQL reserved identifiers and validation."""
 
 import re
@@ -152,3 +155,21 @@ def validate_column_references(
     'dst', 'dest', 'destination', 'to', 'target',
     'type', 'label', 'name'
 }
+
+
+_SHADOW_RESTORE_WANTED: "ContextVar[bool]" = ContextVar("gfql_shadow_restore_wanted", default=False)
+
+
+def shadow_restore_wanted() -> bool:
+    """Whether the executing pipeline will read shadowed alias values back (the Cypher row pipeline does; the op-list surface does not)."""
+    return _SHADOW_RESTORE_WANTED.get()
+
+
+@contextmanager
+def cypher_pipeline() -> Iterator[None]:
+    """Mark a compiled Cypher execution: chains keep shadowed alias values under the restore column for the row pipeline."""
+    token = _SHADOW_RESTORE_WANTED.set(True)
+    try:
+        yield
+    finally:
+        _SHADOW_RESTORE_WANTED.reset(token)

graphistry/compute/gfql/lazy/engine/polars/chain.py

@@ -29,7 +29,7 @@
 from .degrees import get_degrees_polars, get_indegrees_polars, get_outdegrees_polars
 from .predicates import filter_by_dict_polars
 from .reserved_columns import CHAIN_NODE_HOP
-from graphistry.compute.gfql.identifiers import shadow_restore_column
+from graphistry.compute.gfql.identifiers import shadow_restore_column, shadow_restore_wanted
 
 
 def _polars_error_types() -> Tuple[Type[BaseException], ...]:
@@ -370,9 +370,12 @@ def _combine_edges(g: "_LazyShim",
     for op, g_step in label_steps:
         if op._name is not None and isinstance(op, ASTEdge) and g_step._edges is not None and op._name in colnames(g_step._edges):
             named = g_step._edges.filter(pl.col(op._name)).select(pl.col(edge_id)).with_columns(pl.lit(True).alias(op._name))
-            if op._name in colnames(out):  # the alias marker shadows the user column; its values stay under the restore name for Cypher scoping
-                restore = shadow_restore_column(op._name)
-                out = out.drop([c for c in (restore,) if c in colnames(out)]).rename({op._name: restore})
+            if op._name in colnames(out):  # the alias marker shadows the user column; only a Cypher pipeline keeps its values, under the restore name
+                if shadow_restore_wanted():
+                    restore = shadow_restore_column(op._name)
+                    out = out.drop([c for c in (restore,) if c in colnames(out)]).rename({op._name: restore})
+                else:
+                    out = out.drop(op._name)
             out = out.join(named, on=edge_id, how="left").with_columns(pl.col(op._name).fill_null(False))
     return out
 
@@ -1072,6 +1075,7 @@ def _filter_ids(node_op: ASTNode) -> "Optional[PolarsFrame]":
                         endpoint_ids(edges, scol, dcol, ncol), on=ncol, how="semi")
             else:
                 nodes = gf._nodes.join(endpoints, on=ncol, how="semi")
+            nodes = nodes.unique(subset=[ncol], maintain_order=True)  # one row per node id, as the full chain and pandas collapse
             return gf.nodes(nodes, ncol).edges(_restore_edge_dtypes(edges, scol, dcol, restore), scol, dcol)
 
     if start_nodes is not None:

graphistry/compute/gfql_fast_paths.py

@@ -3576,7 +3576,8 @@ def _execute_seeded_node_lookup_fast_path(
         if edges is not None:
             out._edges = edges.head(0) if is_polars else edges.head(0).reset_index(drop=True)
         if suffix_ops:
-            return chain_impl(out, suffix_ops, engine=engine, policy=policy, context=context)
+            tail_ops: List[ASTObject] = list(suffix_ops)
+            return chain_impl(out, tail_ops, engine=engine, policy=policy, context=context)
         return out
     assert projection is not None
     if is_polars:
@@ -3819,7 +3820,8 @@ def _execute_seeded_typed_hop_fast_path(
         out._nodes = out_frame
         out._edges = _empty_edges_with_alias_marker(_edges, e1._name, is_polars)
         if suffix_ops:
-            return chain_impl(out, suffix_ops, engine=engine, policy=policy, context=context)
+            tail_ops: List[ASTObject] = list(suffix_ops)
+            return chain_impl(out, tail_ops, engine=engine, policy=policy, context=context)
         return out
     if bag_rows:
         p_rows = _seeded_typed_hop_bag_rows(
@@ -3854,9 +3856,10 @@ def _execute_seeded_typed_hop_fast_path(
         # edges are the matched hop edges, so take their zero-row head.
         out._edges = _empty_edges_with_alias_marker(_edges, e1._name, is_polars)
         if suffix_ops:
+            tail_ops_indexed: List[ASTObject] = list(suffix_ops)
             return chain_impl(
                 out,
-                suffix_ops,
+                tail_ops_indexed,
                 engine=engine,
                 policy=policy,
                 context=context,

graphistry/compute/gfql_unified.py

@@ -14,6 +14,7 @@
 from .ast import ASTObject, ASTLet, ASTNode, ASTEdge, ASTCall
 from .chain import Chain, chain as chain_impl
 from .gfql.query_types import GFQLQuery
+from .chain import reject_alias_named_like_binding
 from .chain_let import chain_let as chain_let_impl
 from .execution_context import ExecutionContext
 from .gfql.policy import (
@@ -25,7 +26,7 @@
     QueryType,
     expand_policy
 )
-from graphistry.compute.gfql.identifiers import TRAIL_ARM_EDGE_ALIAS_PREFIX
+from graphistry.compute.gfql.identifiers import cypher_pipeline, TRAIL_ARM_EDGE_ALIAS_PREFIX
 from graphistry.compute.gfql.same_path_types import (
     EDGE_IDENTITY_COLUMN,
     NODE_IDENTITY_COLUMN,
@@ -763,7 +764,7 @@ def _apply_connected_match_join(
     cache_store: Dict[str, Any] = {}
 
     for pattern_chain in plan.pattern_chains:
-        _reject_node_alias_shadowing_id_binding(
+        reject_alias_named_like_binding(
             base_graph, pattern_chain, include_edge_endpoint_aliases=True
         )
 
@@ -1591,7 +1592,7 @@ def _execute_compiled_query_chain_non_union(
             compiled_query=compiled_query,
             engine=engine,
         )
-    _reject_node_alias_shadowing_id_binding(
+    reject_alias_named_like_binding(
         base_graph, compiled_query.chain, include_edge_endpoint_aliases=True
     )
 
@@ -1688,6 +1689,19 @@ def _execute_compiled_query_with_reentry(
     engine: Union[EngineAbstract, str],
     policy: Optional[PolicyDict],
     context: ExecutionContext,
+) -> Plottable:
+    with cypher_pipeline():
+        return _execute_compiled_query_with_reentry_impl(
+            base_graph, compiled_query=compiled_query, engine=engine, policy=policy, context=context)
+
+
+def _execute_compiled_query_with_reentry_impl(
+    base_graph: Plottable,
+    *,
+    compiled_query: Union[CompiledCypherQuery, CompiledCypherUnionQuery],
+    engine: Union[EngineAbstract, str],
+    policy: Optional[PolicyDict],
+    context: ExecutionContext,
 ) -> Plottable:
     if isinstance(compiled_query, CompiledCypherUnionQuery):
         return _execute_compiled_query(
@@ -2765,51 +2779,6 @@ def _gfql_with_strictness(
             context.policy_depth = policy_depth
 
 
-def _reject_node_alias_shadowing_id_binding(
-    g: Plottable, chain_obj: Chain, *, include_edge_endpoint_aliases: bool = False
-) -> None:
-    """Typed decline for a node alias named after the node-ID binding column.
-
-    The alias marker is stamped as ``<alias> = True``, so an alias equal to the node-id
-    column overwrites the ids themselves: pandas then died with a raw
-    ``ValueError: The column label 'id' is not unique`` from the chain's own merge while
-    polars answered ``True``. Neither is a usable result; decline the same way on both.
-    """
-    node_id = getattr(g, "_node", None)
-    endpoint_cols = {
-        col for col in (getattr(g, "_source", None), getattr(g, "_destination", None))
-        if isinstance(col, str)
-    }
-    for op in chain_obj.chain:
-        if isinstance(node_id, str) and isinstance(op, ASTNode) and getattr(op, "_name", None) == node_id:
-            raise GFQLValidationError(
-                ErrorCode.E108,
-                "A node alias cannot be named after the node-ID binding column",
-                field="chain.name",
-                value=node_id,
-                suggestion=(
-                    f"The alias flag is materialized as a column named '{node_id}', which would "
-                    f"overwrite the node-ID binding. Rename the alias."
-                ),
-            )
-        # Cypher-only decline; raw GFQL chains keep their documented overwrite parity.
-        if (
-            include_edge_endpoint_aliases
-            and isinstance(op, ASTEdge)
-            and getattr(op, "_name", None) in endpoint_cols
-        ):
-            raise GFQLValidationError(
-                ErrorCode.E108,
-                "An edge alias cannot be named after an edge endpoint binding column",
-                field="chain.name",
-                value=getattr(op, "_name", None),
-                suggestion=(
-                    "The alias flag is materialized as a column named like the edge "
-                    "source/destination binding, which would overwrite the endpoints. "
-                    "Rename the alias."
-                ),
-            )
-
 
 def _chain_dispatch(
     g: Plottable,
@@ -2819,7 +2788,7 @@ def _chain_dispatch(
     context: ExecutionContext,
     start_nodes: Optional[DataFrameT] = None,
 ) -> Plottable:
-    _reject_node_alias_shadowing_id_binding(g, chain_obj)
+    reject_alias_named_like_binding(g, chain_obj, include_edge_endpoint_aliases=True)
     engine_name = engine.value if hasattr(engine, "value") else str(engine)
     if chain_obj.where and engine_name in (Engine.POLARS.value, Engine.POLARS_GPU.value):
         # Cross-entity / same-path WHERE routes through DFSamePathExecutor

graphistry/tests/compute/gfql/lazy/engine/polars/test_chain_duplicate_node_rows_2051.py

@@ -3,6 +3,7 @@
 Sibling of #1993 (polars ``hop()`` de-dups its node table): the unnamed, untyped single-hop
 chain shape still keeps the duplicate (#2051, strict expected failure); the named, typed,
 multi-hop and undirected shapes and ``hop()`` already collapse it and are pinned green.
+chain shape kept the duplicate (#2051); every shape now collapses it, pinned against pandas.
 """
 import pandas as pd
 import pytest
@@ -47,7 +48,6 @@ def test_hop_collapses_the_duplicate():
 
 
 @pytest.mark.parametrize("shape", ["unnamed untyped single hop", "unnamed single hop with destination filter"])
-@pytest.mark.xfail(strict=True, reason="graphistry/pygraphistry#2051")
 def test_unnamed_untyped_single_hop_collapses_the_duplicate(shape):
     g_pd, g_pl = _graph()
     ops = [n({"key": 1}), e_forward(), n()] if shape == "unnamed untyped single hop" else [n({"key": 1}), e_forward(), n({"id": 20})]