Skip to content

Commit dc9983b

Browse files
lmeyerovclaude
andcommitted
feat(gfql/cypher): searchAny WHERE predicate + docs (viz-filter L2-b/c)
Cypher surface: searchAny(alias, 'term'[, {caseSensitive, regex, columns}]) lifts at lowering assembly into a search_any pre-filter + fresh marker column (the pattern-predicate marker mechanism) — composes through AND/OR/NOT; strict opts validation (unknown keys/bad bools/bad lists/unbound alias -> clear E108); oracle-pinned cypher-surface cases + 4-engine parity-or-NIE incl. NOT/AND composition; docs + CHANGELOG. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 37a0fd8 commit dc9983b

6 files changed

Lines changed: 184 additions & 2 deletions

File tree

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
99
<!-- Do Not Erase This Section - Used for tracking unreleased changes -->
1010

1111
### Added
12+
- **GFQL Cypher `searchAny(entity, term[, opts])` cross-column search predicate + `g.search_nodes()`/`g.search_edges()` (viz-filter L2, native on all four engines)**: True where ANY of the entity's columns matches the term — the streamgl-viz inspector's table-search semantics as a composable WHERE predicate: OR across columns; case-insensitive substring by DEFAULT (folded via lowercase, never regex — the common call avoids every engine regex limit); regex opt-in obeying the same per-engine decline rules as `=~`; dtype gate AS SEMANTICS (string columns always; integer columns iff the term is a numeric literal, per the inspector's `/^[0-9.-]+$/` gate; floats/dates/booleans reachable via the explicit `columns:` list). Options map `{caseSensitive, regex, columns}` is strict-validated (unknown keys error listing the valid ones); unbound aliases and missing explicit columns error clearly; null cells never match. Lowered like the pattern-predicate markers (a `search_any` row op + fresh marker column), so it composes through AND/OR/NOT and different node/edge terms coexist in one pipeline. Per-column matching reuses the parity-hardened `Contains` predicate on pandas/cuDF and a lowercase-fold/any_horizontal lowering on polars; oracle-pinned + 4-engine parity-or-NIE conformance cases. Python twins `g.search_nodes(term, columns=, case_sensitive=, regex=)` / `g.search_edges(...)` filter their own table and return a Plottable (polars-frame twins decline honestly for now — use the cypher op).
1213
- **GFQL Cypher `EXISTS { <pattern> }` pattern-existence subqueries (openCypher-standard), native on all four engines**: `WHERE EXISTS { (n)-[:R]->() }` and `WHERE NOT EXISTS { (n)--() }` now parse and run — the declarative prune-isolated building blocks for the streamgl-viz filter pipeline. An `EXISTS` body reuses the existing pattern-predicate lowering wholesale (`semi_apply_mark` / `anti_semi_apply` row ops), so pandas/cuDF worked immediately; the polars engine gains NATIVE lowerings for the semi-apply family (correlated key sets computed by the polars chain executor's named-flag columns; order-preserving `is_in` joins) plus `rows(binding_ops=...)` for the single-entity row table — previously all honest-NIE. Aliases introduced inside the braces are existentially scoped (`EXISTS { (n)--(m) }` allowed with `m` unbound outside — bare pattern predicates keep the conservative guard), inline property maps work, and the one supported inner `WHERE` form is endpoint inequality — `EXISTS { (n)--(m) WHERE m <> n }`, the drop-self-loop prune-isolated flavor (pandas/cuDF filter the correlated bindings; polars excludes self-loop edges, which is exactly the `m <> n` witness). Both prune flavors are oracle-pinned in the conformance matrix on a self-loop discriminator graph, 4-engine parity-or-NIE. Honest declines with clear errors: `EXISTS` in RETURN/WITH projections, general inner `WHERE`, multi-pattern bodies, full `MATCH..RETURN` subquery bodies, multi-alias correlation on polars.
1314
- **GFQL polars execution config is Python-settable and live**: `set_cpu_streaming(bool)` and `set_gpu_executor('in-memory'|'streaming')` in `graphistry.compute.gfql.lazy` (plus the public `GPU_EXECUTORS` options and `GpuExecutor` type) set the CPU-streaming / GPU-executor knobs from Python. They resolve **Python override > environment variable > default**, read **live** per collect — previously these were env-only (`GFQL_POLARS_CPU_STREAMING` / `GFQL_POLARS_GPU_EXECUTOR`) and frozen at import, so neither a Python setting nor a post-import env change took effect. `None` resets a setter to env/default.
1415
- **GFQL engine conversion honors the `validate`/`warn` convention**: `Engine.df_to_engine(df, engine, *, validate=, warn=)` threads the repo-wide `validate` (`'strict'`/`'strict-fast'`/`'autofix'`; `True`→strict, `False`→autofix) + `warn` protocol into the pandas→polars and pandas→cuDF converters. On a mixed-type object column that Arrow/polars/cuDF cannot represent, `strict` raises (`NotImplementedError` for polars, `ArrowConversionError` for cuDF) and `autofix` coerces the column to string and warns — the same convention as `plot()`/`upload()`. Each engine keeps its established default (polars `strict` = parity-or-raise; cuDF `autofix` = its shipped best-effort coercion, now `warn`-suppressible).

docs/source/gfql/cypher.rst

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -314,6 +314,18 @@ and ``RETURN`` expressions:
314314
``size``, and conversions ``toInteger`` / ``toFloat`` / ``toString`` /
315315
``toBoolean`` and ``coalesce``.
316316
- Regex ``=~`` (see WHERE Forms above).
317+
- ``searchAny(entity, term[, opts])`` — cross-column search predicate (WHERE
318+
position; GFQL extension for the viz filter pipeline): True where ANY of the
319+
entity's columns matches ``term``. Inspector semantics: OR across columns,
320+
case-insensitive substring by default, regex opt-in; dtype-gated — string
321+
columns always, integer columns iff the term is a numeric literal
322+
(``/^[0-9.-]+$/``); floats/dates/booleans only via the explicit list. Options
323+
map: ``{caseSensitive: true, regex: true, columns: ['name', ...]}`` (unknown
324+
keys error, listing the valid ones). Composes with other WHERE predicates
325+
through AND/OR/NOT; nodes and edges independently searchable with different
326+
terms. Runs natively on all four engines; the regex path obeys the same
327+
per-engine decline rules as ``=~``. Python twins:
328+
:meth:`ComputeMixin.search_nodes` / :meth:`ComputeMixin.search_edges`.
317329

318330
``LIKE`` / ``ILIKE`` and ``BETWEEN`` are intentionally not provided — they are
319331
not part of Cypher or GQL; use ``=~`` / ``CONTAINS`` / ``STARTS WITH`` and

graphistry/compute/gfql/cypher/lowering.py

Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@
2424
order_by,
2525
return_,
2626
rows,
27+
search_any,
2728
semi_apply_mark,
2829
serialize_binding_ops,
2930
select,
@@ -3843,6 +3844,103 @@ def _append_page_ops(
38433844
params=params,
38443845
)
38453846

3847+
_SEARCH_ANY_CALL_RE = re.compile(
3848+
r"\bsearchAny\s*\(\s*([A-Za-z_]\w*)\s*,\s*"
3849+
r"('(?:\\.|[^'\\])*'|\"(?:\\.|[^\"\\])*\")"
3850+
r"\s*(?:,\s*\{([^{}]*)\})?\s*\)",
3851+
re.IGNORECASE,
3852+
)
3853+
_SEARCH_ANY_OPT_KEYS = {"casesensitive": "case_sensitive", "regex": "regex", "columns": "columns"}
3854+
_SEARCH_ANY_COLUMNS_RE = re.compile(
3855+
r"^\[\s*(?:'[^']*'|\"[^\"]*\")(?:\s*,\s*(?:'[^']*'|\"[^\"]*\"))*\s*\]$")
3856+
_SEARCH_ANY_COL_ITEM_RE = re.compile(r"'([^']*)'|\"([^\"]*)\"")
3857+
3858+
3859+
def _parse_search_any_opts(opts_text: str, *, line: int, column: int) -> Dict[str, Any]:
3860+
"""Parse searchAny's option map literal — strict: unknown keys error listing the
3861+
valid ones (persona pass: predictable options beat silent typo-tolerance)."""
3862+
out: Dict[str, Any] = {}
3863+
if not opts_text.strip():
3864+
return out
3865+
depth = 0
3866+
parts: List[str] = []
3867+
cur = ""
3868+
for ch in opts_text:
3869+
if ch in "[(":
3870+
depth += 1
3871+
elif ch in "])":
3872+
depth -= 1
3873+
if ch == "," and depth == 0:
3874+
parts.append(cur)
3875+
cur = ""
3876+
else:
3877+
cur += ch
3878+
parts.append(cur)
3879+
for part in parts:
3880+
m = re.match(r"\s*([A-Za-z_]\w*)\s*:\s*(.+?)\s*$", part)
3881+
key = m.group(1) if m else part.strip()
3882+
canon = _SEARCH_ANY_OPT_KEYS.get(key.lower()) if m else None
3883+
if m is None or canon is None:
3884+
raise _unsupported(
3885+
f"searchAny got an unsupported option {key!r}",
3886+
field="where",
3887+
value=opts_text,
3888+
line=line,
3889+
column=column,
3890+
)
3891+
val_text = m.group(2)
3892+
if canon in ("case_sensitive", "regex"):
3893+
low = val_text.lower()
3894+
if low not in ("true", "false"):
3895+
raise _unsupported(
3896+
f"searchAny option {key!r} must be true or false",
3897+
field="where", value=val_text, line=line, column=column,
3898+
)
3899+
out[canon] = low == "true"
3900+
else:
3901+
if not _SEARCH_ANY_COLUMNS_RE.match(val_text):
3902+
raise _unsupported(
3903+
"searchAny option 'columns' must be a list of string literals",
3904+
field="where", value=val_text, line=line, column=column,
3905+
)
3906+
out[canon] = [a or b for a, b in _SEARCH_ANY_COL_ITEM_RE.findall(val_text)]
3907+
return out
3908+
3909+
3910+
def _lift_search_any_from_row_where(
3911+
expr: ExpressionText,
3912+
*,
3913+
alias_targets: Mapping[str, ASTObject],
3914+
existing_cols: AbstractSet[str],
3915+
) -> Tuple[str, List[ASTCall]]:
3916+
"""viz-filter L2-b: rewrite each ``searchAny(alias, 'term'[, {opts}])`` in the
3917+
WHERE row-expression into a fresh boolean MARKER column + a ``search_any`` row
3918+
pre-filter (exactly the pattern-predicate marker mechanism), so the remaining
3919+
boolean expression composes through AND/OR/NOT unchanged."""
3920+
calls: List[ASTCall] = []
3921+
used: set = set(existing_cols)
3922+
3923+
def _sub(m: "re.Match[str]") -> str:
3924+
alias, term_lit, opts_text = m.group(1), m.group(2), m.group(3) or ""
3925+
if alias not in alias_targets:
3926+
raise _unsupported(
3927+
f"searchAny references alias {alias!r} not bound in the active MATCH scope",
3928+
field="where", value=alias,
3929+
line=expr.span.line, column=expr.span.column,
3930+
)
3931+
term = term_lit[1:-1].replace("\\'", "'").replace('\\"', '"')
3932+
opts = _parse_search_any_opts(
3933+
opts_text, line=expr.span.line, column=expr.span.column)
3934+
base = f"__gfql_search_any_{expr.span.line}_{expr.span.column}_{len(calls)}__"
3935+
out_col = _fresh_temp_name(used, base)
3936+
used.add(out_col)
3937+
calls.append(search_any(alias=alias, term=term, out_col=out_col, **opts))
3938+
return out_col
3939+
3940+
new_text = _SEARCH_ANY_CALL_RE.sub(_sub, expr.text)
3941+
return new_text, calls
3942+
3943+
38463944
def _append_match_row_where(
38473945
row_steps: List[ASTObject],
38483946
*,
@@ -6113,6 +6211,16 @@ def lower_match_query(
61136211
span=query.where.span if query.where is not None else merged_match.span,
61146212
)
61156213

6214+
if row_where is not None:
6215+
# viz-filter L2-b: lift searchAny(...) calls into search_any pre-filters +
6216+
# marker columns HERE (assembly level, like the pattern-predicate markers),
6217+
# so every LoweredCypherMatch consumer sees the lifted form.
6218+
lifted_text, search_calls = _lift_search_any_from_row_where(
6219+
row_where, alias_targets=alias_targets, existing_cols=frozenset())
6220+
if search_calls:
6221+
row_pre_filters.extend(search_calls)
6222+
row_where = ExpressionText(text=lifted_text, span=row_where.span)
6223+
61166224
return LoweredCypherMatch(
61176225
query=ops,
61186226
where=where_out,

graphistry/compute/gfql/lazy/engine/polars/chain.py

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -353,9 +353,12 @@ def _try_native_row_op(g_cur, op):
353353
from graphistry.Engine import Engine
354354
from .row_pipeline import (
355355
select_polars, with_columns_polars, order_by_polars, group_by_polars,
356-
unwind_polars, where_rows_polars, rows_binding_ops_polars, search_any_polars,
357-
semi_apply_mark_polars, anti_semi_apply_polars,
356+
unwind_polars, where_rows_polars,
358357
)
358+
from .pattern_apply import (
359+
rows_binding_ops_polars, semi_apply_mark_polars, anti_semi_apply_polars,
360+
)
361+
from .search import search_any_polars
359362

360363
fn = getattr(op, "function", None)
361364
if _call_native_on_polars(op):

graphistry/tests/compute/gfql/cypher/test_lowering.py

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2022,6 +2022,39 @@ def test_lower_exists_drop_self_flavor_emits_neq_semi_apply() -> None:
20222022
assert sorted(anti.params.get("neq") or []) == ["m", "n"]
20232023

20242024

2025+
def test_lower_search_any_where_emits_marker_prefilter() -> None:
2026+
"""viz-filter L2-b: WHERE searchAny(a, 'x'[, {opts}]) lifts to a search_any row
2027+
pre-filter + a fresh marker column in the residual boolean expression (the
2028+
pattern-predicate marker mechanism), composing through AND/OR/NOT."""
2029+
lowered = lower_match_query(_parse_query(
2030+
"MATCH (a) WHERE searchAny(a, 'foo') AND a.x > 1 RETURN a.id AS id"))
2031+
pre = [op for op in lowered.row_pre_filters
2032+
if isinstance(op, ASTCall) and op.function == "search_any"]
2033+
assert len(pre) == 1
2034+
assert pre[0].params["alias"] == "a"
2035+
assert pre[0].params["term"] == "foo"
2036+
marker = pre[0].params["out_col"]
2037+
assert marker.startswith("__gfql_search_any_")
2038+
assert lowered.row_where is not None and marker in lowered.row_where.text
2039+
assert "searchAny" not in lowered.row_where.text
2040+
2041+
lowered2 = lower_match_query(_parse_query(
2042+
"MATCH (a) WHERE searchAny(a, '7', {caseSensitive: true, regex: false, columns: ['name']}) RETURN a.id AS id"))
2043+
op2 = [op for op in lowered2.row_pre_filters
2044+
if isinstance(op, ASTCall) and op.function == "search_any"][0]
2045+
assert op2.params.get("case_sensitive") is True
2046+
assert op2.params.get("columns") == ["name"]
2047+
2048+
for q, phrase in [
2049+
("MATCH (a) WHERE searchAny(a, 'x', {nope: true}) RETURN a", "unsupported option"),
2050+
("MATCH (a) WHERE searchAny(zzz, 'x') RETURN a", "not bound"),
2051+
("MATCH (a) WHERE searchAny(a, 'x', {caseSensitive: 'yes'}) RETURN a", "true or false"),
2052+
]:
2053+
with pytest.raises(GFQLValidationError) as exc_info:
2054+
lower_match_query(_parse_query(q))
2055+
assert phrase in exc_info.value.message, (q, exc_info.value.message)
2056+
2057+
20252058
def test_exists_subquery_unsupported_bodies_decline_clearly() -> None:
20262059
"""viz-filter L1 v1 boundaries: inner WHERE, multi-pattern bodies, and full
20272060
MATCH..RETURN subquery bodies decline with a clear message (never a wrong answer)."""

graphistry/tests/compute/gfql/test_engine_polars_conformance_matrix.py

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -426,6 +426,31 @@ def q(**kw):
426426
g.gfql(q(term="x", columns=["nope"]), engine="pandas")
427427

428428

429+
def test_search_any_cypher_surface_all_engines():
430+
"""viz-filter L2-b: the cypher WHERE searchAny(...) surface — marker lift +
431+
composition with other predicates; oracle-pinned + 4-engine parity-or-NIE."""
432+
import pandas as pd
433+
nd = pd.DataFrame({
434+
"id": [0, 1, 2, 3],
435+
"name": ["Alpha", "beta", None, "gamma7"],
436+
"num": pd.Series([7, 77, 3, 4], dtype="int64"),
437+
})
438+
ed = pd.DataFrame({"s": [0, 1], "d": [1, 2], "eid": [0, 1]})
439+
g = graphistry.nodes(nd, "id").edges(ed, "s", "d").bind(edge="eid")
440+
oracle = {
441+
"MATCH (a) WHERE searchAny(a, 'ALPHA') RETURN a.id AS id": [0],
442+
"MATCH (a) WHERE searchAny(a, '7') RETURN a.id AS id": [0, 1, 3],
443+
"MATCH (a) WHERE searchAny(a, '7', {columns: ['name']}) RETURN a.id AS id": [3],
444+
"MATCH (a) WHERE searchAny(a, 'a.pha', {regex: true}) RETURN a.id AS id": [0],
445+
"MATCH (a) WHERE searchAny(a, 'a') AND a.num > 10 RETURN a.id AS id": [1],
446+
"MATCH (a) WHERE NOT searchAny(a, 'a') RETURN a.id AS id": [2],
447+
}
448+
for q, expected in oracle.items():
449+
pdf = _to_pd(g.gfql(q, engine="pandas")._nodes)
450+
assert sorted(pdf["id"].tolist()) == expected, f"oracle drift: {q}"
451+
_assert_invariant(g, q, f"searchAny-cypher {q}")
452+
453+
429454
def test_search_nodes_edges_twins():
430455
"""g.search_nodes/search_edges (persona-pass python twins): same kernel, own table
431456
only, Plottable out; polars frames decline honestly."""

0 commit comments

Comments
 (0)