GFQL Cypher dialect: host the streamgl-viz filter language (regex/case, floor/ceil/round/pow, prune-isolated, two-mask via GRAPH{})

[lmeyerov]

Goal

Extend the GFQL Cypher dialect so it can be the single target for a (net-new) streamgl-viz filter/exclusion → Cypher translation layer, running on all four engines (pandas, cudf, polars, polars-gpu). Strategy: extend the Cypher flavor (we own it — GRAPH { … } constructor, CALL … YIELD procedures incl. graph-level, growable WHERE grammar + scalar-fn allow-list), rather than route around it to raw GFQL-AST.

Grounded in a per-engine conformance analysis of the viz filter AST (~/Work/graphistry2/apps/core/viz + forge shaping) vs the GFQL engine.

Frontend (dialect) gaps

• G1 — regex operator =~ (openCypher-standard; + optional LIKE/ILIKE) → lower to match/fullmatch/contains(regex=True); case-insensitivity via inline (?i) and/or a case flag. Flagship viz gap — Cypher WHERE has no regex/LIKE/ILIKE/case-insensitivity today.
• G2 — scalar functions floor / ceil / round / pow → add to GFQL_SCALAR_FUNCTIONS + evaluator (all engines). Currently absent from the GFQL language entirely (rejected on every engine).
• G3 — graph-level CALL graphistry.pruneIsolated() + graphistry.pruneEdgelist() (graph_only procedures, like the existing networkx.* graph algos). Match viz semantics: pruneEdgelist = unconditional both-endpoint induced subgraph; pruneIsolated = opt-in, undirected degree, drops self-loop-only nodes, re-prunes. (No "prune isolated" op exists today; a [n,e,n] chain drops orphans but that's the opposite of viz's keep-by-default default and isn't attribute-preserving.)
• G4 — two-mask composition via GRAPH { … } — viz produces two independent row masks (node ⟂ edge) + prune; express as a GRAPH pipeline. May require closing the "GRAPH bindings + UNION" gap (parser) — scope carefully; the prune CALLs may make a single-pipeline form sufficient.

Backend (per-engine execution) prerequisites — so the extended Cypher actually runs everywhere

• B1 — polars contains ignores regex=/flags= (proven: contains('B.R', regex=False) matched 'BAR.baz'). Literal filters with regex metacharacters over-match on polars. Correctness bug — fix first.
• B2 — polars Match/Fullmatch → NotImplementedError. Add native lowering (needed for G1 on polars).
• B3 — polars CASE / sqrt / sign → NotImplementedError. Add native lowering.
• B4 — GPU verify (cudf + polars-gpu): regex + inline (?i) execution, and B1–B3 parity. dgx-only; keep tiny.

Semantics to preserve

• S1 — null inclusion: viz filters keep nulls by default; GFQL predicates default na=False (exclude). Translation must OR-in IS NULL (or add a dialect knob). Easy to miss.
• S2 — regex dialect: viz emits JavaScript RegExp; engines use Python re / cuDF / Rust regex. Common patterns port; validate/reject JS-only constructs (lookbehind, named-group syntax).

Engine support today (summary)

Comparisons, AND/OR/NOT, arithmetic, IS NULL, IN, BETWEEN, Contains/StartsWith/EndsWith (incl. case-insensitive) — native on pandas/cudf/polars. Match/Fullmatch, CASE, sqrt, sign — pandas/cudf only (polars NIE). floor/ceil/round/pow — not in GFQL at all. GPU regex/case execution unverified.

Approach

Incremental stacked PRs, each CPU-green + differential-parity-tested (pandas oracle), honest NotImplementedError (no silent bridges); GPU parity deferred to a small dgx pass. Suggested order: B1 → B2/B3 → G1 → G3 → G2 → G4 → S1/S2 → dgx verify.

Full working plan + file:line extension points tracked locally (plan skill).

---

SPEC ADDENDUM — standards vetting vs neo4j Cypher 25 + ISO GQL (2026-07-02)

Deep re-vet of every shipped/planned extension against primary sources: the current neo4j Cypher manual, neo4j's shipped Cypher 5/25 parser source, the openCypher 9 EBNF + TCK, and the ISO-derived GQL grammar (opengql/grammar) + neo4j's GQL-conformance appendix. Verdicts:

construct	neo4j Cypher (current)	ISO GQL 39075:2024	our verdict
=~ regex	✅ standard, not deprecated; Java regex, inline (?i)(?m)(?s); full-string match	❌ (listed by neo4j as beyond-GQL)	CONFORMANT (Cypher extension) — keep
round() ties	precision 0: ties toward +∞ (round(-1.5)=-1.0, manual Ex. 8/10); precision >0: ties away from zero (HALF_UP, round(-1.55,1)=-1.6, Ex. 11). (Args-table "(default: UP)" contradicts Ex. 10 → neo4j docs bug.)	❌ no ROUND in §20.22	DIVERGED → FIXED (we shipped numpy half-to-even; round(2.5) gave 2.0 vs neo4j 3.0)
ceil/ceiling	✅ ceiling = Cypher 25 GQL-conformance alias	✅ grammar (CEIL|CEILING)	CONFORMANT — keep both
toLower/toUpper + lower/upper	✅ both; lower/upper are GQL-conformance aliases (present since Cypher 5 manual)	✅ lower/upper mandatory §20.24	ADDED aliases lower/upper
^ power	Manual says "right to left" but the shipped parser left-folds Pow (Cypher25Parser.g4 expression4: expression3 (POW expression3)* + astBinaryFold=foldLeft) → 2^3^2 = 64. openCypher TCK also pins left (4^(3*2)^3 = (4^6)^3) and unary-tighter (-3^2 = 9.0)	❌ no ^; POWER(x,y) fn (GF03)	SETTLED: left-associative. Manual's "right to left" = neo4j docs bug. Re-add = coordinated tck-corpus xfail→pass change (deferred)
chained a<b<c	✅	❓	CONFORMANT (already worked)
XOR	✅	✅ GQL HAS XOR (optional GE07; precedence groups with OR)	CORRECTION to earlier note ("GQL lacks XOR" was wrong)
LIKE/ILIKE/BETWEEN	❌	❌ re-verified hard: GQL's only LIKE is CREATE GRAPH TYPE … LIKE g (DDL false-positive); only BETWEEN token is DURATION_BETWEEN()	DECLINE STANDS — viz layer lowers (LIKE→=~/STARTS WITH, BETWEEN→a<=x AND x<=b)
GRAPH g = GRAPH { MATCH … }	n/a	GQL §10.1 GRAPH g = <graph expression> exists but takes references only (graphRef / CURRENT_GRAPH); no inline anonymous constructor with a MATCH body	NON-STANDARD EXTENSION — outer GRAPH g = … surface matches GQL §10.1; the { MATCH … } literal body is ours. Document as extension (closest std: CREATE GRAPH … AS COPY OF + INSERT)
CALL ns.proc() YIELD	✅ standard	✅ GQL §15.1–15.3 named procedure calls incl. YIELD (GP04), plus OPTIONAL CALL	CONFORMANT
CREATE GFQL INDEX	neo4j's own CREATE INDEX is listed beyond-GQL	❌ GQL defines no index DDL at all	NON-STANDARD, nothing to align to — vendor DDL (same posture as neo4j)

Actions taken

• Fixed round() tie-breaking to neo4j semantics (PR fix(gfql/cypher): neo4j round() tie-breaking + lower/upper GQL aliases (#1673) #1677) (pandas/cuDF + polars, parity-tested against the manual's example values) — was a wrong answer on every positive tie.
• Added lower()/upper() GQL-conformance aliases (PR fix(gfql/cypher): neo4j round() tie-breaking + lower/upper GQL aliases (#1673) #1677) (as neo4j does).
• Spec notes above correct the earlier research on XOR (GQL has it) and settle ^ as left-associative for the eventual re-add.
• Docs (cypher.rst) + CHANGELOG updated to the vetted semantics.

Full citations in the working notes (neo4j manual pages, Cypher25Parser.g4/ExpressionBuilder.scala, TCK Precedence2.feature, GQL.g4, GQL-conformance appendix).

[lmeyerov]

Standards vetting complete (2026-07-02) — issue body now carries a SPEC ADDENDUM with the full verdict table (neo4j Cypher 25 + ISO GQL, primary sources incl. neo4j's shipped parser source and the ISO-derived GQL grammar).

Headlines:

• round() tie-breaking diverged from neo4j (we had half-to-even; neo4j = ties toward +∞ at precision 0, away-from-zero at precision >0) → fixed in fix(gfql/cypher): neo4j round() tie-breaking + lower/upper GQL aliases (#1673) #1677 (stacked on feat(gfql/cypher): standard =~ regex + numeric fns + toLower/toUpper (#1673) #1675), parity-tested pandas+polars against the manual's example values.
• lower()/upper() added as GQL-conformance aliases (ISO GQL mandatory §20.24; matches neo4j) → also fix(gfql/cypher): neo4j round() tie-breaking + lower/upper GQL aliases (#1673) #1677.
• ^ associativity settled: LEFT (openCypher TCK + neo4j's shipped parser both left-fold; the manual's "right to left" is a docs bug) — recorded for the deferred re-add, which needs a coordinated tck-corpus xfail→pass change.
• Corrections/records: GQL does have XOR (GE07); LIKE/BETWEEN re-verified absent from GQL; GRAPH g = GRAPH { … } and CREATE GFQL INDEX are documented extensions (GQL §10.1 takes graph references only; GQL defines no index DDL).

Current stack: #1660 (polars engine) ← #1674 (polars contains regex fix) ← #1675 (=~ + numeric fns + toLower/toUpper, incl. cuDF (?i) fix from the dgx pass) ← #1677 (this vetting's fixes).