feat: introduce pine variables (|= syntax)

[ahmadnazir]

Summary

Introduces variables to the pine language via a |= assignment syntax. A variable is a named pine expression that can be referenced by name in subsequent expressions.

Syntax

company | where: active = true |= active_companies
active_companies | employee | s: name

Both |= and | = (with optional space) are valid.

How it works

Instead of sending a single expression, the API now accepts a list of pine expressions evaluated sequentially. Variables defined in earlier expressions are available in later ones. The backend is stateless — no global state is touched.

{
  "expressions": [
    "company | where: active = true |= active_companies",
    "active_companies | employee | s: name"
  ]
}

Only the last expression's result is returned.

Design: Variables as Nested ASTs

A variable is a first-class entry in :tables. Real table entries look like {:table "company" :schema "public" :alias "c_0"}. Variable entries carry an additional :ast field — its presence is the discriminator:

{:table "active_companies" :schema nil :ast <nested-AST> :alias "ac_0"}

There is no separate :variable flag. :ast present → variable. :ast absent → real table.

A top-level :variables {} map is maintained in the AST state and populated incrementally as assignments are processed. This is included in the API response for debugging and frontend use.

Join resolution

The existing join-helper / join-tables code requires zero changes. Before the main pipeline runs, pre-handle augments the local :references map with entries for each variable:

1. Inspect the variable's nested AST :columns
2. Filter out auto-id columns to find explicitly selected columns
3. If none explicitly selected → use the :current table only; otherwise trace each column's :alias through :aliases to find all source real tables
4. For each source real table, copy (get-in refs [:table source-table]) into (get-in refs [:table varname])

The join system then sees active_companies exactly as it sees company — no special casing.

SQL generation

Variable tables emit CTEs. build-select-query is refactored into:

• build-bare-select — generates SELECT ... FROM ... WHERE ... (no CTE prefix), using the variable name (no schema) for variable tables in FROM/JOIN clauses
• collect-ctes — recurses into nested variable ASTs in topological order, deduplicates by name, returns a flat list of [name bare-sql] pairs
• build-cte-body — like build-bare-select but strips the column-alias from the current table's auto-id column so it is exposed as plain id in the CTE (required for join conditions like active_companies.id = employee.company_id to resolve)
• build-select-query — wraps the above: collects CTEs, prepends WITH ... AS (...) clause

Composed variables (variable referencing another variable) are handled naturally: collect-ctes recurses, all CTEs flatten to a single top-level WITH clause (PostgreSQL does not allow nested WITH).

Files changed

File	Change
src/pine/pine.bnf	Add ASSIGN rule: |= symbol as optional tail on OPERATIONS
src/pine/parser.clj	Normalize ASSIGN node; extract :assign from ops list, return as separate key so it never enters handle-ops
src/pine/ast/main.clj	Add :variables {} and :assign nil to default state; extend pre-handle to accept variables and seed references; record :assign after handle-ops; keep :variables through post-handle
src/pine/ast/table.clj	Check :variables first in handle; variable entries carry :ast; join resolution unchanged
src/pine/eval.clj	collect-ctes, build-cte-body, build-bare-select, updated build-select-query; FROM/JOIN use CTE name for variable tables
src/pine/api.clj	api-eval and api-build accept expressions: [...]; sequential evaluation with local variable map; return last expression's result

Related

• Frontend PR: beamlynx/beamlynx-ui#(TBD) — multi-expression editor and variable list UI

🤖 Generated with Claude Code

[Koziar]