Skip to content

Export a describeDialect() so consumers can retrieve the SQL surface instead of hand-maintaining it #51

Description

@bgmcmullen

Motivation

Downstream consumers (hypaware's hyp query CLI and its agent-facing skill docs) need to tell users and AI agents what SQL surface the engine supports: which functions exist, which cast types parse, what the statement surface is. Today that description has to be hand-maintained outside squirreling, and it drifts. We just caught a real case: docs written against an older squirreling still claimed CAST(... AS TIMESTAMP) and TIMESTAMP '...' literals do not exist (both work in 0.15.0), quoted a JSON_EXTRACT error message that has since changed, and implied REGEXP_EXTRACT's third argument selects a capture group (it is a 1-based position; the function always returns the full match).

Any copy of the dialect that lives outside the package describes some past version of it. The description should travel with the version pin.

Proposal

Export a describeDialect(options?) function from the public index that returns a JSON-able description of the SQL surface:

{
  version: '0.15.0',            // package version, for provenance
  statements: { select: true, with: true, withRecursive: false, setOps: ['UNION', 'INTERSECT', 'EXCEPT'] },
  castTypes: [ { canonical: 'STRING', aliases: ['TEXT', 'VARCHAR'] }, /* INT, BIGINT, FLOAT, BOOL, TIMESTAMP */ ],
  functions: { REGEXP_EXTRACT: { category: 'regexp', min: 2, max: 4, signature: 'string, pattern[, position[, occurrence]]' }, /* ... */ },
  operators: ['AND', 'OR', 'LIKE', '=', /* ... */],
  notes: [ /* the few semantic facts the tables cannot carry, see below */ ],
}

It should accept the same functions option that executeSql takes, so host-registered user-defined functions appear in the same inventory as built-ins.

The pieces already exist

Most of this is aggregation over data already in src/validation/functions.js:

  • FUNCTION_SIGNATURES is the master function registry (isKnownFunction and validateFunctionArgs check it) and is already an export const, just not re-exported from src/index.js.
  • Cast types are the literal array inside isCastType.
  • Categories are the literal arrays inside isAggregateFunc / isMathFunc / isStringFunc / isRegexpFunc / isWindowFunc / isSpatialFunc / isTableFunction / niladicFuncs.
  • The statement surface is implicit in src/parse/parse.js (SELECT-only, WITH without RECURSIVE, UNION/INTERSECT/EXCEPT).

Anti-drift requirement

Exporting these as-is would also export their skew: FUNCTION_SIGNATURES and the per-category predicate arrays are parallel hand-maintained structures. Either:

  1. invert the design: one registry with a category field per function, and derive the is*Func predicates and FUNCTION_SIGNATURES from it (preferred: removes the existing duplication, not just the new one), or
  2. keep the shape and add a test asserting the category lists and FUNCTION_SIGNATURES agree in both directions.

The description must be generated from the same structures the parser and validator enforce, or it becomes a second thing that can rot.

Semantic notes

A small hand-written notes list covers the facts no table encodes, colocated with the implementations that make them true:

  • REGEXP_EXTRACT / REGEXP_SUBSTR return the full match, never a capture group; the optional third and fourth arguments are 1-based position and occurrence.
  • Patterns are JavaScript RegExp syntax (so lookbehind works as a capture-group substitute).
  • JSON functions require valid JSON input and error on plain-string values.
  • IS TRUE / IS NOT TRUE are not parsed; only IS [NOT] NULL.
  • LIKE ... ESCAPE is not parsed.

Out of scope

Rendering. Downstream CLIs (e.g. a hyp query dialect verb) render the object as text or JSON themselves; squirreling only owns the data.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions