Motivation
Downstream consumers (hypaware's hyp query CLI and its agent-facing skill docs) need to tell users and AI agents what SQL surface the engine supports: which functions exist, which cast types parse, what the statement surface is. Today that description has to be hand-maintained outside squirreling, and it drifts. We just caught a real case: docs written against an older squirreling still claimed CAST(... AS TIMESTAMP) and TIMESTAMP '...' literals do not exist (both work in 0.15.0), quoted a JSON_EXTRACT error message that has since changed, and implied REGEXP_EXTRACT's third argument selects a capture group (it is a 1-based position; the function always returns the full match).
Any copy of the dialect that lives outside the package describes some past version of it. The description should travel with the version pin.
Proposal
Export a describeDialect(options?) function from the public index that returns a JSON-able description of the SQL surface:
{
version: '0.15.0', // package version, for provenance
statements: { select: true, with: true, withRecursive: false, setOps: ['UNION', 'INTERSECT', 'EXCEPT'] },
castTypes: [ { canonical: 'STRING', aliases: ['TEXT', 'VARCHAR'] }, /* INT, BIGINT, FLOAT, BOOL, TIMESTAMP */ ],
functions: { REGEXP_EXTRACT: { category: 'regexp', min: 2, max: 4, signature: 'string, pattern[, position[, occurrence]]' }, /* ... */ },
operators: ['AND', 'OR', 'LIKE', '=', /* ... */],
notes: [ /* the few semantic facts the tables cannot carry, see below */ ],
}
It should accept the same functions option that executeSql takes, so host-registered user-defined functions appear in the same inventory as built-ins.
The pieces already exist
Most of this is aggregation over data already in src/validation/functions.js:
FUNCTION_SIGNATURES is the master function registry (isKnownFunction and validateFunctionArgs check it) and is already an export const, just not re-exported from src/index.js.
- Cast types are the literal array inside
isCastType.
- Categories are the literal arrays inside
isAggregateFunc / isMathFunc / isStringFunc / isRegexpFunc / isWindowFunc / isSpatialFunc / isTableFunction / niladicFuncs.
- The statement surface is implicit in
src/parse/parse.js (SELECT-only, WITH without RECURSIVE, UNION/INTERSECT/EXCEPT).
Anti-drift requirement
Exporting these as-is would also export their skew: FUNCTION_SIGNATURES and the per-category predicate arrays are parallel hand-maintained structures. Either:
- invert the design: one registry with a
category field per function, and derive the is*Func predicates and FUNCTION_SIGNATURES from it (preferred: removes the existing duplication, not just the new one), or
- keep the shape and add a test asserting the category lists and
FUNCTION_SIGNATURES agree in both directions.
The description must be generated from the same structures the parser and validator enforce, or it becomes a second thing that can rot.
Semantic notes
A small hand-written notes list covers the facts no table encodes, colocated with the implementations that make them true:
REGEXP_EXTRACT / REGEXP_SUBSTR return the full match, never a capture group; the optional third and fourth arguments are 1-based position and occurrence.
- Patterns are JavaScript
RegExp syntax (so lookbehind works as a capture-group substitute).
- JSON functions require valid JSON input and error on plain-string values.
IS TRUE / IS NOT TRUE are not parsed; only IS [NOT] NULL.
LIKE ... ESCAPE is not parsed.
Out of scope
Rendering. Downstream CLIs (e.g. a hyp query dialect verb) render the object as text or JSON themselves; squirreling only owns the data.
Motivation
Downstream consumers (hypaware's
hyp queryCLI and its agent-facing skill docs) need to tell users and AI agents what SQL surface the engine supports: which functions exist, which cast types parse, what the statement surface is. Today that description has to be hand-maintained outside squirreling, and it drifts. We just caught a real case: docs written against an older squirreling still claimedCAST(... AS TIMESTAMP)andTIMESTAMP '...'literals do not exist (both work in 0.15.0), quoted aJSON_EXTRACTerror message that has since changed, and impliedREGEXP_EXTRACT's third argument selects a capture group (it is a 1-based position; the function always returns the full match).Any copy of the dialect that lives outside the package describes some past version of it. The description should travel with the version pin.
Proposal
Export a
describeDialect(options?)function from the public index that returns a JSON-able description of the SQL surface:It should accept the same
functionsoption thatexecuteSqltakes, so host-registered user-defined functions appear in the same inventory as built-ins.The pieces already exist
Most of this is aggregation over data already in
src/validation/functions.js:FUNCTION_SIGNATURESis the master function registry (isKnownFunctionandvalidateFunctionArgscheck it) and is already anexport const, just not re-exported fromsrc/index.js.isCastType.isAggregateFunc/isMathFunc/isStringFunc/isRegexpFunc/isWindowFunc/isSpatialFunc/isTableFunction/niladicFuncs.src/parse/parse.js(SELECT-only, WITH without RECURSIVE, UNION/INTERSECT/EXCEPT).Anti-drift requirement
Exporting these as-is would also export their skew:
FUNCTION_SIGNATURESand the per-category predicate arrays are parallel hand-maintained structures. Either:categoryfield per function, and derive theis*Funcpredicates andFUNCTION_SIGNATURESfrom it (preferred: removes the existing duplication, not just the new one), orFUNCTION_SIGNATURESagree in both directions.The description must be generated from the same structures the parser and validator enforce, or it becomes a second thing that can rot.
Semantic notes
A small hand-written
noteslist covers the facts no table encodes, colocated with the implementations that make them true:REGEXP_EXTRACT/REGEXP_SUBSTRreturn the full match, never a capture group; the optional third and fourth arguments are 1-based position and occurrence.RegExpsyntax (so lookbehind works as a capture-group substitute).IS TRUE/IS NOT TRUEare not parsed; onlyIS [NOT] NULL.LIKE ... ESCAPEis not parsed.Out of scope
Rendering. Downstream CLIs (e.g. a
hyp query dialectverb) render the object as text or JSON themselves; squirreling only owns the data.