simple-builder 3.0.0: sql`` tagged template, real SQL lexer, identifier safety, TS rewrite, dual CJS/ESM#5
Open
Acro wants to merge 8 commits into
Open
simple-builder 3.0.0: sql`` tagged template, real SQL lexer, identifier safety, TS rewrite, dual CJS/ESM#5Acro wants to merge 8 commits into
Acro wants to merge 8 commits into
Conversation
Full modernization of the library while preserving the public API and byte-for- byte output for all documented usage (proven by a differential fuzzer against the vendored 2.4.2 builder over 10k+ seeded scenarios). Packaging & tooling - Reimplement in TypeScript (src/index.ts); ship compiled CJS (dist/) plus a static ESM wrapper (esm/) via an exports map. `main`/`types` updated. - Add first-class, complete type definitions (objects, Date, bigint, null, varargs, projection arrays, optional `values`) — the old index.d.ts only typed scalar arrays. - tsconfig, engines (Node >=16), files allowlist, sideEffects, richer metadata. - GitHub Actions CI (Node 16–24 matrix, differential fuzz, package-boundary smoke for CJS+ESM, typings gate) and a provenance publish workflow. - Add LICENSE, AGENTS.md, CLAUDE.md, llms.txt, .gitignore, issue templates. Tests (replacing the old console.log script) - Assertion-based unit suite, exact type-level tests, package-boundary smoke tests (installed tarball, CJS + ESM), and a seeded differential fuzzer. Bug fixes (behaviour changes vs 2.4.2, covered by unit tests) - Lowercase `in ?` is now expanded; the matcher was case-insensitive but the substitution used a hard-coded uppercase ` IN ?`, so lowercase left a stray placeholder and mis-bound values. - The caller's partials array is no longer mutated (projection join wrote back into the input array). - An empty object passed to VALUES ?/SET ?/WHERE ? now throws a clear error instead of emitting invalid SQL like `() VALUES ()`. - Array detection uses Array.isArray; error messages are clearer. Docs - Rewrite README with current async/await, a Security section documenting that object KEYS are interpolated as identifiers (values remain parameterised). Bump to 3.0.0 (packaging/module-format change; API-compatible). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Both surfaced by the packaging research: publint's one suggestion (declare the package type so Node skips type detection) and a strict publint + attw gate in the package-boundary CI job. `publint --strict` is now clean and attw reports no problems across node10 / node16-CJS / node16-ESM / bundler resolution. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implements the security / correctness / DX roadmap from the architecture review
into the unreleased 3.0.0, keeping the library one file and zero-dependency
(~16 kB compiled). The `?` partials API stays byte-identical to 2.4.2 for all
documented usage — the differential fuzzer proves it on every run.
SECURITY — identifiers are the only injection vector, since no driver can bind
an identifier (OWASP: placeholders name data, never tables/columns).
- Object keys in VALUES ?/SET ?/WHERE ? are now allow-listed to plain
identifiers (`col`, `tbl.col`) and throw otherwise. Legitimate keys pass
through byte-for-byte, so this changes output for nothing except payloads.
Chosen over auto-quoting, which would silently break case semantics (quoted
identifiers are case-sensitive; Postgres folds unquoted names to lower case).
- NEW `sql.id(...parts)` quotes dynamic identifiers per dialect ("pg" / `mysql`)
and doubles embedded quotes; the helper adds its own quotes so callers cannot
create a quote mismatch. Rejects NUL and empty names.
- README/llms.txt document the residual footguns the library cannot fix for you:
LIKE wildcards in bound values, and second-order injection.
CORRECTNESS — replace the regex/flag state machine with a real lexer.
A naive `?` scan corrupts real SQL. Previously:
pg(["... WHERE tags ?| ARRAY['x'] AND id = ?", 5])
→ "... WHERE tags $1| ARRAY['x'] AND id = ?" // operator destroyed, value misbound
The lexer now skips string literals ('' and MySQL \ escapes), quoted identifiers
("" and ``), line/block comments (nested), and dollar-quoted bodies ($$…$$ /
$tag$…$tag$ — the pgx injection class); treats `?|`/`?&`/`??` as operators; and
honours `\?` as an escaped literal `?` for the ambiguous bare jsonb operator.
Clause markers are now classified positionally with `\b`, which also fixes
`OFFSET ?` being misread as `SET ?`. Too few values now throws instead of
silently binding undefined.
DX — NEW sql`` tagged template, the recommended API.
Every ${interpolation} is always a bound parameter, so accidental injection is
structurally impossible; nothing is scanned for `?`, so jsonb operators need no
escaping. Fragments nest and compose; arrays expand to `IN ($1,$2)`. Helpers:
sql.id / sql.value / sql.join / sql.raw / sql.empty. Both APIs lower to one node
representation and render through one Renderer, so `pg`/`mysql` accept either.
Rendering is pure — a fragment renders repeatedly, for either dialect.
TESTS — 22 → 50 unit tests, plus a second fuzzer.
The `sql` tag and lexer have no 2.4.2 counterpart, so they get property-based
fuzzing over injection-shaped inputs: values bind in order and never reach the
text, pg renders exactly $1..$n, mysql renders n `?`, and rendering is pure.
20k differential + 20k property scenarios per CI run. Also gates publint --strict
and are-the-types-wrong in CI and on publish.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Self-review of the new lexer found a real bug: Postgres E'...' escape strings honour backslash escapes, but the lexer only did so for MySQL. So pg(["SELECT E'a\\'b ?' AS s FROM t WHERE id = ?", 1]) ended the string early at the escaped quote, bound the value to the `?` INSIDE the string literal, and left the real `id = ?` as a literal question mark: before: "SELECT E'a\\'b $1' AS s FROM t WHERE id = ?" after : "SELECT E'a\\'b ?' AS s FROM t WHERE id = $1" Same class of gap for MySQL's backslash-escaped "..." strings. Quote runs now go through one `consumeQuoted` helper that takes an `escapes` flag, set per dialect and per quote style: - MySQL: backslash escapes in both '...' and "..." - Postgres: backslash escapes ONLY in an E'...' string; a standard string treats `\` as an ordinary character (standard_conforming_strings), and a "..." identifier uses "" doubling alone. Adds 3 regression tests (50 → 53) covering both dialects and both directions (escapes honoured where they apply; `\` ordinary where it doesn't). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
All three reproduced before fixing; each now has a regression test.
1. Marker classes enumerated as a Row at a clause-marker value position
(src/index.ts). Sql/Identifier/Raw/Single are plain objects, and the
`clause && isObject(value)` branch ran before the instanceof checks, so:
pg(['SELECT * FROM t WHERE ?', sql.id('foo')])
→ { text: 'SELECT * FROM t WHERE parts=$1', values: [['foo']] }
The marker's own field names (parts/nodes/value/text) all satisfy
PLAIN_IDENTIFIER, so nothing threw — it silently emitted wrong SQL against
fabricated columns. Same for VALUES ?/SET ?/IN ? and for raw/value/tag.
Fixed two ways (independent guards): the instanceof branches now run first,
and isObject excludes marker instances via a new isMarker predicate.
This also surfaced a related gap the review noted: an sql`` fragment at a
value position had NO branch at all and was bound as raw node data, even
without a clause marker. It now splices via a shared renderNodes(), so
placeholder numbering stays continuous:
pg(['SELECT * FROM t WHERE x = ?', 9, 'AND ?', sql`a = ${1}`, 'AND b = ?', 2])
→ 'SELECT * FROM t WHERE x = $1 AND a = $2 AND b = $3'
`build()` now uses isMarker too, so pg(sql.value(x)) no longer stringifies
to "[object Object]".
2. MySQL `#` line comments were not lexed (src/index.ts). A `?` inside one was
counted as a placeholder, so valid MySQL either threw a bogus
"3 placeholder(s) but only 2 value(s)" or silently misaligned parameters.
Gated on dialect: Postgres has no `#` comment — it starts the #> and #-
jsonb operators, which are covered by a test to prevent an over-eager fix.
3. Script injection in publish.yml: the release tag_name was interpolated into
a bash script body. `${{ }}` expands before bash runs and git permits $ ` " ;
| in tag names, so a crafted tag could execute shell in a job holding
id-token: write and NPM_TOKEN. Now passed via env: and referenced as $TAG.
Low severity (requires contents: write to cut a release) but it is the
documented anti-pattern and the fix is free.
Tests: 53 → 58 unit tests, plus a third fuzzer. The existing fuzzers could not
have caught #1 — the tag fuzzer never routes a marker through buildPartials.
The new marker fuzzer does, asserting a marker's internal field names never
surface as SQL identifiers. Verified it earns its place by reintroducing the
original bug and confirming it fails:
MARKER PROPERTY FAILURE: marker field "nodes" surfaced as a column:
UPDATE t SET nodes=$1
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… bug
The unit suite asserts on the generated { text, values }, which only ever proves
the builder agrees with OUR reading of the SQL grammars. Both lexer bugs found
so far were exactly that kind of misreading, so this suite executes the built
queries against real servers and asserts on the returned rows.
It immediately earned its place by finding a third one:
MySQL treats `--` as a comment ONLY when followed by whitespace or EOF.
Verified on MySQL 8.4: `SELECT 1--2` is 3 (two minus signs), and
`SELECT 1--?` really does bind the placeholder. The lexer treated `--`
as a comment unconditionally, so it swallowed the `?`:
mysql(['SELECT 1--? AS v', 2])
→ Error: expected an SQL string fragment but got number at position 1
Postgres, by contrast, always treats `--` as a comment — confirmed against
PG 16, where `SELECT 1--$1 AS v\n, $1 ::int AS n` comments out the first
and binds the second. So the rule is dialect-gated, like `#`.
Verified the regression test earns its place by reverting the fix and watching
it fail with exactly that error.
test/integration.cjs — 27 tests over Postgres 16 + MySQL 8.4:
- INSERT/UPDATE/WHERE/IN object expansion round-trips (assert on rows)
- $1..$n numbering correct across mixed clauses
- injection payloads stored as literal data, table still standing
- jsonb ?| ?& and the bare ? (via \? escape, and via the tag with no escape)
- #> / #- are operators, not comments
- ? inside string literals, dollar-quotes ($$ and $tag$), and comments
- pg E'…' backslash escapes vs standard_conforming_strings literal backslash
- MySQL # comments, `--` whitespace rule, backslash escapes in both quote styles
- sql.id quoting: case-sensitivity, embedded-quote escaping, schema qualifying
- sql tag composition and fragments spliced at a ? value position
- LIKE wildcards in a bound value (pins the documented behaviour)
Tooling:
- pg + mysql2 as devDependencies ONLY — the published tarball still ships zero
dependencies (verified: installed package's `dependencies` is {}).
- npm run test:integration; npm run db:up / db:down for local docker servers.
- CI job with postgres:16 and mysql:8 service containers.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Closes the last risk flagged: the integration suite only pinned DEFAULT server
configuration, and three settings invert assumptions the lexer encodes.
Measured against MySQL 8.4 / PG 16 (each contradicts a plausible assumption):
- MySQL ANSI_QUOTES: "…" is an identifier escaped by "" doubling — `"a\"b"` is
a SYNTAX ERROR there, so backslash escaping must be off for `"`.
- MySQL NO_BACKSLASH_ESCAPES: `\` is ordinary in literals ('a\' is a\).
- pg standard_conforming_strings=off: `\` escapes inside '…' — the exact
OPPOSITE of the default the lexer assumed.
- `''` doubling lexes identically in EVERY mode of both engines. It is the only
mode-proof escape; the sql tag never lexes, so it is mode-proof too.
There is no conservative rule that satisfies both readings — the placeholder
count genuinely differs — so the builder has to be told. Added a minimal,
default-inert opt-in:
const my = mysql.withMode({ ansiQuotes: true, noBackslashEscapes: true })
const pgOld = pg.withMode({ standardConformingStrings: false })
withMode returns a NEW builder (original untouched) and merges when chained.
Only affects the `?` partials API. Needed only when your SQL puts a backslash
immediately before a quote inside a literal.
Tests: 30 matrix tests (4 MySQL modes × 2 pg modes) that set the real SESSION to
the mode AND configure the builder to match, so any disagreement fails. Each
mode asserts: '' doubling is mode-proof, identifiers work, the tag is mode-proof,
object expansion round-trips, and backslash-in-literal follows the mode.
Plus 6 unit tests + type/boundary coverage for the new surface. 58 → 64 unit.
Also switches the MySQL integration path from query() to execute(), which found
a driver-level fact worth documenting: mysql2's query() substitutes values
CLIENT-side with backslash escaping, so under NO_BACKSLASH_ESCAPES it errors and
round-trips `x\` as `x\\`. execute() binds server-side and is correct in every
mode. Using execute() also means MySQL's own parser — not mysql2's scanner —
validates our text. README/llms.txt now tell MySQL users to prefer execute().
Corrected an overclaim while verifying: the server is NOT a full oracle for
MySQL. Measured — pg rejects too-few AND too-many params; MySQL execute()
rejects too-few but silently ignores extras. So the tests assert on returned
rows, not merely that the query ran. Recorded in AGENTS.md.
Published package still ships zero dependencies (verified: {}).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three surfaces an agent actually reads — llms.txt in the tarball, the shipped .d.ts via LSP, and runtime errors — audited and closed. Executable documentation (test/docs.test.cjs, wired into npm test) - Parses every ```javascript block in README.md and llms.txt, runs it against the build, and compares with the documented result. Docs are what agents copy verbatim, so drift is now a build failure rather than a silent lie. - 26 examples verified (36 evaluated). Verified the test earns its place by drifting a recipe and watching it fail with a precise diff. - Rejects ellipses in expected values, which caught a real doc bug: the INSERT example documented `values: [...]` — an agent would have copied that literally. It now shows the real values. - A floor on the checked count guards against the parser silently matching nothing and "passing" vacuously. llms.txt — added a Recipes section (12 machine-verified, self-contained) Previously it had no runnable code at all, so an agent had nothing to copy. Covers: value binding, conditional WHERE via sql.empty, sql.join for a variable number of clauses, IN lists, INSERT..RETURNING, UPDATE from an object, dynamic identifiers via sql.id, dynamic ORDER BY via allow-list + sql.raw, pagination, LIKE with wildcard escaping, jsonb `?` in the tag, and one fragment rendered for both dialects. Shipped .d.ts — added 8 @example blocks on pg / mysql / sql / id / raw / value / join, so an LSP hover teaches the safe pattern at the moment of use (e.g. sql.raw's example shows the allow-list first; sql.id's notes the case-sensitivity trap; mysql's points at execute() over query()). Discoverability - README now points at llms.txt and says where it lands on disk. - GitHub repo had NO description, topics, or homepage — all three set. - npm keywords already cover the tagged-template/injection/dialect surface. Verified after a real install: llms.txt (228 lines), the @example-carrying d.ts, and `dependencies: {}` are all present in node_modules/simple-builder. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
A ground-up modernization of
simple-builderfor 2026, staying true to the original purpose: your SQL stays visible. Still one file, still zero dependencies, ~16 kB compiled.The
?partials API is byte-identical to 2.4.2 for all documented usage — a differential fuzzer proves it against the vendored original on every CI run (20k scenarios). Existingrequire('simple-builder').pgcallers are unaffected.Three things changed: the SQL is now lexed instead of
?-counted, identifiers are safe by default, and there's a newsql``tagged template that makes injection structurally impossible.1. Security — identifiers are the only real vector
Values were always parameterised and safe. Identifiers are the problem: no driver can bind an identifier (OWASP — placeholders name data, never tables or columns), and object keys become column names.
col,tbl.col); anything else throws. Legitimate keys pass through byte-for-byte, so this changes output for nothing except payloads:{ userName: … }would stop matching ausernamecolumn and break existing consumers.sql.id(...parts)for the dynamic case — quotes per dialect ("pg"/`mysql`), doubles embedded quotes, and adds its own quotes so callers can't create a quote mismatch. Rejects NUL/empty.%/_still match), and second-order injection.2. Correctness — a real lexer replaces the
?scannerNaive
?-counting corrupts real SQL. Reproduced on the old code:This is the exact bug class that broke Aura.SqlQuery. The lexer now skips:
''escapes, plus MySQL\escapes)"…"and`…`)$$…$$,$tag$…$tag$) — the pgx injection class…recognises
?|/?&/??as operators, and honours\?as an escaped literal?for the genuinely-ambiguous bare jsonb?. Clause markers are now classified positionally with\b, which also fixesOFFSET ?being misread asSET ?. Too few values now throws instead of silently bindingundefined.3. DX — the new
sqltagged template (recommended)Every
${interpolation}is always a bound parameter — you cannot forget to parameterise, so accidental injection is structurally impossible. Nothing is scanned for?, so jsonb operators need no escaping. Fragments nest and compose:Helpers:
sql.id/sql.value/sql.join/sql.raw(unsafe by construction, documented) /sql.empty.Both APIs lower to one node representation and render through one
Renderer, sopg()/mysql()accept either. Rendering is pure — a fragment renders repeatedly, for either dialect.Packaging & tooling
dist/) + static ESM wrapper (esm/) via anexportsmap; no dual-package hazard. Complete types (the oldindex.d.tsonly typed scalar arrays).publint --strictclean;attwgreen across node10 / node16-CJS / node16-ESM / bundler — both now gate CI and publish.Tests
50 unit tests (from 22), exact type-level tests, CJS+ESM boundary tests against the installed tarball, and two fuzzers:
pgrenders exactly$1..$n,mysqlrendersn?, rendering is pure.Behaviour changes vs 2.4.2 (all unit-tested)
Each is a fix; all are kept out of the differential fuzzer's domain:
in ?now expands (was left as a literal placeholder, mis-binding values).VALUES ?/SET ?/WHERE ?throws instead of emitting() VALUES ().undefined.OFFSET ?is no longer misread asSET ?.?inside literals/identifiers/comments/dollar-quotes is no longer a placeholder;?|/?&survive.Verification
npm test→ 50/50 + type gate.npm run fuzz -- 20000→ 20k differential scenarios agree with 2.4.2, 20k property scenarios hold all invariants. Packed tarball passes CJS+ESM boundary tests and the latest-TypeScript typings gate undernode16resolution.publint --strictandattwclean.Bumped to 3.0.0 — API-compatible for documented usage; major for the packaging/module-format change and the behaviour fixes above.
🤖 Generated with Claude Code