Skip to content

simple-builder 3.0.0: sql`` tagged template, real SQL lexer, identifier safety, TS rewrite, dual CJS/ESM#5

Open
Acro wants to merge 8 commits into
masterfrom
modernize-2026
Open

simple-builder 3.0.0: sql`` tagged template, real SQL lexer, identifier safety, TS rewrite, dual CJS/ESM#5
Acro wants to merge 8 commits into
masterfrom
modernize-2026

Conversation

@Acro

@Acro Acro commented Jul 14, 2026

Copy link
Copy Markdown
Owner

Overview

A ground-up modernization of simple-builder for 2026, staying true to the original purpose: your SQL stays visible. Still one file, still zero dependencies, ~16 kB compiled.

The ? partials API is byte-identical to 2.4.2 for all documented usage — a differential fuzzer proves it against the vendored original on every CI run (20k scenarios). Existing require('simple-builder').pg callers are unaffected.

Three things changed: the SQL is now lexed instead of ?-counted, identifiers are safe by default, and there's a new sql`` tagged template that makes injection structurally impossible.


1. Security — identifiers are the only real vector

Values were always parameterised and safe. Identifiers are the problem: no driver can bind an identifier (OWASP — placeholders name data, never tables or columns), and object keys become column names.

  • Object keys are now allow-listed to plain identifiers (col, tbl.col); anything else throws. Legitimate keys pass through byte-for-byte, so this changes output for nothing except payloads:
    pg(['UPDATE users SET ?', { 'x=1; DROP TABLE users; --': 1 }])  // now throws
  • Chosen over auto-quoting deliberately. Quoting would silently break case semantics — quoted identifiers are case-sensitive while Postgres folds unquoted names to lower case, so auto-quoting { userName: … } would stop matching a username column and break existing consumers.
  • New sql.id(...parts) for the dynamic case — quotes per dialect ("pg" / `mysql`), doubles embedded quotes, and adds its own quotes so callers can't create a quote mismatch. Rejects NUL/empty.
  • Documented the two footguns the library can't fix for you: LIKE wildcards in bound values (%/_ still match), and second-order injection.

2. Correctness — a real lexer replaces the ? scanner

Naive ?-counting corrupts real SQL. Reproduced on the old code:

pg(["SELECT * FROM t WHERE tags ?| ARRAY['x'] AND id = ?", 5])
// 2.4.2 → "... WHERE tags $1| ARRAY['x'] AND id = ?"   ← operator destroyed, value misbound
// 3.0.0 → "... WHERE tags ?| ARRAY['x'] AND id = $1"   ✓

This is the exact bug class that broke Aura.SqlQuery. The lexer now skips:

  • single-quoted string literals ('' escapes, plus MySQL \ escapes)
  • quoted identifiers ("…" and `…`)
  • line comments and block comments (nested)
  • dollar-quoted bodies ($$…$$, $tag$…$tag$) — the pgx injection class

…recognises ?| / ?& / ?? as operators, and honours \? as an escaped literal ? for the genuinely-ambiguous bare jsonb ?. Clause markers are now classified positionally with \b, which also fixes OFFSET ? being misread as SET ?. Too few values now throws instead of silently binding undefined.

3. DX — the new sql tagged template (recommended)

const { pg, sql } = require('simple-builder')

pg(sql`SELECT * FROM users WHERE id = ${id}`)
// { text: 'SELECT * FROM users WHERE id = $1', values: [id] }

Every ${interpolation} is always a bound parameter — you cannot forget to parameterise, so accidental injection is structurally impossible. Nothing is scanned for ?, so jsonb operators need no escaping. Fragments nest and compose:

const active = onlyActive ? sql`AND active = ${true}` : sql.empty
pg(sql`SELECT * FROM users WHERE org = ${org} ${active}`)

pg(sql`SELECT * FROM ${sql.id('public','users')} WHERE id IN ${[1,2,3]}`)
// { text: 'SELECT * FROM "public"."users" WHERE id IN ($1,$2,$3)', values: [1,2,3] }

Helpers: sql.id / sql.value / sql.join / sql.raw (unsafe by construction, documented) / sql.empty.

Both APIs lower to one node representation and render through one Renderer, so pg()/mysql() accept either. Rendering is pure — a fragment renders repeatedly, for either dialect.

Packaging & tooling

  • TypeScript source → compiled CJS (dist/) + static ESM wrapper (esm/) via an exports map; no dual-package hazard. Complete types (the old index.d.ts only typed scalar arrays).
  • publint --strict clean; attw green across node10 / node16-CJS / node16-ESM / bundler — both now gate CI and publish.
  • CI: Node 16–24 matrix, both fuzzers, CJS+ESM package-boundary smoke against the packed tarball, typings gate. Provenance publish workflow.
  • Added LICENSE, AGENTS.md/CLAUDE.md, llms.txt, .gitignore, issue templates.

Tests

50 unit tests (from 22), exact type-level tests, CJS+ESM boundary tests against the installed tarball, and two fuzzers:

  • Differential — random well-formed queries through the new build and the vendored 2.4.2 builder; output must match byte-for-byte.
  • Property — the tag/lexer have no 2.4.2 counterpart, so they're checked against invariants using injection-shaped inputs: values bind in order and never reach the text, pg renders exactly $1..$n, mysql renders n ?, rendering is pure.

Behaviour changes vs 2.4.2 (all unit-tested)

Each is a fix; all are kept out of the differential fuzzer's domain:

  1. Lowercase in ? now expands (was left as a literal placeholder, mis-binding values).
  2. The caller's partials array is no longer mutated.
  3. Empty object to VALUES ?/SET ?/WHERE ? throws instead of emitting () VALUES ().
  4. Non-identifier object keys throw.
  5. Fewer values than placeholders throws instead of binding undefined.
  6. OFFSET ? is no longer misread as SET ?.
  7. ? inside literals/identifiers/comments/dollar-quotes is no longer a placeholder; ?|/?& survive.

Verification

npm test → 50/50 + type gate. npm run fuzz -- 20000 → 20k differential scenarios agree with 2.4.2, 20k property scenarios hold all invariants. Packed tarball passes CJS+ESM boundary tests and the latest-TypeScript typings gate under node16 resolution. publint --strict and attw clean.

Bumped to 3.0.0 — API-compatible for documented usage; major for the packaging/module-format change and the behaviour fixes above.

🤖 Generated with Claude Code

Acro and others added 3 commits July 14, 2026 12:13
Full modernization of the library while preserving the public API and byte-for-
byte output for all documented usage (proven by a differential fuzzer against
the vendored 2.4.2 builder over 10k+ seeded scenarios).

Packaging & tooling
- Reimplement in TypeScript (src/index.ts); ship compiled CJS (dist/) plus a
  static ESM wrapper (esm/) via an exports map. `main`/`types` updated.
- Add first-class, complete type definitions (objects, Date, bigint, null,
  varargs, projection arrays, optional `values`) — the old index.d.ts only
  typed scalar arrays.
- tsconfig, engines (Node >=16), files allowlist, sideEffects, richer metadata.
- GitHub Actions CI (Node 16–24 matrix, differential fuzz, package-boundary
  smoke for CJS+ESM, typings gate) and a provenance publish workflow.
- Add LICENSE, AGENTS.md, CLAUDE.md, llms.txt, .gitignore, issue templates.

Tests (replacing the old console.log script)
- Assertion-based unit suite, exact type-level tests, package-boundary smoke
  tests (installed tarball, CJS + ESM), and a seeded differential fuzzer.

Bug fixes (behaviour changes vs 2.4.2, covered by unit tests)
- Lowercase `in ?` is now expanded; the matcher was case-insensitive but the
  substitution used a hard-coded uppercase ` IN ?`, so lowercase left a stray
  placeholder and mis-bound values.
- The caller's partials array is no longer mutated (projection join wrote back
  into the input array).
- An empty object passed to VALUES ?/SET ?/WHERE ? now throws a clear error
  instead of emitting invalid SQL like `() VALUES ()`.
- Array detection uses Array.isArray; error messages are clearer.

Docs
- Rewrite README with current async/await, a Security section documenting that
  object KEYS are interpolated as identifiers (values remain parameterised).

Bump to 3.0.0 (packaging/module-format change; API-compatible).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Both surfaced by the packaging research: publint's one suggestion (declare the
package type so Node skips type detection) and a strict publint + attw gate in
the package-boundary CI job. `publint --strict` is now clean and attw reports
no problems across node10 / node16-CJS / node16-ESM / bundler resolution.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implements the security / correctness / DX roadmap from the architecture review
into the unreleased 3.0.0, keeping the library one file and zero-dependency
(~16 kB compiled). The `?` partials API stays byte-identical to 2.4.2 for all
documented usage — the differential fuzzer proves it on every run.

SECURITY — identifiers are the only injection vector, since no driver can bind
an identifier (OWASP: placeholders name data, never tables/columns).
- Object keys in VALUES ?/SET ?/WHERE ? are now allow-listed to plain
  identifiers (`col`, `tbl.col`) and throw otherwise. Legitimate keys pass
  through byte-for-byte, so this changes output for nothing except payloads.
  Chosen over auto-quoting, which would silently break case semantics (quoted
  identifiers are case-sensitive; Postgres folds unquoted names to lower case).
- NEW `sql.id(...parts)` quotes dynamic identifiers per dialect ("pg" / `mysql`)
  and doubles embedded quotes; the helper adds its own quotes so callers cannot
  create a quote mismatch. Rejects NUL and empty names.
- README/llms.txt document the residual footguns the library cannot fix for you:
  LIKE wildcards in bound values, and second-order injection.

CORRECTNESS — replace the regex/flag state machine with a real lexer.
A naive `?` scan corrupts real SQL. Previously:
  pg(["... WHERE tags ?| ARRAY['x'] AND id = ?", 5])
  → "... WHERE tags $1| ARRAY['x'] AND id = ?"   // operator destroyed, value misbound
The lexer now skips string literals ('' and MySQL \ escapes), quoted identifiers
("" and ``), line/block comments (nested), and dollar-quoted bodies ($$…$$ /
$tag$…$tag$ — the pgx injection class); treats `?|`/`?&`/`??` as operators; and
honours `\?` as an escaped literal `?` for the ambiguous bare jsonb operator.
Clause markers are now classified positionally with `\b`, which also fixes
`OFFSET ?` being misread as `SET ?`. Too few values now throws instead of
silently binding undefined.

DX — NEW sql`` tagged template, the recommended API.
Every ${interpolation} is always a bound parameter, so accidental injection is
structurally impossible; nothing is scanned for `?`, so jsonb operators need no
escaping. Fragments nest and compose; arrays expand to `IN ($1,$2)`. Helpers:
sql.id / sql.value / sql.join / sql.raw / sql.empty. Both APIs lower to one node
representation and render through one Renderer, so `pg`/`mysql` accept either.
Rendering is pure — a fragment renders repeatedly, for either dialect.

TESTS — 22 → 50 unit tests, plus a second fuzzer.
The `sql` tag and lexer have no 2.4.2 counterpart, so they get property-based
fuzzing over injection-shaped inputs: values bind in order and never reach the
text, pg renders exactly $1..$n, mysql renders n `?`, and rendering is pure.
20k differential + 20k property scenarios per CI run. Also gates publint --strict
and are-the-types-wrong in CI and on publish.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@Acro Acro changed the title Modernize for 2026: TypeScript rewrite, dual CJS/ESM, tests & CI (v3.0.0) simple-builder 3.0.0: sql`` tagged template, real SQL lexer, identifier safety, TS rewrite, dual CJS/ESM Jul 14, 2026
Acro and others added 5 commits July 14, 2026 14:34
Self-review of the new lexer found a real bug: Postgres E'...' escape strings
honour backslash escapes, but the lexer only did so for MySQL. So

  pg(["SELECT E'a\\'b ?' AS s FROM t WHERE id = ?", 1])

ended the string early at the escaped quote, bound the value to the `?` INSIDE
the string literal, and left the real `id = ?` as a literal question mark:

  before: "SELECT E'a\\'b $1' AS s FROM t WHERE id = ?"
  after : "SELECT E'a\\'b ?' AS s FROM t WHERE id = $1"

Same class of gap for MySQL's backslash-escaped "..." strings.

Quote runs now go through one `consumeQuoted` helper that takes an `escapes`
flag, set per dialect and per quote style:
- MySQL: backslash escapes in both '...' and "..."
- Postgres: backslash escapes ONLY in an E'...' string; a standard string
  treats `\` as an ordinary character (standard_conforming_strings), and a
  "..." identifier uses "" doubling alone.

Adds 3 regression tests (50 → 53) covering both dialects and both directions
(escapes honoured where they apply; `\` ordinary where it doesn't).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
All three reproduced before fixing; each now has a regression test.

1. Marker classes enumerated as a Row at a clause-marker value position
   (src/index.ts). Sql/Identifier/Raw/Single are plain objects, and the
   `clause && isObject(value)` branch ran before the instanceof checks, so:

     pg(['SELECT * FROM t WHERE ?', sql.id('foo')])
     → { text: 'SELECT * FROM t WHERE parts=$1', values: [['foo']] }

   The marker's own field names (parts/nodes/value/text) all satisfy
   PLAIN_IDENTIFIER, so nothing threw — it silently emitted wrong SQL against
   fabricated columns. Same for VALUES ?/SET ?/IN ? and for raw/value/tag.
   Fixed two ways (independent guards): the instanceof branches now run first,
   and isObject excludes marker instances via a new isMarker predicate.

   This also surfaced a related gap the review noted: an sql`` fragment at a
   value position had NO branch at all and was bound as raw node data, even
   without a clause marker. It now splices via a shared renderNodes(), so
   placeholder numbering stays continuous:
     pg(['SELECT * FROM t WHERE x = ?', 9, 'AND ?', sql`a = ${1}`, 'AND b = ?', 2])
     → 'SELECT * FROM t WHERE x = $1 AND a = $2 AND b = $3'
   `build()` now uses isMarker too, so pg(sql.value(x)) no longer stringifies
   to "[object Object]".

2. MySQL `#` line comments were not lexed (src/index.ts). A `?` inside one was
   counted as a placeholder, so valid MySQL either threw a bogus
   "3 placeholder(s) but only 2 value(s)" or silently misaligned parameters.
   Gated on dialect: Postgres has no `#` comment — it starts the #> and #-
   jsonb operators, which are covered by a test to prevent an over-eager fix.

3. Script injection in publish.yml: the release tag_name was interpolated into
   a bash script body. `${{ }}` expands before bash runs and git permits $ ` " ;
   | in tag names, so a crafted tag could execute shell in a job holding
   id-token: write and NPM_TOKEN. Now passed via env: and referenced as $TAG.
   Low severity (requires contents: write to cut a release) but it is the
   documented anti-pattern and the fix is free.

Tests: 53 → 58 unit tests, plus a third fuzzer. The existing fuzzers could not
have caught #1 — the tag fuzzer never routes a marker through buildPartials.
The new marker fuzzer does, asserting a marker's internal field names never
surface as SQL identifiers. Verified it earns its place by reintroducing the
original bug and confirming it fails:
  MARKER PROPERTY FAILURE: marker field "nodes" surfaced as a column:
  UPDATE t SET nodes=$1

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… bug

The unit suite asserts on the generated { text, values }, which only ever proves
the builder agrees with OUR reading of the SQL grammars. Both lexer bugs found
so far were exactly that kind of misreading, so this suite executes the built
queries against real servers and asserts on the returned rows.

It immediately earned its place by finding a third one:

  MySQL treats `--` as a comment ONLY when followed by whitespace or EOF.
  Verified on MySQL 8.4: `SELECT 1--2` is 3 (two minus signs), and
  `SELECT 1--?` really does bind the placeholder. The lexer treated `--`
  as a comment unconditionally, so it swallowed the `?`:

    mysql(['SELECT 1--? AS v', 2])
    → Error: expected an SQL string fragment but got number at position 1

  Postgres, by contrast, always treats `--` as a comment — confirmed against
  PG 16, where `SELECT 1--$1 AS v\n, $1 ::int AS n` comments out the first
  and binds the second. So the rule is dialect-gated, like `#`.

Verified the regression test earns its place by reverting the fix and watching
it fail with exactly that error.

test/integration.cjs — 27 tests over Postgres 16 + MySQL 8.4:
- INSERT/UPDATE/WHERE/IN object expansion round-trips (assert on rows)
- $1..$n numbering correct across mixed clauses
- injection payloads stored as literal data, table still standing
- jsonb ?| ?& and the bare ? (via \? escape, and via the tag with no escape)
- #> / #- are operators, not comments
- ? inside string literals, dollar-quotes ($$ and $tag$), and comments
- pg E'…' backslash escapes vs standard_conforming_strings literal backslash
- MySQL # comments, `--` whitespace rule, backslash escapes in both quote styles
- sql.id quoting: case-sensitivity, embedded-quote escaping, schema qualifying
- sql tag composition and fragments spliced at a ? value position
- LIKE wildcards in a bound value (pins the documented behaviour)

Tooling:
- pg + mysql2 as devDependencies ONLY — the published tarball still ships zero
  dependencies (verified: installed package's `dependencies` is {}).
- npm run test:integration; npm run db:up / db:down for local docker servers.
- CI job with postgres:16 and mysql:8 service containers.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Closes the last risk flagged: the integration suite only pinned DEFAULT server
configuration, and three settings invert assumptions the lexer encodes.

Measured against MySQL 8.4 / PG 16 (each contradicts a plausible assumption):
- MySQL ANSI_QUOTES: "…" is an identifier escaped by "" doubling — `"a\"b"` is
  a SYNTAX ERROR there, so backslash escaping must be off for `"`.
- MySQL NO_BACKSLASH_ESCAPES: `\` is ordinary in literals ('a\' is a\).
- pg standard_conforming_strings=off: `\` escapes inside '…' — the exact
  OPPOSITE of the default the lexer assumed.
- `''` doubling lexes identically in EVERY mode of both engines. It is the only
  mode-proof escape; the sql tag never lexes, so it is mode-proof too.

There is no conservative rule that satisfies both readings — the placeholder
count genuinely differs — so the builder has to be told. Added a minimal,
default-inert opt-in:

    const my = mysql.withMode({ ansiQuotes: true, noBackslashEscapes: true })
    const pgOld = pg.withMode({ standardConformingStrings: false })

withMode returns a NEW builder (original untouched) and merges when chained.
Only affects the `?` partials API. Needed only when your SQL puts a backslash
immediately before a quote inside a literal.

Tests: 30 matrix tests (4 MySQL modes × 2 pg modes) that set the real SESSION to
the mode AND configure the builder to match, so any disagreement fails. Each
mode asserts: '' doubling is mode-proof, identifiers work, the tag is mode-proof,
object expansion round-trips, and backslash-in-literal follows the mode.
Plus 6 unit tests + type/boundary coverage for the new surface. 58 → 64 unit.

Also switches the MySQL integration path from query() to execute(), which found
a driver-level fact worth documenting: mysql2's query() substitutes values
CLIENT-side with backslash escaping, so under NO_BACKSLASH_ESCAPES it errors and
round-trips `x\` as `x\\`. execute() binds server-side and is correct in every
mode. Using execute() also means MySQL's own parser — not mysql2's scanner —
validates our text. README/llms.txt now tell MySQL users to prefer execute().

Corrected an overclaim while verifying: the server is NOT a full oracle for
MySQL. Measured — pg rejects too-few AND too-many params; MySQL execute()
rejects too-few but silently ignores extras. So the tests assert on returned
rows, not merely that the query ran. Recorded in AGENTS.md.

Published package still ships zero dependencies (verified: {}).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three surfaces an agent actually reads — llms.txt in the tarball, the shipped
.d.ts via LSP, and runtime errors — audited and closed.

Executable documentation (test/docs.test.cjs, wired into npm test)
- Parses every ```javascript block in README.md and llms.txt, runs it against
  the build, and compares with the documented result. Docs are what agents copy
  verbatim, so drift is now a build failure rather than a silent lie.
- 26 examples verified (36 evaluated). Verified the test earns its place by
  drifting a recipe and watching it fail with a precise diff.
- Rejects ellipses in expected values, which caught a real doc bug: the INSERT
  example documented `values: [...]` — an agent would have copied that
  literally. It now shows the real values.
- A floor on the checked count guards against the parser silently matching
  nothing and "passing" vacuously.

llms.txt — added a Recipes section (12 machine-verified, self-contained)
Previously it had no runnable code at all, so an agent had nothing to copy.
Covers: value binding, conditional WHERE via sql.empty, sql.join for a variable
number of clauses, IN lists, INSERT..RETURNING, UPDATE from an object, dynamic
identifiers via sql.id, dynamic ORDER BY via allow-list + sql.raw, pagination,
LIKE with wildcard escaping, jsonb `?` in the tag, and one fragment rendered for
both dialects.

Shipped .d.ts — added 8 @example blocks on pg / mysql / sql / id / raw / value /
join, so an LSP hover teaches the safe pattern at the moment of use (e.g.
sql.raw's example shows the allow-list first; sql.id's notes the case-sensitivity
trap; mysql's points at execute() over query()).

Discoverability
- README now points at llms.txt and says where it lands on disk.
- GitHub repo had NO description, topics, or homepage — all three set.
- npm keywords already cover the tagged-template/injection/dialect surface.

Verified after a real install: llms.txt (228 lines), the @example-carrying d.ts,
and `dependencies: {}` are all present in node_modules/simple-builder.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant