simple-builder 3.0.0: sql`` tagged template, real SQL lexer, identifier safety, TS rewrite, dual CJS/ESM

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,32 @@
+---
+name: Bug report
+about: A query builds to the wrong text/values, or an error you didn't expect
+title: ''
+labels: bug
+assignees: ''
+---
+
+**What you called**
+
+```javascript
+const { pg } = require('simple-builder') // or mysql
+pg([ /* your partials */ ])
+```
+
+**What you got**
+
+```
+{ text: '...', values: [...] }   // or the error message
+```
+
+**What you expected**
+
+```
+{ text: '...', values: [...] }
+```
+
+**Environment**
+
+- simple-builder version:
+- Node.js version:
+- Driver (pg / mysql / mysql2):

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,23 @@
+---
+name: Feature request
+about: Suggest an addition that keeps the SQL visible and the API tiny
+title: ''
+labels: enhancement
+assignees: ''
+---
+
+**The problem**
+
+What are you trying to write, and how does it feel awkward today?
+
+**The query you'd like to write**
+
+```javascript
+pg([ /* the ideal partials */ ])
+// desired: { text: '...', values: [...] }
+```
+
+**Notes**
+
+Anything about dialect differences (pg vs mysql), backward compatibility, or
+alternatives you considered.

.github/workflows/ci.yml

@@ -0,0 +1,105 @@
+name: CI
+
+on:
+  push:
+    branches: [master, main]
+  pull_request:
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        node-version: [16.x, 18.x, 20.x, 22.x, 24.x]
+    steps:
+      - uses: actions/checkout@v4
+      - name: Use Node.js ${{ matrix.node-version }}
+        uses: actions/setup-node@v4
+        with:
+          node-version: ${{ matrix.node-version }}
+      - run: npm install
+      - run: npm test
+
+  fuzz:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 22.x
+      - run: npm install
+      - name: Fuzz — differential vs the 2.4.2 oracle + sql-tag property fuzz
+        run: npm run fuzz -- 20000
+
+  # Executes the built queries against real servers and asserts on the rows that
+  # come back. The unit suite only ever proves the builder agrees with OUR
+  # reading of the SQL grammars — every lexer bug found so far was exactly that
+  # kind of mistake, so this is the gate that actually retires the risk.
+  integration:
+    runs-on: ubuntu-latest
+    services:
+      postgres:
+        image: postgres:16
+        env:
+          POSTGRES_PASSWORD: secret
+          POSTGRES_DB: sbtest
+        ports: ['5432:5432']
+        options: >-
+          --health-cmd pg_isready
+          --health-interval 10s
+          --health-timeout 5s
+          --health-retries 10
+      mysql:
+        image: mysql:8
+        env:
+          MYSQL_ROOT_PASSWORD: secret
+          MYSQL_DATABASE: sbtest
+        ports: ['3306:3306']
+        options: >-
+          --health-cmd "mysqladmin ping -uroot -psecret --silent"
+          --health-interval 10s
+          --health-timeout 5s
+          --health-retries 10
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 22.x
+      - run: npm install
+      - name: Integration tests against real Postgres + MySQL
+        env:
+          PG_URL: postgres://postgres:secret@localhost:5432/sbtest
+          MYSQL_URL: mysql://root:secret@127.0.0.1:3306/sbtest
+        run: npm run test:integration
+
+  package-boundary:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 22.x
+      - run: npm install
+      - name: Lint the published package surface (publint + are-the-types-wrong)
+        run: |
+          npm run build
+          npx --yes publint@latest --strict
+          npx --yes @arethetypeswrong/cli@latest --pack . --ignore-rules cjs-resolves-to-esm
+      - name: Pack tarball
+        run: |
+          npm pack --pack-destination /tmp
+          echo "TARBALL=$(ls /tmp/simple-builder-*.tgz)" >> "$GITHUB_ENV"
+      - name: Install into a fresh consumer and run the smoke tests (CJS + ESM)
+        run: |
+          mkdir -p /tmp/consumer && cd /tmp/consumer
+          npm init -y > /dev/null
+          npm install "$TARBALL"
+          cp "$GITHUB_WORKSPACE/test/boundary/smoke.cjs" "$GITHUB_WORKSPACE/test/boundary/smoke.mjs" .
+          node --unhandled-rejections=strict smoke.cjs
+          node --unhandled-rejections=strict smoke.mjs
+      - name: Typings gate — latest TypeScript compiles the installed d.ts
+        run: |
+          cd /tmp/consumer
+          cp "$GITHUB_WORKSPACE/test/boundary/consumer.ts" .
+          npm install --no-save typescript@latest
+          ./node_modules/.bin/tsc --noEmit --strict --target es2017 --lib es2017 --module node16 --moduleResolution node16 consumer.ts

.github/workflows/publish.yml

@@ -0,0 +1,76 @@
+name: Publish
+
+# Publishes to npm with provenance attestations. Trigger by publishing a
+# GitHub Release (tag `vX.Y.Z`), or manually via workflow_dispatch.
+#
+# One-time setup: add an npm Automation token as the `NPM_TOKEN` repository
+# secret (Settings → Secrets and variables → Actions), OR configure a trusted
+# publisher on npmjs.com (OIDC, no token needed). Provenance requires a public
+# repo and the `id-token: write` permission below.
+
+on:
+  release:
+    types: [published]
+  workflow_dispatch:
+
+permissions:
+  contents: read
+  id-token: write # required for npm provenance (OIDC)
+
+jobs:
+  publish:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 22.x
+          registry-url: 'https://registry.npmjs.org'
+
+      - run: npm install
+
+      - name: Guard — tag matches package.json version
+        if: github.event_name == 'release'
+        # The tag is passed through `env:`, never interpolated into the script
+        # body: `${{ }}` expands before bash runs, so a tag name containing shell
+        # metacharacters (git permits $ ` " ; |) would otherwise execute here —
+        # in a job holding id-token: write and NPM_TOKEN.
+        env:
+          TAG: ${{ github.event.release.tag_name }}
+        run: |
+          PKG_VERSION="$(node -p "require('./package.json').version")"
+          if [ "v$PKG_VERSION" != "$TAG" ] && [ "$PKG_VERSION" != "$TAG" ]; then
+            echo "::error::package.json is $PKG_VERSION but the release tag is $TAG"
+            exit 1
+          fi
+
+      - name: Full gate — build + type gate + unit suite
+        run: npm test
+
+      - name: Full gate — differential + property fuzz
+        run: npm run fuzz -- 20000
+
+      - name: Full gate — package surface (publint + are-the-types-wrong)
+        run: |
+          npx --yes publint@latest --strict
+          npx --yes @arethetypeswrong/cli@latest --pack . --ignore-rules cjs-resolves-to-esm
+
+      - name: Full gate — package-boundary smoke (CJS + ESM) from the tarball
+        run: |
+          npm pack --pack-destination /tmp
+          TARBALL="$(ls /tmp/simple-builder-*.tgz)"
+          mkdir -p /tmp/consumer && cd /tmp/consumer
+          npm init -y > /dev/null
+          npm install "$TARBALL"
+          cp "$GITHUB_WORKSPACE/test/boundary/smoke.cjs" "$GITHUB_WORKSPACE/test/boundary/smoke.mjs" .
+          node --unhandled-rejections=strict smoke.cjs
+          node --unhandled-rejections=strict smoke.mjs
+
+      - name: Publish with provenance
+        run: |
+          npm install -g npm@latest
+          npm --version
+          npm publish --provenance --access public
+        env:
+          NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

.gitignore

@@ -0,0 +1,3 @@
+node_modules/
+dist/
+*.tgz

AGENTS.md

@@ -0,0 +1,151 @@
+# Agent guide — simple-builder
+
+Zero-dependency npm package: a tiny SQL builder. The implementation is one file,
+`src/index.ts` (~540 lines with comments, ~16 kB compiled); everything else is
+tests and packaging.
+
+## Consumer API
+
+See `llms.txt` for the complete API, lexing rules, security model, recipes, and
+gotchas in one file. It is the primary surface for AI agents and ships in the
+npm tarball, so it lands at `node_modules/simple-builder/llms.txt`. Keep it in
+sync with the API — its examples are executed by `npm test`, so a wrong one
+fails the build, but a *missing* one fails silently. The other two agent-facing
+surfaces are the shipped `dist/index.d.ts` (JSDoc + `@example` blocks, which is
+what an LSP hover shows) and the runtime error messages — all three should teach
+the fix, not just state the problem.
+
+## Working on this repo
+
+- Build: `npm run build` (tsc → `dist/`). `dist/` is git-ignored; `prepare`
+  builds on install/publish.
+- Test: `npm test` — builds, compiles `test/types.test.ts` (exact-type
+  assertions; a wrong inference is a build failure), runs the unit suite under
+  `--unhandled-rejections=strict`, then runs the doc tests.
+- Docs are executable: `test/docs.test.cjs` parses every ```javascript block in
+  `README.md` and `llms.txt`, runs it, and compares against the documented
+  result. Docs are what humans AND AI agents copy verbatim, so drift is a build
+  failure. To be checked, an example must be a `pg(…)`/`mysql(…)` expression
+  followed by a `// { … }` comment; anything else in the block is treated as
+  setup and evaluated. Ellipses (`values: [...]`) are rejected — write the real
+  value. A floor on the checked count guards against the parser silently
+  matching nothing.
+- Fuzz: `npm run fuzz -- [iterations] [seed]` — see below. Failures print the
+  seed for exact reproduction.
+- Integration: `npm run db:up` (docker Postgres + MySQL), then
+  `npm run test:integration`; `npm run db:down` to clean up. CI runs this via
+  service containers. **This is the only gate that checks the lexer against the
+  engines rather than against our reading of their manuals** — every lexer bug
+  found so far (pg `E'…'` escapes, MySQL's whitespace-after-`--` rule) was a
+  grammar misreading that unit tests happily confirmed. Add a case here whenever
+  you touch `lex()`.
+- Package surface: `npx publint --strict` and `npx @arethetypeswrong/cli --pack .`
+  must both stay clean; CI gates on them.
+
+## Architecture
+
+`src/index.ts`, in dependency order:
+
+1. **Identifiers** — `assertPlainIdentifier` (allow-list for object keys) and
+   `quoteIdentifier` (dialect quoting for `sql.id`). Identifiers can never be
+   parameterised, so these are the only things that write non-value text.
+2. **Nodes / `Sql`** — the shared representation: `text` | `value` | `id` nodes.
+   Both APIs lower to this.
+3. **`lex()`** — the reason the `?` API is correct. Skips string literals,
+   quoted identifiers, line/block comments, and dollar-quoted bodies; treats
+   `?|`/`?&`/`??` as operators; honours `\?` as an escaped literal `?`. Only
+   what survives all that is a placeholder.
+
+   Lexing settings that differ per server (`Mode`) are threaded in via
+   `withMode`; `makeBuild` closes over them. Defaults match a stock server, so
+   the option is inert unless set.
+
+   Every dialect difference here is load-bearing and verified against real
+   servers by `test/integration.cjs` — do not "simplify" any of them:
+   - Quote runs go through `consumeQuoted`, dialect-aware about backslash
+     escapes: they apply in MySQL and in a Postgres `E'…'` escape string, but
+     NOT in a standard pg string (`standard_conforming_strings` makes `\`
+     ordinary). Getting this wrong binds a value *inside* a string literal.
+   - `--` is a comment in MySQL only when followed by whitespace or EOF
+     (`SELECT 1--2` is 3 there); Postgres always treats it as a comment.
+   - `#` is a comment in MySQL only; in Postgres it starts `#>` / `#-`.
+   - Under `ANSI_QUOTES`, MySQL `"…"` is an identifier (doubling only), and
+     `"a\"b"` is a *syntax error* on the server — so backslash escaping must be
+     off for `"` in that mode. Under `NO_BACKSLASH_ESCAPES`, `\` is ordinary in
+     every literal.
+
+## Facts measured against real servers (do not "correct" from memory)
+
+MySQL 8.4 / PG 16, recorded because each one contradicts a plausible assumption:
+
+- `''` doubling lexes identically in **every** mode of both engines. It is the
+  only mode-proof escape. The `sql` tag never lexes, so it is mode-proof too.
+- Placeholder-count strictness: **pg rejects both too-few and too-many** params;
+  **MySQL `execute()` rejects too-few but silently ignores extras**. So the
+  server is a full oracle for pg and only a half-oracle for MySQL — integration
+  tests must assert on returned ROWS, not merely that the query ran.
+- mysql2's `query()` escapes values client-side with backslashes, which is wrong
+  under `NO_BACKSLASH_ESCAPES` (errors; round-trips `x\` as `x\\`). The
+  integration suite therefore drives MySQL through `execute()`, which also means
+  MySQL's own parser — not mysql2's scanner — validates our text.
+4. **`classify()`** — positional clause detection from the text immediately
+   before each placeholder (`\bVALUES\s+$` etc). `\b` is load-bearing: it keeps
+   `OFFSET ?` from reading as `SET ?` and `JOIN ?` from reading as `IN ?`.
+5. **`Renderer`** — owns placeholder numbering and the values array; `pg` emits
+   `$n`, `mysql` emits `?`.
+6. **`buildPartials`** / **`buildSql`** — the two front ends.
+7. **`pg` / `mysql`** — dialect-bound entry points that accept either API.
+
+ESM entry is a static wrapper (`esm/index.mjs`) re-exporting the CJS build — do
+NOT introduce a second compiled implementation (dual-package hazard). When you
+add an export, update `esm/index.mjs` AND `esm/index.d.mts`.
+
+## Invariants you must not break (enforced by tests + fuzzers + CI)
+
+1. **The `?` API's output is byte-identical to 2.4.2** for all documented usage.
+   The differential fuzzer is the guard — keep it green.
+2. Values are ALWAYS parameterised and never appear in `text`, in both APIs.
+   Only identifiers (allow-listed keys, or quoted `sql.id`) and `sql.raw` write
+   text.
+3. `pg` renders exactly `$1..$n` in interpolation order; `mysql` renders `n` `?`.
+4. Rendering is pure: the same `Sql` fragment renders identically, repeatedly,
+   for either dialect.
+5. The caller's partials array is never mutated.
+6. `values` is omitted from the result when no value was bound.
+7. Both CJS `require` and ESM `import` resolve to the same `{ pg, mysql, sql }`.
+
+## The two fuzzers (`test/fuzz.cjs`)
+
+- **Differential** — random well-formed queries through both the current build
+  and the vendored 2.4.2 builder (`test/legacy-oracle.cjs`); output must match
+  byte-for-byte. Its generator deliberately stays inside the domain where the
+  two MUST agree.
+- **Property** — the `sql` tag and lexer have no 2.4.2 counterpart, so they are
+  checked against invariants 2–4 above using injection-shaped "poison" strings.
+  Value poison and identifier poison are kept DISJOINT, and placeholder counting
+  strips quoted identifiers **dialect-aware** (backticks are not quotes in pg) —
+  both are checker correctness requirements, not cosmetics.
+- **Marker** — routes the marker classes through `buildPartials` (the tag fuzzer
+  never does), asserting that a marker's internal field names (`parts`, `nodes`,
+  `value`, `text`) never surface as SQL identifiers. This guards a real bug:
+  markers are objects, so a clause marker before the `?` used to enumerate them
+  as a Row and emit `WHERE parts=$1`. Verify changes here by reintroducing that
+  bug and confirming the fuzzer fails.
+
+## Intentional 2.4.2 → 3.0.0 behaviour changes
+
+Kept OUT of the differential fuzzer's domain (covered by unit tests instead):
+
+- Lowercase `in ?` now expands (was left as a literal placeholder).
+- An empty object to `VALUES ?`/`SET ?`/`WHERE ?` now throws.
+- A non-identifier object key now throws.
+- Fewer values than placeholders now throws (was binding `undefined`).
+- `OFFSET ?` is no longer misread as `SET ?`.
+- `?` inside literals/identifiers/comments/dollar-quotes is no longer a
+  placeholder; `?|`/`?&` are operators.
+
+## Releasing
+
+Bump `version` in `package.json`, then publish a GitHub Release tagged `vX.Y.Z`.
+The publish workflow re-runs the full gate and publishes to npm with provenance.
+See `.github/workflows/publish.yml`.