Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,13 @@ Semver applies from 1.0.0. A breaking change to a documented API needs a major

## [Unreleased]

Nothing yet.
### Fixed

Tier 2 — entity. Tier 3 — realtime.

- **An `arrayOf(uuid())` column reads on a real Postgres.** `Bun.SQL` hands back the array types it has no parser for as their TEXT literal (`'{01a1…,01a2…}'` for `uuid[]`, the empty `'{}'` included), and `decodeRow` passed that string to `arrayOf`'s parser, which refuses anything that is not an `Array`: every read of a row holding one, the `returning` of the insert that wrote it included, was `X_INVARIANT_VIOLATED` ("column.array: expected an array, got a string of 38 characters", a 422 on the HTTP surface). PGlite parses `uuid[]`, so `x dev`, the `unit` step and an app's own tests never saw it; only a server did. An `arrayOf()` cell is now read by its declaration: an `Array` as before, a typed array (`Bun.SQL` answers `int4[]` with an `Int32Array` on a pool that prepares statements) as a plain array, and a string through the Postgres array literal grammar, with quoting, backslash escapes, NULL members and `{}`, each element then parsed by the element column. A cell that is none of these is still `X_INVARIANT_VIOLATED`, now naming the entity and the property; a string in a non-array column is never inspected. Writes, and `contains` / `overlaps` with a bound array, were measured for every element type and were already correct. Nothing to change in an app; rows written before this release read back as written.
- **Every column kind is round-tripped on both drivers.** One entity declares every column kind and an array of every element kind `arrayOf()` accepts (nullable elements included), and seven rows go through insert, a keyed read, an unfiltered read and an update on PGlite (`unit` step) and on `Bun.SQL` against Postgres (`live` step), each compared with `memoryDriver()`. A kind added to `COLUMN_KINDS` without a column in that entity fails the suite. The only live array test before this used `arrayOf(text())`, the one element type both drivers parse.
- **The array literal grammar has one reader.** `parsePgArray` moved from `@ultimat3/realtime` into `@ultimat3/entity` (exported from the barrel), and the WAL decoder imports it, so a live row and a repository row cannot disagree about a quoted `"NULL"`.

## 27.6.0 - 2026-10-10

Expand Down
56 changes: 56 additions & 0 deletions docs/history/entity.md
Original file line number Diff line number Diff line change
Expand Up @@ -1324,6 +1324,7 @@ string (a `lookup` column's is equal for equal values).
| `jit-preload.ts` | a page's foreign key values → one `in` statement for the whole `for … of` loop |
| `preload.ts` | the relation `preload()` names → one related-rows statement → attached to the page |
| `pg-sql.ts` / `pg-row.ts` | plan → parameterised SQL; physical row ⇄ entity row (money is three columns) |
| `pg-array-literal.ts` / `pg-array-decode.ts` | the Postgres array literal grammar; an `arrayOf()` cell in any shape a driver hands back → a JS array |
| `row-observer.ts` | `setRowObserver` — committed row changes, above the driver, for a change feed that has no log to read. A change made inside a keyed request carries `write` (`currentWriteOrigin()`) |
| `write-tag.ts` | a keyed request's write names itself in the WAL: `pg_logical_emit_message(true, WRITE_ORIGIN_WAL_PREFIX, digest)` opens its transaction, once per transaction; a write outside one gets a transaction of its own; a role that may not execute it is probed once and its writes go out untagged; a pinned repository is never wrapped |
| `registry.ts` | duplicate detection, `describeEntities()` for the manifest, `references()` per entry |
Expand Down Expand Up @@ -1363,3 +1364,58 @@ the same function, which reads either side as plain digits. So `where({ total: 1
`bigint()` column matches the row holding `'10'` in the memory driver, as it always did in
Postgres. `compare-parity.test.ts` runs the rows against both drivers; emitted DDL did not move
(`ddl-pin.test.ts`).

## 2026-10-10 — an array cell is read by its declaration, not by the driver's type table

`arrayOf(uuid())` could be written and never read on a real server: `decodeRow` handed the driver's
cell to `arrayOf`'s `$parse`, and `Bun.SQL` answers `uuid[]` with the array's TEXT literal
(`'{01a1…}'`), which is not an `Array`. Every read of a row holding one was `X_INVARIANT_VIOLATED`
("expected an array, got a string of 38 characters"), including the `returning *` of the insert
that wrote it. PGlite parses `uuid[]`, and PGlite is what `x dev`, the `unit` step and an app's own
tests run on, so nothing in this tree or in an app's gate could see it; a built image against
Postgres was the first thing that did. The one live array test (`columns-data.live.test.ts`) used
`arrayOf(text())`, the one element type both drivers parse.

Measured on Bun 1.4.2 against Postgres 17 (the framework's pool, `prepare: false`) and PGlite, for
`select <expr>` of each type:

| Postgres type | `Bun.SQL`, `prepare: false` (what `postgresClient` opens) | `Bun.SQL`, prepared (binary results) | PGlite |
|---|---|---|---|
| `uuid[]` (and `'{}'::uuid[]`) | **`string`, the literal** | **`string`, the literal** | `Array` |
| `text[]`, `varchar[]` | `Array` | `Array` | `Array` |
| `int4[]` | `Array` of `number` | **`Int32Array`**; with a NULL member **`Failed to read data`** and the connection is dropped | `Array` |
| `float4[]` (no column kind) | `Array` | **`Float32Array`**; NULL member fails as above | `Array` |
| `int8[]` | `Array` of `string` | same | `Array` of `number` / `bigint` |
| `numeric[]` | `Array` of `string` | same | same |
| `bool[]` | `Array` of `boolean` | same | same |
| `timestamptz[]`, `date[]` | `Array` of `Date` | same | same |
| `jsonb[]`, `bytea[]` (refused by `arrayOf`) | `Array` | same | `Array` |
| an `enum` type's array (no column kind) | `string`, the literal | same | `string`, the literal |
| every scalar kind | as `columns-data.ts` records | same | `int8` is a `bigint` |

So the unparsed cases are exactly the array types the driver has no entry for, and a user-defined
type's oid is per-database: a table keyed by oid cannot be complete. `Bun.SQL` also has no
type-parser registration to hang one on. The declared column is the only thing that knows the cell
is an array, so `decodeRow` asks `arrayFromDriver(meta, value)` (`pg-array-decode.ts`) for an array
column and for no other: an `Array` passes through, a typed array becomes a plain one, a string is
read by `parsePgArray` with each element's text turned into the value a parsing driver hands over
(`number`, `boolean`, `Date`, or the text itself), and anything else is `X_INVARIANT_VIOLATED`
naming the entity and the property. A string in a `text()` column is never inspected.

`parsePgArray` moved here from `@ultimat3/realtime`'s `pg-array.ts`, which read the same literal
off the WAL; realtime imports it, so the grammar has one reader. `pg-array-literal.test.ts` holds
it to the writer (`arrayLiteral` in `pg-row.ts`).

The write direction was measured in the same pass and was already right: an array is bound as its
quoted literal (`bindable`), an untyped parameter takes the column's type, and `@>` / `&&` with a
bound operand match on every element type.

`column-matrix-fixture.ts` is the guard: one entity declaring every `COLUMN_KINDS` member and an
array of every element kind `arrayOf()` accepts (a kind added to the list without a column there is
red), seven adversarial rows, each through insert `returning`, a keyed read, an unfiltered read and
update `returning`, compared with `memoryDriver()`. `column-matrix-parity.test.ts` runs it on
PGlite in the `unit` step and `column-matrix.live.test.ts` on `Bun.SQL` in the `live` step.

Not fixed, because the framework never opens such a pool: on a PREPARING `Bun.SQL` connection an
`int4[]` with a NULL member fails inside the driver. `prepare: false` is `bun-sql.ts`'s; a change
to it has to answer the third column of the table above first.
14 changes: 11 additions & 3 deletions packages/cli/src/serve-graph.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -92,8 +92,12 @@ const NEVER_AT_BOOT: readonly (readonly [string, RegExp])[] = [
* both reached through the core barrel.
* raised 636 → 637, measured 637 (2026-10-09, #710 row 9): `auth/src/login-identity.ts` — which
* account a login names, an email or a handle `defineAuth({ handles })` resolves, through the auth barrel.
* raised 637 → 639, measured 639 (2026-10-10, #752): `entity/src/pg-array-literal.ts` and
* `entity/src/pg-array-decode.ts` — an `arrayOf()` cell read by its declaration, because `Bun.SQL`
* hands `uuid[]` back as its text literal and every read of such a row was refused. The grammar
* moved out of `realtime/src/pg-array.ts`, which stays for its oid table, so nothing left the graph.
*/
const MIGRATE_CEILING = 637;
const MIGRATE_CEILING = 639;

/**
* measured: 796 — the 558 above plus what `serve-boot.ts` adds: the services and the roles.
Expand Down Expand Up @@ -175,8 +179,10 @@ const MIGRATE_CEILING = 637;
* raised 935 → 936, measured 936 (2026-10-10, channel events from app code):
* `realtime/src/channel-publish.ts` — `publishChannelEvent` and the process bus the boot installs
* (`setChannelTransport`), so a job or an action publishes on a declared channel with no hub.
* raised 936 → 938, measured 938 (2026-10-10, #752): the two entity modules named on
* `MIGRATE_CEILING` for #752, which every role reaches through the entity barrel.
*/
const SERVING_ROLE_CEILING = 936;
const SERVING_ROLE_CEILING = 938;

/**
* measured: 888 — the 796 above plus the 92 `serve-web.ts` adds (41 CLI, 36 MCP, 15 PWA).
Expand Down Expand Up @@ -255,8 +261,10 @@ const SERVING_ROLE_CEILING = 936;
* raised 1062 → 1063, measured 1063 (2026-10-10, events-only islands are never held):
* `cli/src/island-realtime-state.ts` — which realtime islands read the record store or the outbox
* (held, #506) and which only follow a channel, read off the names each imports from the barrel.
* raised 1063 → 1065, measured 1065 (2026-10-10, #752): the two entity modules named on
* `MIGRATE_CEILING` for #752.
*/
const WEB_ROLE_CEILING = 1063;
const WEB_ROLE_CEILING = 1065;

interface MetaInput {
readonly imports: readonly { readonly path: string; readonly kind: string }[];
Expand Down
11 changes: 5 additions & 6 deletions packages/entity/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ Columns + invariants; the row type is derived from the columns. Tier 2.
list or nothing, and a NULL in it emits `(col in (…) or col is null)`; a column the row never NAMED
is NULL for `eq`/`neq`/`in`.
- **The Postgres driver is proved against a real Postgres** (`pg-driver.live.test.ts`, skipped without
`TEST_DATABASE_URL`). A new operator, column kind or write path is not done until it round-trips.
`TEST_DATABASE_URL`). A new operator, kind or write path is not done until it round-trips.
- **A repository call rejects, never throws synchronously** (`tableFor`'s writes are `async`).
- **`defaultDriver()`** is the process default; `Driver.reset?()` is optional (memory only), resets
repositories in place. Test seam only.
Expand Down Expand Up @@ -102,7 +102,7 @@ Columns + invariants; the row type is derived from the columns. Tier 2.
`seekFrom(entity, plan)`; **`assertSeekable` runs in `planFor`**, before a statement exists. A
`timestamptz` sort key is refused when its `<column>$US` alias would pass 63 bytes. A cursor is bound
to `planScope(plan)` (entity, filters, sort); a bad one is `X_CURSOR_INVALID`, never "from the top".
- **A NULLABLE sort key orders** `asc nulls last` / `desc nulls first` (written down); the cursor tags
- **A NULLABLE sort key orders** `asc nulls last` / `desc nulls first`; the cursor tags
`~` for NULL and `!` for present; the seek reaches NULLs; a nullable key has no row comparison. Only a
nullable PRIMARY-KEY column is refused. `pg-null-order.live.test.ts`.
- **A `timestamp` cursor carries MICROSECONDS**: `seekPrecision` projects `(col at time zone
Expand Down Expand Up @@ -145,8 +145,7 @@ Columns + invariants; the row type is derived from the columns. Tier 2.
NAME and the TABLE are different things** — cache tags and policies key on the name; index names are
the table's.
- **Invariants run twice and only ONE side is rendered here**: app-side `assertInvariants`, and the
CHECK/UNIQUE DDL is `@ultimat3/db`'s `invariant-ddl.ts` reading `$describe()` (`$migration()` and the
local renderers are deleted). An untranslatable predicate is `kind: 'assert'`, `sql: null`.
CHECK/UNIQUE DDL is `@ultimat3/db`'s `invariant-ddl.ts` reading `$describe()`. An untranslatable predicate is `kind: 'assert'`, `sql: null`.
**`InvariantDescription.columns` is projected** (`describe-invariant.test.ts`).
- **The two halves must AGREE, term by term** (`expr.ts`): `matches(/…/i)` is `~*`, other flags are
refused (`matchOperator`); `minLength` counts code points; `trimmed()` strips U+0020 only (`btrim`);
Expand Down Expand Up @@ -175,8 +174,8 @@ Columns + invariants; the row type is derived from the columns. Tier 2.
`scale: null` means a two-column amount (`decodeRow` branches on `$meta.kind === 'money'`).
- **Timestamps are `timestamptz`.** A naive timestamp stays inexpressible.
- **A `jsonb` value is bound `::text::jsonb`** (load-bearing); an array is written as a quoted `{…}`
literal. **Wide column types are normalised in `$parse`** to one row type per driver (`bigint()` and
`decimal()` are strings).
literal, read by its DECLARATION (`pg-array-decode.ts`); `column-matrix-fixture.ts` is every kind
on both drivers. **Wide types are normalised in `$parse`** (`bigint()`, `decimal()`: strings).
- **Every framework member on an entity is `$`-prefixed.** Row types are derived, never re-declared.
**A branded id survives to the signature** (`IdOf<Row>`). `type-pins.ts` enforces all of it.
- **`$parse` tells absence from `null`** (`raw === undefined ? defaultValue(...) : raw`).
Expand Down
2 changes: 1 addition & 1 deletion packages/entity/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -240,7 +240,7 @@ for a table it was going to create; these are the shapes a table already has, `A
| `date()` | `date` | `@ultimat3/time`'s `PlainDate` — a calendar date, no time, no zone. `effective_on` is the date a rate applies, and as a `timestamptz` it is a different date on either side of midnight for half the planet |
| `bigint()` | `bigint` | a decimal `string`. A JS `bigint` is what `JSON.stringify` throws on and a `number` loses digits past 2^53 — which is exactly where a legacy `int8` key lives. Both driver spellings (a string from Bun's `sql`, a `bigint` from PGlite) arrive as one. Bounded at `int8` (−2^63 … 2^63−1): one past either end is refused where Postgres answers `22003` |
| `bytes()` | `bytea` | a plain `Uint8Array`, normalised: Bun's `sql` returns a `Buffer` and PGlite a `Uint8Array`, and the two do not serialise alike |
| `arrayOf(column)` | `<element>[]` | `readonly T[]`, each member parsed by the element column it was given. Money and nested arrays are refused — an element is one scalar column |
| `arrayOf(column)` | `<element>[]` | `readonly T[]`, each member parsed by the element column it was given. Money and nested arrays are refused — an element is one scalar column. Read by the declaration, not by the driver: `Bun.SQL` hands `uuid[]` back as its text literal (`'{…}'`) and PGlite as an `Array`, and both decode to the same row |

## Adopting an existing table

Expand Down
Loading
Loading