feat: content branching - #17878
Draft
paulpopus wants to merge 49 commits into
Draft
feat: content branching#17878paulpopus wants to merge 49 commits into
paulpopus wants to merge 49 commits into
Conversation
Design proposal for named content branches: create a branch, CRUD and publish documents on it in isolation, then merge selected changes back to main. Core mechanism is copy-on-write rows in the same table, discriminated by a `_branch` column, so branch state is filtered by the database and pagination, totalDocs, sorting and count stay correct. Covers collections and globals with drafts/versions, access control via existing Payload primitives, hook semantics on merge, and the enablement and migration story. Recommends core rather than a plugin, since joins are resolved inside the database adapters. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Verifies the three claims the branching design was inferred from, before building anything on them. Confirmed on SQLite: - `not_in` on a hasMany field is unusable for shadow tracking, with two defects rather than the one predicted. A doc shadowed on ['a','b'] leaks through `not_in: ['a']`, and a doc shadowed on nothing is excluded entirely because it produces no joined rows. The second would have hidden every untouched main document in a branch read. - A compound (field, branch) unique index with a non-null sentinel keeps enforcing uniqueness within a branch while allowing the same value across branches. - `latest` on collection versions is scoped per parent, so branch shadow rows can carry independent version chains with no version changes. Postgres and Mongo require Docker and are not yet covered; running the spike on all three is a prerequisite for closing phase 0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
First implementation slice of content branching (CONTENT_BRANCHING_PLAN.md phase 1). Covers the schema layer only; the read path, write path and merge are not yet implemented. - `branching` on the root config, plus per-entity `branching` on collections and globals for opting in or out. - `sanitizeBranchingConfig` resolves the branchable set during `sanitizeConfig`. Ordering is load-bearing: after the default user collection is injected so auth detection sees it, and before the `sanitizeCollection` loop that consumes the result. Auth collections are detected by the `auth` flag rather than by slug, since the auth collection can be named anything and a project may have several. - Built-in Payload collections and auth collections are off by default but can opt in. `payload-branches` and `payload-branch-changes` are hard excluded, since branching the registry that resolves branch state is circular. - `injectBranchFields` adds `_branch`, `_branchDocID` and `_branchOp`, and rewrites `unique: true` fields into branch-scoped compound indexes so a branch copy does not collide with its main row. `_branch` defaults to the `'main'` sentinel rather than null, because Postgres treats nulls as distinct in unique indexes. - `appendBranchFilter` builds the read predicate. Not yet wired into the adapters. Verified on SQLite: 15 integration tests and 7 unit tests. The rest of the matrix is scaffolded as todos. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Makes branching functional end to end for documents created on a branch. Reads are now filtered by the database, so pagination and counts are correct rather than patched after the fact. - `payload-branches` and `payload-branch-changes` registered as core-owned collections when branching is enabled. The changeset registry references documents through a polymorphic relationship so it spans collections with different ID types without a coercion step that fails open. - `resolveBranch` resolves the active branch once per request, in order: explicit Local API argument, `branch` query param, `X-Payload-Branch` header, `payload-branch` cookie, then main. - `loadBranchManifest` loads a branch's changes in a single query per request and groups them by collection in memory, rather than querying per collection. Marked loaded before reading so the manifest read cannot recurse into itself. - `resolveBranchQuery` is the single entry point both adapters call, so Mongo and Drizzle cannot drift apart on branch semantics. Wired into find, findOne and count in db-mongodb and drizzle. - `branch` is a typed Local API option on find, findByID, create, update, delete and count. `branch: false` bypasses branching for merge and cross-branch reads. - Creates on a branch are stamped via a beforeChange hook and recorded in the registry via afterChange. `_branchDocID` is left null for branch-created documents, since their canonical ID is their own. Verified on SQLite: 18 integration tests including pagination integrity — 25 documents on main plus 5 on a branch reads as 25/3 pages on main and 30/3 pages on the branch, with no document appearing on two pages, and count agreeing with find. Regression checked against versions, collections-rest, database and access-control. Updates and deletes on a branch are not implemented yet. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Editing a document on a branch now forks it into a branch-owned copy instead of writing to production. - `forkDocument` copies the main row wholesale on first branch edit, stamping `_branchDocID` and recording the change in the registry. Subsequent edits reuse the copy. - `branchIDs` provides the canonical-identity translation. A shadow row's primary key is not the document's ID, so reads rewrite `id` constraints to match either a shadow row's `_branchDocID` or a main row's own `id`, and results project the canonical ID back over the row's key. - `resolveBranchRowID` does the same for writes, which address rows by primary key directly and so cannot go through the where-tree rewrite. Canonical IDs stay the only identity the API exposes. - Both are skipped under `branch: false`, since the fork and write-target lookups address rows by real primary key and would be corrupted by having canonical IDs projected over them. Two bugs found by the tests: - The shadowed-main-row exclusion compared `_branchDocID`, which is null on main rows, so it never excluded anything and a branch read returned both copies of an edited document. It must compare the main row's own `id`, since for a main row the canonical ID is its primary key. This also removes the design's stated dependency on relationship-field null-safe `not_in` for the exclusion. - `findByID` passes a stub req to the adapter carrying only `transactionID`, so the branch predicate never applied and reads by ID silently resolved against main. Verified on SQLite: 31 branching integration tests and 15 unit tests. Regression checked against versions, collections-rest, database, access-control and relationships. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Deleting on a branch no longer removes production content. - Deleting a main document from a branch records the intent as a shadow row marked `_branchOp: delete`, which the read predicate hides on that branch and nowhere else. The main row is untouched and other branches still see the document. - A document created on the branch is deleted outright, since no main row stands behind it and nothing is left to hide. - An already-forked document is converted into a tombstone in place rather than gaining a second row for the same document. - The absorbed delete still returns the document, so `afterDelete` hooks receive the payload they expect. `resolveBranchDelete` resolves the target through the branch predicate rather than by extracting an ID from the where clause, so it addresses the branch's own copy when one exists and the main row otherwise. Also corrects the plan: the shadowed-main-row exclusion compares a main row's own `id`, not `_branchDocID`, which is null there. `_branchDocID` remains a relationship field for ID typing, but its null-safe `not_in` behavior is no longer load-bearing. Verified on SQLite: 29 branching integration tests and 15 unit tests. Regression checked against versions, collections-rest, database, access-control, relationships and _community. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Ran the branching suite and the regression set on all three adapters. Everything passes unchanged, so no adapter-specific fixes were needed. Closes phase 0. The two anti-join defects that rule out array-based shadow tracking are now confirmed on Postgres as well as SQLite, and Mongo is confirmed correct on both counts — meaning a Mongo-only test run would have missed the hazard entirely. The compound unique index with the `'main'` sentinel and per-parent `latest` scoping are confirmed on all three. One prediction did not hold: `_branchDocID` was expected to surface as an ObjectID on Mongo and break canonical-ID projection. IDs are normalised before the projection runs, so a single code path serves all adapters. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Drafts, versions and publishing now work per-branch, isolated from main. - Version collections of branch-enabled collections gain `_branch` and `_branchParent`. A branch version's `parent` points at the shadow row, so `_branchParent` carries the canonical document the editor knows. - `createVersion` attaches new versions to the branch's own copy and stamps the discriminator, so a draft saved on a branch no longer appends to main's version chain. - `queryDrafts` applies the branch predicate and maps results back to canonical IDs. - `getLatestCollectionVersion` and `replaceWithDraftIfAvailable` resolve the branch row before querying by `parent`; without this, draft reads on a branch silently returned main's content. `latest` bookkeeping needed no branch scoping, confirming the design's claim: a branch version's parent is the shadow row's own primary key, so main's chain and the branch's chain are disjoint by construction and cannot clear each other. Noted at the call site in db-mongodb. Verified on MongoDB, Postgres and SQLite: 40 branching integration tests each. Regression checked against versions (all three), plus collections-rest, database, access-control, relationships, fields and _community. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Globals now branch alongside collections, including drafts and publishing. Globals are structurally simpler than collections: a global is a single document, so there is no result set to paginate and the branch's row and main's row can be fetched together and resolved in memory. They need only `_branch` — no `_branchDocID`, since a global's identity is its slug and is stable across branches, and no `_branchOp`, since globals cannot be created or deleted and so have no tombstones. - `_branch` injected into global tables and their version collections. - `findGlobal` reads the branch's row and main's together, preferring the branch's, and falls through to main for a global the branch never touched. - `updateGlobal` targets the branch's own row, seeding it from main's content the first time the global is edited, and registers the change. - `findGlobalVersions` filters strictly by branch rather than reading through, so a version list cannot interleave two branches' histories. Fixes the corruption case the design predicted. Unlike collection versions, global versions have no `parent` to scope `latest` by — every version of a global shares one stream — so the unscoped clearing in `createGlobalVersion` would wipe main's latest flag when a draft was saved on a branch, losing main's draft with no error. Now scoped by `_branch` in both adapters. One adapter divergence found by the tests: `Model.find` and the `findOne` calls in mongo's `updateGlobal` do not return lean results the way the surrounding code assumed, so transform received mongoose Documents. Made explicit. Verified on MongoDB, Postgres and SQLite: 46 branching integration tests each, plus the globals suite and regression across versions, collections-rest, database, access-control, relationships, fields and _community. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Applies a branch's changes to main. Branch data wins outright — there is
no field-level reconciliation. What replaces conflict resolution is
selection: callers choose which changes to apply, and anything left behind
keeps the branch open.
- `payload.branches.merge({ branch, changes, dryRun })`, following the
`payload.jobs` local API pattern.
- `dryRun` reports what would be applied and why, without writing.
- Selective merge applies only the chosen changes; the branch is marked
merged once its changeset is empty.
- `main-moved` warnings report documents main changed after they were
branched. Informational only — merging still overwrites, which is the
chosen semantic.
- `beforeMerge` can throw to block. `afterMerge` fires after commit, so a
failing deploy webhook cannot undo a merge.
- Writes run in a transaction, through the ordinary Local API, so every
document hook, validation and version creation behaves as it would for a
hand-made edit on main.
A branch-created document is merged by updating its existing row in place
rather than recreating it. The row already holds the ID inbound
relationships point at, and deleting it would cascade those relationship
rows away — recreating the row does not bring them back.
That made the hook `operation` label wrong: such a write is semantically a
create on main, but reported as an update. `updateDocument` hardcoded
`operation: 'update'` in eight places, so it now takes the label as a
parameter defaulting to `'update'` — existing paths are unchanged. One of
those eight turned out not to be a hook label at all: `ensureUsernameOrEmail`
uses it to pick a validation mode and takes a narrower type, caught by the
type checker and left alone.
Verified on MongoDB, Postgres and SQLite: 58 branching integration tests
each. Regression checked against versions, collections-rest, auth, hooks,
access-control, fields, relationships and database.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Join fields now resolve against the active branch. This is the reason branching belongs in core: joins are assembled inside the adapters, below any layer a plugin could reach, so a plugin implementation would have returned main's related documents while on a branch. Both adapters and both mongo join strategies are covered: - `buildJoinAggregation` (mongo, the default path) applies the branch predicate to the `$lookup` pipeline. - `resolveJoins` (mongo, used when join aggregations are disabled) applies it to the per-collection query. - The drizzle join subquery applies it before `buildQuery`. Join results also had to report canonical document IDs. A forked document appears in a join as its shadow row, whose primary key nothing outside branching knows about — population would then fail to resolve it. Mongo projects it with a `$set` stage on the lookup pipeline, drizzle with a `COALESCE` in the subquery select. Join query builders are synchronous, so they cannot load the change manifest themselves. `getBranchPredicateSync` reads the manifest the top-level read already loaded and memoized on the request earlier in the same query, and returns null when branching is inactive. Verified on MongoDB, Postgres and SQLite: 62 branching integration tests each. Regression checked against the joins suite on all three, plus relationships, versions, collections-rest, fields and database. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Merge now checks the merging user's production permissions per document, closing the gap left when merge landed without enforcement. A branch is a proposal — nothing on it is real until merge — which is what makes permissive branch writes safe. The preflight is therefore the enforcement boundary for branching, not a second check layered on a role check. Payload has access functions rather than roles, so who may merge is derived entirely from the access already defined on the documents being merged. - Effective operation is resolved per change from the shadow row, so editing a published document on a branch is checked as a `publish` against main rather than a plain update. Publishing is not a distinct access type in Payload — it is `update` evaluated against published data, which is how the admin UI derives the publish button. - Two tiers, so a large branch does not cost one access call per document: the collection's access function runs once per (collection, operation) group, and only a `Where` result falls through to a single query resolving that group per document. - Denials are collected and reported per document rather than thrown, each naming the collection, document, operation and reason. Blocked changes are excluded from `mergeable` and left on the branch; the rest still merge. - `overrideAccess` defaults to `true`, matching every other Local API operation. HTTP callers pass `false` with `user`. Also fixes an import regression in the drizzle join builder: a lint autofix merged an added type import into the existing value import and marked the whole block `import type`, erasing five runtime imports. This only failed on Postgres, because the nine tests that exercise those paths are skipped on SQLite — a reminder that "green on three adapters" only covers tests that actually run on each. Verified on MongoDB, Postgres and SQLite: 68 branching integration tests each, plus the joins suite on all three. Regression checked against access-control, auth, versions, collections-rest, fields and database. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`POST /<branches>/:id/merge`, accepting `{ changes?, dryRun? }`.
Runs with `overrideAccess: false` and the authenticated user, so the
per-document preflight applies. This is where branching's access model is
actually enforced: the Local API, like every other Payload operation,
trusts server-side callers by default, so HTTP is the boundary.
Responds 403 with `blocked` populated when nothing could be applied and
something was refused, rather than an empty success — a programmatic
caller sees the same per-document reasons the admin UI would show.
Unknown branch is 404; unauthenticated is refused before any work.
Endpoints are registered through `wrapInternalEndpoints` so the POST body
is parsed onto `req.data`, matching every other built-in endpoint. Without
it `dryRun` silently read as false and a preview mutated main.
Verified on MongoDB, Postgres and SQLite: 73 branching integration tests
each. Regression checked against endpoints, auth, access-control,
collections-rest, joins, versions, database and fields.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Starts phase 6. The switcher is the first crumb in the breadcrumb trail: a searchable popup listing `main` plus every open branch the user can read, with a pinned "New branch…" action routing to the standard create view. Creating a branch switches onto it, via the document event rather than the create view, so any route that creates a branch is covered. Branch selection is stored in the user's `branch` preference, so it follows them across browsers and machines. Core stays argument-only — `resolveBranch` reads the operation argument or `?branch=` and nothing else, so an unbranched request costs no lookup. Turning the stored preference into that argument is the admin UI's job, in `getRequestBranch` beside `getRequestLocale`, and every admin API call now passes `branch` explicitly the way it passes `locale`. `payload-branches` is no longer admin-hidden, since the create view 404s on a hidden collection. Its slug is derived from the name and immutable after create (it was `required` + `admin.readOnly`, so the form could never submit), and bulk delete is off given the cascade on shadow rows. The injected `_branch` / `_branchDocID` / `_branchOp` fields are now internally writable only. They were `admin.hidden` but not read-only, and the admin form round-trips every field it loads — so an edit made on a branch submitted the `_branch` it was read with, stamping the shadow row back onto `main` and duplicating the document in production. Field access is skipped under `overrideAccess: true`, so merge can still flip `_branch`, and copy-on-write writes through `payload.db` below field access entirely. Also fixes three pre-existing type errors that blocked a build: a missing `branch` on `FindGlobalVersionsArgs`, a missing `PayloadRequest` import in mongodb's `resolveJoins`, and an aliased SQL expression assigned into drizzle's column map. `Combobox` gains the column selector's search treatment — icon, no-results state, width lock — and the two now share one matcher, promoted to `matchesSearchQuery`. Known gaps are recorded in the plan: the rest of phase 6, scheduled publish still targeting main, and `resolveBranch`-level branch validation. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds the merge and discard workflows end to end, makes globals, localized
fields and nested fields work on a branch, closes the REST and GraphQL gaps,
and roughly halves the query cost of a branch write.
A coverage audit found eight defects that all produced wrong data rather than
an error, each hiding behind a sibling code path that still worked:
- `enforceMaxVersions` pruned by canonical parent with no `_branch` scoping, so
saving on a branch deleted main's whole version chain
- `updateLatestVersion` rewrote main's latest version row with branch content
- `resolveBranch` ignored the `_branchBypass` sentinel and fell through to the
query param, so merging over HTTP with `?branch=` silently did nothing
- bulk `update({ where, branch })` never forked, writing straight to main
- `findDistinct` had no branch predicate in either adapter
- the drafts predicate never rewrote `parent`, so filtering a drafts read by
document ID returned nothing on a branch
- `_status` was read as a scalar, but localization makes it per-locale, so
every publish on a branch looked like an untouched fork and merged nothing
- forks copied array and block rows verbatim, colliding on their primary keys
under a relational adapter
An explicit `branch` on a Local API operation now wins over whatever the
request already resolved, on an isolated request, matching how `locale`
behaves; and the dataloader keys populated documents by branch.
Query cost is asserted rather than estimated: reads add one query per request
and none on main, writes add 5 on first touch and 3 after.
Docs are split into a PR-style overview, a status document and the original
design proposal.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Resolved one conflict in `updateByID`, taking main's base access control (#17662) which passes the entity slug into every access check. Three things the merge broke silently and needed fixing: - the merge preflight called access functions without `slug`, now required - `updateGlobal`'s branch-copy write passed a third argument to `Model.create`, which selects a different overload under Mongoose 9 (#17747) - `payload` and `translations` dist were stale, so `ui` could not see main's new clipboard keys or the `slug` access argument Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ment` Mongoose 9 (#17747) deprecated the `new` option for `findOneAndUpdate` and `findOneAndReplace`, and logs a warning when it is used. The Next dev server forwards server logs to the browser console, and the e2e harness fails any test that produces a console error — so every e2e test that saved a document failed, 34 of 37 in the branching suite. Not branching-specific: `updateOne` is the generic document save. Any suite that updates a document through the admin panel hits it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributor
📦 esbuild Bundle Analysis for payloadThis analysis was generated by esbuild-bundle-analyzer. 🤖
Largest pathsThese visualization shows top 20 largest paths in the bundle.Meta file: packages/next/meta_index.json, Out file: esbuild/index.js
Meta file: packages/payload/meta_index.json, Out file: esbuild/index.js
Meta file: packages/payload/meta_shared.json, Out file: esbuild/exports/shared.js
Meta file: packages/richtext-lexical/meta_client.json, Out file: esbuild/exports/client_optimized/index.js
Meta file: packages/ui/meta_client.json, Out file: esbuild/exports/client_optimized/index.js
Meta file: packages/ui/meta_shared.json, Out file: esbuild/exports/shared_optimized/index.js
DetailsNext to the size is how much the size has increased or decreased compared with the base branch of this PR.
|
Written with AI
Written with AI
Written with AI
Written with AI
Written with AI
Written with AI
Written with AI
Written with AI
Written with AI
Written with AI
Written with AI
Written with AI
Written with AI
Written with AI
Written with AI
This draft PR implements the agreed content branching updates on top of `feat/content-branching`. ## Review status Implementation is in progress. The branch starts from `b03fbca79d7e93e916407a772efc8c913801b51b`. ## Shared processing - Adds ordered, bounded batch processing for iterable and async iterable input. - Uses groups of 100 by default and supports an explicit group size. - Stops on processor failure without retrying or consuming later input. - Lets the processor stop further input consumption. ## Database adapters - [x] Add and verify the `db.copy` contract across MongoDB and Drizzle adapters. - [x] Add typed `db.batchProcessing` operations and outcomes. - [x] Enforce conditional SQL `updateOne` predicates at the write boundary and limit broad predicates to one row. - [x] Preserve caller-owned requests in the default batch processor without adding hidden transactions or retries. - [x] Verify caller-owned transaction behaviour across adapters. ## Branch storage - [x] Use change records as the operation authority and remove `_branchOp` safely. - [x] Keep branch Trash state isolated through reads, restore, discard, permanent deletion, merge, and access preflight. - [x] Keep remaining branch reads, indexes, and existing data consistent. - [x] Refresh branch manifests and populated-document caches after branch mutations. - [x] Preserve stable logical identity during concurrent first edits, including caller-owned transactions. - Missing shadow rows and orphaned branch rows now fail with a 409 before merge, update, delete, or duplicate-shadow recovery can continue. - Collection reads and synchronous join predicates now derive deletion visibility from authoritative change records instead of `_branchOp` row metadata. - Delete classification and cleanup decisions use change operations cached by canonical document ID. - `_branchOp` is removed from schemas, stored-row writers, restore and merge paths, public exports, and generated types. - Collection change records now store a scalar logical document ID and enforce one record per branch, collection, and logical document. - Concurrent recovery reads outside an unchanged caller transaction so MongoDB can observe the winning shadow row. - First branch updates, bulk updates, and version restores now commit or roll back with a caller-owned transaction. - Create, update, bulk update, delete, restore, merge, and discard now refresh the caller request after they change branch-visible content. - Cache refreshes retain the branch, collection, document, and request boundaries already used by branch reads and relationship population. - A caller-owned first-copy race leaves transaction resolution with each caller: one transaction commits, the competing transaction fails without being replaced, and storage retains one consistent shadow/change pair. - Trashed branch copies continue to shadow main before ordinary Trash filtering, so direct reads, lists, and counts do not reveal inherited content. - Merge applies branch trash and restore state to main without publishing draft-only field changes. - Branch-created trashed documents remain hidden after merge, and merge preflight reports denied delete access before writing. - MongoDB relationship-path, has-many, and join filters now apply the active branch before matching parent documents. - Related branch rows are converted back to canonical document IDs before parent filters are built, so a branch replacement can match without exposing its storage ID. - Branch and request context now reaches MongoDB query translation for collection/global reads, draft/version reads, join resolution, counts, and write predicates. - Automatic top-level and nested unique fields now retain optional-field constraints while adding `_branch` to their compound index. - Developer-defined unique indexes and custom upload filename indexes now add `_branch` once while retaining their existing fields and `requireExists` rules. - Localised MongoDB uniqueness now uses one branch-scoped index per locale instead of one index that combines all locale values. - Drizzle locale tables carry `_branch` only when a localised branch index requires it, and locale-row writes retain the parent row's branch. - Version-derived indexes keep the branch discriminator at the version-row level while localised content remains in the locale table. - MongoDB provides a generated `branching` migration that backfills missing or null main identities in content, globals, and version stores before synchronising schema indexes. - A migration backfill conflict stops before stale indexes are removed, so the existing constraint remains available while the conflicting data is corrected. ## Merge preparation and execution - [x] Merge only the latest source state into main. - [x] Share candidate selection between preview and execution. - [x] Reject an invalid selected set before content writes when possible. - [x] Keep preview read-only for writes made through its validation request. - [x] Record durable outcomes, version references, recovery, and cleanup state. - [x] Integrate adapter batching only where it preserves Payload lifecycle behaviour. - Collection and global merges apply the newest source content and status without replaying earlier source publications. - A newer draft leaves main's published content unchanged, including source-created content and autosave-only edits. - Per-locale lifecycle writes are retained, but all locales for one candidate produce one ordinary target version with the complete localised result. - `branching.validate` is sanitized to a default best-effort structural precheck and can be replaced by a server callback for both preview and execution. - Validation receives the latest prepared collection/global candidates in a main-scoped request, ignores configured read-only join and virtual fields, returns structured errors, and propagates unexpected callback failures. - Permission or validation rejection now rejects the whole selected set. Callers can still submit a smaller permitted selection. - Access checks and normal content hooks use a main-scoped request without the internal branch bypass. Raw source reads and cleanup retain a separate storage request. - Execution repeats candidate, access, dependency and validation checks after `beforeMerge`; ordinary writes can still reject content after a successful preview. - Validation callbacks receive a main-scoped request marked with `operation: 'validate'`; collection and global content writes through that request fail before hooks, transactions, files, or database work. - Preview skips `beforeMerge`, preserves the caller request and transaction, and leaves pending source changes intact when a later real write fails. - The inspected on-demand validation dependency is not present in this checkout. The default precheck therefore uses Payload's current entity-input structural validation and does not promise database or unknown JSON-schema simulation. - Collection shadows, source version chains, global branch copies, and consumed change records are now cleaned only after every selected target write succeeds. - A later transaction-free write failure retains earlier update, delete, and global source state instead of consuming each item during application. Branch-created promotions remain part of the pending recovery work because they reuse the source row identity. - Every real merge now creates a durable event before target writes and records each item as unattempted, attempted, applied, committed, failed, rolled back, or unknown. - Merge-owned rollback, transaction-free partial failure, and caller-owned commit now leave distinct durable outcomes. Caller-owned merges remain awaiting commit until the outer transaction succeeds. - A branch-created dependency that this merge has already promoted is treated as available while its source change record remains available for final cleanup. - Final source cleanup now waits for the target transaction to commit, including caller-owned transactions. A caller rollback records rolled-back application and leaves source work intact. - Collection shadows, deletion markers, global copies, source version chains, and change records report cleanup failure without undoing committed target content. - Cleanup compares current collection and global source state with the state that was merged. Newer branch work is preserved and recorded as superseding the old cleanup target. - Merge events record the usable pre-write target version and the version ID created by the actual merge write. - New merge events do not store full document snapshots. Existing snapshot events retain a compatibility rendering path. - The history view reads exact target versions with the current user's version access. Missing, pruned, inaccessible, or disabled history is shown as unavailable. - Transaction-free recovery restores earlier versioned collection and global writes in reverse order, and removes promoted branch-created targets. - A lost progress update records an unknown outcome and does not replay the target write. Recovery and merge errors remain separate. - A missing recovery version records an unavailable recovery outcome and keeps the partial target effect visible for manual review. - Cleanup-failed events retain the source row identity and revision, global source revision, and branch-created source version IDs needed for a safe retry. - A later merge retries failed cleanup before ordinary target writes and updates the original event instead of creating a duplicate event. - Cleanup retry covers collection updates, deletion markers, globals, and branch-created version chains without repeating committed target hooks. - Newer collection or global source work marks the old cleanup as superseded. The retry keeps that work pending and does not apply it. - Recovery can overwrite an intervening target edit. A failed restore keeps the original merge error and records its separate recovery error. - Target content writes remain ordered through Payload so access, validation, hooks, versions, dependencies, and transaction-visible reads keep their normal behaviour. - Merge-event application checkpoints now cover groups of 1,000 collection changes instead of rewriting the full event after every document. In-memory outcome updates use a change-ID index, and cleanup outcomes are persisted during finalisation. - Effective-operation resolution reads shadow rows in adapter-safe groups of 400 while preserving input order. Create IDs and forked document IDs use separate indexed predicates. - Dependency preflight avoids branch-create queries when selected data has no configured relationship value and narrows safe queries to collections that the selected field trees can reference. Rich-text relationships and unknown stored block types retain the conservative lookup path. - After source cleanup succeeds, server-owned branch-change registry deletions use one ordered adapter batch with stable index-to-outcome mapping. - Failed, unattempted, and rejected cleanup batches remain visible as per-item cleanup failures. A later merge retries cleanup without repeating committed target writes. ## Product surfaces - [x] Align Local API, REST, GraphQL, scheduled merges, and Admin outcomes. - [x] Update history, progress, error states, generated types, and documentation. - [x] Retain the branch lifecycle and the documented multi-tenant limitation. - The branch change pill, merge summary, and scheduled merge list use shared server functions that check Admin access and normal branch read access before reading internal records. - The upcoming scheduled-merge predicate is isolated from server-only dependencies, so server views and client components keep their bundle boundary. - Merge preparation removes Payload-managed authorship fields and lets normal hooks attribute the merge actor. Custom fields named `createdBy` or `updatedBy` remain content fields. - Merge-history controls expose their expanded panel relationship. Loading, unavailable, and error states provide status text for assistive technology. - GraphQL global-version restore now handles a draft-only branch with no stored published row: it returns the restored data, records it as the latest branch version, and leaves main history unchanged. - Forced draft restores of globals with localised status now store one draft status per configured locale. - Scheduled merges pass the same current candidate, permission, validation, cleanup, and branch lifecycle coverage as direct merges. - Rich-text dependency checks verify that blocked selected merges preserve the exact original main field value. - The measured first-fork and later-write overhead remains eleven database calls in total, split as eight for the first fork and three for the later write. - Startup configuration warns only when branching and an enabled `@payloadcms/plugin-multi-tenant` plugin are present. It does not infer plugin use from a custom `tenant` field, and the warning states that the combination does not provide tenant isolation. - A new Content Branching guide covers configuration, Admin and API use, merge results, scheduled merges, partial visibility without transactions, optional-version recovery limits, cleanup outcomes, branch-scoped indexes, migrations, secure branch-aware Preview and Live Preview, and the unsupported multi-tenant combination. ## Validation - The shared processor has 11 focused passing unit tests. - The default batch processor has focused unit coverage for ordered outcomes, no-match results, failure stopping, and explicit continuation. - Real adapter batch-processing integration tests pass on MongoDB, SQLite, and PostgreSQL. - Real batch tests cover caller-owned commit, rollback, and later-group failure on MongoDB and PostgreSQL; SQLite verifies transaction-free execution because its fixture has transactions disabled. - `db.copy` contract and isolation tests pass on MongoDB, SQLite, and PostgreSQL; caller-owned commit and rollback pass on MongoDB and PostgreSQL, where the test fixtures enable transactions. - First-edit nested array and block forking through `db.copy` passes on MongoDB, SQLite, and PostgreSQL. - Concurrent first edits recover adapter-specific uniqueness and transaction conflicts on MongoDB, SQLite, and PostgreSQL, with one shadow row, one change record, and no cleanup-scope error. - Collection change identity, writer, and collection race tests pass on MongoDB, SQLite, and PostgreSQL. - Caller-owned first update commit and rollback, bulk update, and first restore tests pass on MongoDB and PostgreSQL. - Simultaneous caller-owned first-copy tests pass on MongoDB and PostgreSQL with one committed winner, one caller-controlled failure, and one consistent shadow/change pair. - Change-record deletion visibility passes direct-read and join tests on MongoDB, SQLite, and PostgreSQL; stored tombstones no longer contain `_branchOp`. - Change-record create and update operations drive hard-delete and tombstone decisions on MongoDB, SQLite, and PostgreSQL. - Schema, row-shape, restore, merge, discard, relationship and race tests pass without `_branchOp` on MongoDB, SQLite, and PostgreSQL; the focused Drizzle join suite passes 16 tests. - Missing-shadow and orphaned-row integrity tests pass on MongoDB, SQLite, and PostgreSQL; the focused shadow helper suite passes 21 tests. - Same-request mutation and populated-relationship cache tests pass on MongoDB, SQLite, and PostgreSQL for create, update, bulk update, delete, restore, merge, and discard. - Ten focused Trash behaviour and permission tests pass on MongoDB. They cover direct reads, lists, counts, restore, discard, permanent deletion, merge, branch-created trashed documents, and delete-access preflight. - The complete MongoDB deletion-safety and merge-security suites pass 52 tests. - Three focused MongoDB predicate-time tests pass for direct filters, relationship paths, join paths, counts, and pagination totals. - The MongoDB adapter build passes, targeted source lint has no errors, and the database-isolation suite passes 3 tests. - Six focused schema tests and eight focused MongoDB tests pass for optional, nested, custom compound, upload filename, and cross-branch unique-index behaviour. - Fifteen focused unit tests pass for localised schema injection, MongoDB index expansion, Drizzle locale-table indexes and writes, and version index paths. - Ten focused MongoDB tests pass for the complete branch unique-index set, including per-locale and per-branch enforcement. - Payload, MongoDB and Drizzle builds pass after the localised index changes. - Two migration unit tests and two real MongoDB migration tests pass for missing, null and explicit-main data, stale unrestricted indexes, collection/global/version reads, and duplicate-key conflicts. - Latest-state MongoDB coverage passes for repeated publications, newer drafts, autosaves, source-created documents, globals, localised values, mixed localised status, access checks, and nested row identity. - The primary MongoDB branching file passes 267 tests with the three baseline scheduled-merge transaction-conflict cases excluded; 9 tests are skipped and 4 remain todo. - The Payload build passes after the latest-state merge changes. - Six branching-config sanitizer unit tests pass for default, disabled, and replacement merge validation. - Six focused MongoDB validation tests pass for malformed prepared content, latest candidate data, replacement behaviour, post-hook rechecks, exception propagation, and main-scoped lifecycle hooks. - Selected-set MongoDB tests pass for full rejection when one item is blocked and for an explicitly smaller permitted selection. - The primary MongoDB branching file passes 274 tests with scheduled merge tests excluded; 9 tests are skipped and 4 remain todo after the validation and target-context changes. - The Payload build passes, focused lint has no errors, and whitespace checks pass after the validation changes. - Ten focused MongoDB merge-validation tests pass for structural validation, replacement callbacks, validation-request write rejection, preview request preservation, post-hook rechecks, database constraint limits, and main-scoped hooks. - The primary MongoDB branching file passes 278 tests with scheduled merge tests excluded; 9 tests are skipped and 4 remain todo after completing Phase 4. - Dedicated MongoDB suites pass 6 merge-security tests, 5 dependency-graph tests, 5 dependency-preflight tests, and 14 write-path-security tests. - The Payload build and Prettier checks pass. Focused source lint has no errors and retains only existing warnings. - A focused MongoDB transaction-free failure test confirms that a later write failure keeps both selected update shadows and change records. - Eighty-nine MongoDB merge, access, global, lifecycle, and transaction tests pass after moving source cleanup to the final phase. - The complete primary MongoDB branching file passes 280 tests with scheduled merge tests excluded after durable outcome tracking; 9 tests are skipped and 4 remain todo. - Internal merge-collection coverage passes 8 tests, the Payload build passes, and focused production lint retains one existing warning with no errors. - Focused MongoDB final-cleanup coverage passes for collection updates, deletions, globals, caller-owned commit and rollback, and concurrent newer source work. - The complete primary MongoDB branching file passes 286 tests with scheduled merge tests excluded after commit-aware cleanup; 9 tests are skipped and 4 remain todo. - Transaction file-cleanup units pass 18 tests. The Payload build and formatting checks pass; focused production lint retains one existing warning with no errors. - Focused MongoDB recovery coverage passes for versioned collections, versioned globals, branch-created targets, lost application progress, lost recovery progress, and pruned versions. - The primary MongoDB branching file passes 294 tests with scheduled merge tests excluded after exact version tracking and transaction-free recovery; 9 tests are skipped and 4 remain todo. - Three merge-history unit tests pass for access-checked version reads, missing target versions, and disabled target versioning. - Payload and UI builds pass. Whitespace checks pass. Focused production lint has no errors and retains three existing `any` warnings. - The primary MongoDB branching file passes 300 tests with scheduled merge tests excluded after cleanup-retry coverage; 9 tests are skipped and 4 remain todo. - Cleanup-retry tests pass for unchanged and newer collection source, deletion markers, unchanged and newer global source, and branch-created version chains. - Three focused registry-batching tests pass for ordered target hooks, a later hook reading an earlier write, stable failed and unattempted outcomes, rejected batches, and cleanup retry without repeated target writes. - All 29 MongoDB transactional-integrity tests pass after the batching integration. - The primary MongoDB branching file passes 308 tests, with 4 skipped and 4 todo, after excluding only the three known scheduled-merge transaction-conflict cases. - Recovery-limit tests pass for an intervening target edit and for a target restore failure with separate error reporting. - PostgreSQL still has the baseline disjoint global-write race failure, which can lose the seeded `navItems`; this change does not modify global storage. - Conditional SQL `updateOne` regression tests pass on SQLite and PostgreSQL, including the existing concurrent compare-and-set test. - The Payload and Drizzle type builds pass. - Targeted lint passes for the new processor, adapter implementation, and focused integration files. - Full Payload package lint still reports two existing ordering errors in `src/exports/shared.ts` and `src/uploads/types.ts`; neither file is changed here. - All 11 branching GraphQL tests pass, including field-level branch reads, writes, version restores, relationships, joins, access checks, and concurrent root-field isolation. - All 8 scheduled-merge tests pass in the complete primary MongoDB file. - All 10 rich-text reference-safety tests and all 6 branching query-cost tests pass. - The complete MongoDB branching directory passes 431 tests across 13 active files, with 2 skipped and 4 todo. The opt-in benchmark file is skipped during this ordinary suite. - Branching configuration has 10 passing unit tests, including enabled and disabled multi-tenant plugin cases, branching disabled, and a custom `tenant` field without the plugin. - The complete MongoDB branching Admin E2E suite passes 40 tests, including branch counts, nested arrays and blocks, newer drafts, version-backed history, unavailable unversioned history, and scheduled merges. - Thirteen focused merge-data, Admin component, scheduling, and history tests pass. Payload types and the UI build pass. Targeted lint has no errors and retains only existing warnings. - The focused dependency helper and grouped shadow resolver suites pass 24 unit tests. Dedicated MongoDB dependency suites pass 10 tests, and 31 focused merge-resolution, checkpoint and transactional-integrity tests pass. - `pnpm benchmark:content-branching` seeds 50,000 MongoDB content rows and submits a known 10,000-document promotion plus 10,000 registry cleanups through the ordered adapter batch contract. The completed run took 53,065 ms after 444 ms of seeding, made 20,005 adapter calls, produced a 3,639,201-byte merge ledger, increased peak heap by 913,355,264 bytes and peak RSS by 1,016,741,888 bytes, and left zero pending changes. - The scale script measures adapter batching, merge-ledger size, history pagination and memory with a known workload. It deliberately does not run 10,000 full Local API lifecycles; the ordinary MongoDB integration suites cover planning, access, hooks, validation, versions, recovery and cleanup. - The Payload package build, targeted source and benchmark lint, and whitespace checks pass after the scale changes. PostgreSQL and SQLite integration tests were not run for this unit because another agent owns the PostgreSQL run. ## Scope The MVP merge destination remains main. The work does not add branch-to-branch merges, three-way conflict resolution, or multi-tenant support. Written with AI
Written with AI
Written with AI
Written with AI
Written with AI
Written with AI
Written with AI
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds content branching to Payload. Editors can create a named branch, edit and publish documents on it in isolation, and later merge selected changes back to main. A branch edit never changes the main document. Instead, it writes a copy of the document in the same table, marked with a
_branchcolumn. The database filters reads by this column, so pagination, sorting, and document counts stay correct without extra application-level correction.Why
Editors often need to make several related changes across documents and publish them together. Content branching gives editors a private working copy of a collection or global. They can draft and review changes there, then merge the changes to main as one reviewed unit.
How
findOneAndUpdatedeprecation warning that was failing e2e tests for any suite that saves a document.Breaking changes
None. Branching stays off unless a project enables it in the config. Enabling branching on an existing collection adds new columns and branch-scoped unique indexes. This needs a normal schema migration, the same as adding any other field or index.
Alternatives considered
The design considered building content branching as a plugin. Core was chosen instead, because branch-aware joins must resolve inside the database adapters, where a plugin cannot reach.