Skip to content

Commit 0f05a06

Browse files
mathieusouflismathieusouflis
committed
docs: record the catalog decision and supersede ADR-0001
ADR-0002 replaces ADR-0001, whose central claim — that instantiating a generator and reading back its output is "strictly accurate" and "can never drift" — is the thing that broke. CONTEXT.md is the glossary. It separates two concepts the codebase had been running together under one word: pin drift (our pinned version has fallen behind the registry) and generator drift (a project's generator version no longer satisfies its recorded constraint). It also names rollup and migration, which are different kinds of work rather than two sizes of the same one. docs/contributor/dep-checker.md described a test-flows gate that never existed.
1 parent e0f4e15 commit 0f05a06

4 files changed

Lines changed: 429 additions & 131 deletions

File tree

‎CONTEXT.md‎

Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
# Context
2+
3+
The domain language of `dot`. This file is a glossary, not a spec — it says what
4+
words mean, never how anything is built.
5+
6+
## Scaffolding
7+
8+
**Generator**
9+
A unit of project scaffolding. Given a set of Answers, it contributes files and
10+
configuration to a Project. Generators compose: a single Project is assembled from
11+
many of them.
12+
13+
**Flow**
14+
An ordered set of Questions presented to a user, producing the Answers that select
15+
and configure Generators.
16+
17+
**Answers**
18+
The user's responses to a Flow. Generators may behave differently depending on
19+
Answers — the same Generator can contribute different Dependencies on different runs.
20+
21+
**Project**
22+
The scaffolded output: the tree of files a user ends up with.
23+
24+
**Manifest Version**
25+
A Generator's declared semver. It describes the *behaviour* of the Generator — the
26+
shape of what it scaffolds and the Answers it responds to. A Project records the
27+
Manifest Version it was scaffolded with, so drift can be detected later.
28+
29+
## Dependencies
30+
31+
**Dependency**
32+
A third-party package that a Generator causes a Project to depend on — `react`,
33+
`vitest`, `bcryptjs`. Distinct from a Generator's own Go imports, which are ordinary
34+
source dependencies and are not part of this domain.
35+
36+
**Ecosystem**
37+
The package universe a Dependency belongs to: npm, Go, Cargo, Maven. An Ecosystem
38+
determines where a Dependency's versions are published, how its version strings are
39+
written, and what kinds of update are mechanically safe. Ecosystem is a property the
40+
Dependency *declares*, not something inferred from the files a Generator happens to
41+
write.
42+
43+
**Pin**
44+
The exact version of a Dependency that Generators scaffold, expressed in that
45+
Ecosystem's native notation. A Pin is a deliberate choice, not a resolved value.
46+
47+
**Catalog**
48+
The single place every Pin lives. There is exactly one Pin per Dependency: a
49+
Dependency cannot be pinned to one version in one Generator and another version
50+
elsewhere. Generators name the Dependencies they need; the Catalog says which
51+
version they get.
52+
53+
**Drift**
54+
Two things wearing one word — keep them apart:
55+
56+
- *Pin drift* — the Catalog's Pin has fallen behind what the Ecosystem publishes.
57+
This is expected, continuous, and is what the dependency check exists to find.
58+
- *Generator drift* — a Project was scaffolded with a Manifest Version that no
59+
longer satisfies the constraint it recorded. This is what `dot doctor` reports.
60+
61+
**Deprecation**
62+
An Ecosystem's signal that a Dependency should no longer be used. Deprecation is a
63+
judgement call for a human — unlike Pin drift, it has no mechanical fix, because the
64+
replacement (if any) is a different Dependency, not a different version.
65+
66+
## Closing Pin drift
67+
68+
Not all Pin drift is the same kind of work, and the two kinds are not
69+
interchangeable.
70+
71+
**Rollup**
72+
The single aggregate change carrying every *low-risk* Pin update — those where the
73+
Ecosystem's own versioning promises compatibility. A Rollup is disposable and
74+
machine-owned: it is rebuilt from scratch on every run to reflect whatever is
75+
currently behind, and no human ever edits it. There is at most one Rollup in flight.
76+
77+
**Migration**
78+
A *single* high-risk Pin update — one where the Ecosystem signals a break — which may
79+
require changing Generator code, not just a version string. A Migration is a piece of
80+
engineering work, not a version bump. It becomes human-owned the moment it is raised,
81+
and may stay unresolved indefinitely; an unresolved Migration is the standing record
82+
that the decision has been deferred.
83+
84+
The distinction is about *who owns the change*. The machine proposes Rollups and
85+
rewrites them at will. It proposes a Migration once, then never touches it again.
86+
87+
## Knowing when a Generator has changed
88+
89+
**Contribution**
90+
What a single Generator adds to a Project when it runs — the files it creates and the
91+
edits it makes to files other Generators own. A Project is the sum of its Generators'
92+
Contributions.
93+
94+
**Fingerprint**
95+
An identity for a Generator's Contribution, taken across every scenario the Generator
96+
is exercised in. Two Generators with the same Fingerprint scaffold the same thing.
97+
98+
A Fingerprint answers the only question that matters when deciding whether a Manifest
99+
Version must move: *did what this Generator scaffolds actually change?* Reformatting
100+
its source does not change the Fingerprint. Editing a template does. Moving a Pin the
101+
Generator names does too — which is why a Pin change is a change to every Generator
102+
that names it, even though no Generator's source was touched.
103+
104+
The Fingerprint is what makes Manifest Version meaningful: a Generator's version moves
105+
when, and only when, its Contribution does.

‎docs/adr/0001-template-dep-checker-architecture.md‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,15 @@
11
# ADR-0001: Template Dependency Checker Architecture
22

3-
**Status:** Accepted
3+
**Status:** Superseded by [ADR-0002](./0002-dependency-catalog.md)
44
**Date:** 2026-05-28
55

6+
> **Superseded 2026-07-11.** The "real-state extraction" premise below is false: calling
7+
> `Generate()` with empty `Answers` executes one arbitrary branch, not real state, so
8+
> dependencies behind conditionals were never checked. Worse, the scanner read *values* while
9+
> the patcher rewrote *source literals* — a gap that made some dependencies structurally
10+
> unpatchable, crashed the weekly run, and merged a mislabeled commit (`997e489`) to `main`.
11+
> See [ADR-0002](./0002-dependency-catalog.md).
12+
613
## Context
714

815
Generator files (e.g. `generators/express_server_typescript_deps/generator.go`) hardcode

‎docs/adr/0002-dependency-catalog.md‎

Lines changed: 175 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,175 @@
1+
# ADR-0002: Dependency Catalog
2+
3+
**Status:** Accepted
4+
**Date:** 2026-07-11
5+
**Supersedes:** [ADR-0001](./0001-template-dep-checker-architecture.md)
6+
7+
## Context
8+
9+
ADR-0001 built `tools/dep-checker` on a principle it called **real-state extraction**:
10+
instantiate each generator with an empty `VirtualProjectState`, call `Generate()`, and read
11+
back whatever `package.json` it wrote. The ADR claimed this was *"strictly accurate"* and
12+
*"can never drift"* from real output.
13+
14+
It is neither. In production the bot produced commit `997e489` — titled
15+
`chore(deps): bump @clerk/clerk-react in templates`, containing 13 files touching ark-ui,
16+
better-auth, vitest, next, react, prettier, vite, posthog and react-router, and **zero clerk
17+
changes**. It was merged to `main`.
18+
19+
Three flaws, each fatal on its own.
20+
21+
**1. Reading back an empty-Answers run is not real state.** Generators branch on
22+
`ctx.Answers`; 15 of them do. Calling `Generate()` with `Answers: map[string]interface{}{}`
23+
executes one arbitrary branch. `auth_clerk_frontend` scaffolds `@clerk/clerk-react` when no
24+
framework is chosen and `@clerk/nextjs` when it is — so `@clerk/nextjs ^7.4.2` was **never
25+
checked once** in the tool's lifetime. The scan did not observe real state; it observed one
26+
default slice of it.
27+
28+
**2. Scanning and patching disagreed about what a dependency is.** The scanner read *values*
29+
out of generated JSON. The patcher had to write back to a *Go string literal in source*, which
30+
it located by regex (`patch.go:96`). Nothing guarantees a value observed in the output exists
31+
as a literal in the source. `auth_clerk_frontend` holds its version in a variable
32+
(`version := "^5.61.3"`), so the patch could never match, and errored every run. The tool
33+
could see a dependency it was structurally incapable of patching.
34+
35+
**3. Failure was not contained.** The patch error set `PATCH_FAILED`, broke the loop, and ran
36+
`git checkout main` **without resetting the working tree** — leaving every generator patched
37+
before clerk dirty on disk. `git add generators/` in the next iteration swept them into an
38+
unrelated commit. That is `997e489`.
39+
40+
The `Deprecated:` fields, the ecosystem table, and the registry checkers all worked. The
41+
failure was entirely in *where versions live*.
42+
43+
Two further defects share the same root. Versions living in 40 hand-written files means the
44+
same package can be pinned twice at different versions, and it was: `vitest` sat at `^4.1.7`
45+
and `^4.1.8`, `bcryptjs` at `^2.4.3` and `^3.0.3` — drift the bot itself caused by dying
46+
mid-run. And `dep-checker` reimplemented semver comparison from scratch
47+
(`checkers.go:46-105`), getting zero-major wrong: `^0.45.2 → 0.46.0` is a **breaking** change
48+
under semver, but it was classified `"minor"` and batched as safe. `drizzle-orm` is pinned at
49+
`^0.45.2`.
50+
51+
## Decision
52+
53+
**Pinned versions move out of generators and into a Catalog: `internal/deps/`.**
54+
55+
There is exactly one Pin per dependency, repo-wide. Generators name the packages they need;
56+
the Catalog supplies the version.
57+
58+
```go
59+
// internal/deps/npm.go — machine-owned; the only file the bot rewrites
60+
var npm = map[string]string{
61+
"@clerk/clerk-react": "^5.61.3",
62+
"@clerk/nextjs": "^7.4.2",
63+
"vitest": "^4.1.8",
64+
}
65+
66+
// generators/react_app/generator.go
67+
"dependencies": deps.NPM("react", "react-dom"),
68+
"devDependencies": deps.NPM("vite", "@types/react"),
69+
```
70+
71+
Ecosystem becomes a property the Pin **declares**, not something inferred from which file a
72+
generator happened to write. Adding Rust means adding a `Registry` implementation, not
73+
teaching a scanner a new filename.
74+
75+
This collapses the failure modes rather than patching them:
76+
77+
- **Scan** is a Go import of the Catalog. No `Generate()`, no Answers, no branches to be blind
78+
to. `@clerk/nextjs` becomes visible because a Catalog has no `if` statements.
79+
- **Patch** rewrites one machine-owned file with one canonical shape. The value the scanner
80+
read *is* the literal the patcher writes. The reverse-mapping problem ceases to exist.
81+
- **Divergent pins become unrepresentable.** One key, one version.
82+
- **Classification** delegates to `internal/versioning`, which already implements caret and
83+
zero-major correctly and is tested (`TestConstraint_Caret_ZeroMajor`). A Pin update is
84+
low-risk **iff the new version satisfies the existing Pin's constraint** — which is what the
85+
caret already means. `^0.45.2` does not permit `0.46.0`, so Drizzle is correctly treated as
86+
breaking, with no special case.
87+
88+
### Proposing changes
89+
90+
| Kind | Vehicle | Ownership |
91+
|---|---|---|
92+
| Satisfies the existing constraint | **Rollup** — one batched PR, all packages | Bot. Force-rebuilt every run. Excludes any package holding an open dep PR of its own. |
93+
| Breaks the existing constraint | **Migration** — one PR per package | Human, from the moment it is raised. Bot never touches it again. |
94+
| Deprecated | Issue only, **never a PR** | Human. |
95+
96+
A deprecated package is excluded from the Rollup entirely. Every deprecation in the repo today
97+
(`@clerk/clerk-react` → `@clerk/react`, `@vercel/flags` → `flags`, `@types/bcryptjs` → delete,
98+
`plausible-tracker` → replace) needs a **rename or a removal**. ADR-0001's rule — *"open the
99+
version-bump PR if a newer version exists under the same name"* — resolves none of them:
100+
bumping to the latest *deprecated* version fixes nothing.
101+
102+
A Rollup whose CI goes red is not bisected by the bot. A human ejects the offending package
103+
into its own Migration PR, and the next Rollup rebuild drops it via the same open-PR check
104+
that protects Migrations. One rule, both cases.
105+
106+
### Manifest bumps are derived at release, not stored in the PR
107+
108+
ADR-0001 bumped `manifest.go` inside the dep PR. That stores a **relative** operation
109+
(`0.8.0 → 0.9.0`) computed against `main` at PR-creation time and applies it at *merge* time,
110+
when `main` has moved. With several dep PRs open, the resulting version is a function of merge
111+
order. Long-lived Migrations make this unsurvivable: a React Migration open for two months
112+
collides with every weekly Rollup that touches the same manifest.
113+
114+
Manifest bumps therefore move to **release**, derived from the final state:
115+
116+
```
117+
dot gen-bump # every generator whose Fingerprint moved since the last tag, bumped once
118+
```
119+
120+
Merge order becomes provably irrelevant, because the bump is no longer a stored delta. A
121+
Migration PR touches only its own Catalog line, so it rebases cleanly for months — the Rollup
122+
edits *other lines of the same map*.
123+
124+
### Fingerprints
125+
126+
A **Fingerprint** is a hash of a generator's Contribution — the files it introduces — taken
127+
across every fixture that invokes it, computed by diffing `VirtualProjectState` before and
128+
after the generator runs. It is entirely in-memory: no `pnpm install`, milliseconds.
129+
130+
The Fingerprint answers the only question that matters: *did what this generator scaffolds
131+
actually change?* Reformatting source does not move it. Editing a template does. Moving a Pin
132+
the generator names does too.
133+
134+
Three CI rules follow:
135+
136+
1. **PR-time** — if the diff touches `generators/**` and a Fingerprint moved, `manifest.go`
137+
must be bumped and the doc's version row must match.
138+
2. **Always** — every generator's doc version row equals its `manifest.go` Version.
139+
3. **Release-time** — every generator whose Fingerprint moved since the last tag has a bumped
140+
manifest and a synced doc.
141+
142+
Rule 1 deliberately **does not fire** on a diff touching only `internal/deps/**`. Enforcing a
143+
manifest bump inside a dep PR is precisely the merge-order bug this ADR exists to remove.
144+
145+
## Alternatives considered
146+
147+
| Option | Reason rejected |
148+
|---|---|
149+
| Keep inline literals; ban variable-held versions with a lint | Fixes the clerk crash, but leaves the scanner blind to conditional deps and the regex patcher load-bearing. Treats the symptom. |
150+
| Run generators under many Answer permutations to flush out hidden deps | Machinery to recover information a Catalog never loses. Enumerating permutations is a combinatorial guess; a Catalog is a list. |
151+
| Per-generator `Dependencies` field in `manifest.go` | ADR-0001 rejected this as *"a second source of truth that can drift"* — correctly, but it drew the wrong conclusion. Drift came from versions living in **40** places. The fix is **one** place, not one-per-generator. |
152+
| Bot bisects a red Rollup automatically | ~5 extra full `test-flows` runs (37 fixtures, real `pnpm install`) per failure, to tell a human what the CI log already says. |
153+
| Allow multiple pinned variants (`bcryptjs@2`, `bcryptjs@3`) | All four divergent pins in the repo today are bugs, not intent. No legacy templates are planned. Add the escape hatch when a use for it exists. |
154+
155+
## Consequences
156+
157+
- **Generators no longer contain versions.** A contributor adding a package adds a Pin to the
158+
Catalog and names it in the generator. `deps.NPM()` panics on an unknown name, so a typo
159+
fails at generate time and `test-flows` catches it.
160+
- **One version per package, repo-wide.** If a genuine need to pin an old major appears, it
161+
requires a deliberate new Catalog key — it cannot happen by accident, which is how the
162+
current `vitest` and `bcryptjs` divergence happened.
163+
- **Between releases, `main` carries generators whose Pin moved but whose manifest has not.**
164+
This is the price of order-proof bumps. No user observes it: users consume released
165+
binaries, and `doctor` compares against the released manifest.
166+
- **The bot never runs `test-flows`.** The PR's own CI already does. ADR-0001's promised gate
167+
(run `test-flows`, open an issue on failure) would double the most expensive job in CI for
168+
no added signal. It was never implemented, and it should not be.
169+
- **Registry traffic drops to one request per package**, instead of one per
170+
(generator, package) pair.
171+
- **Cargo, Maven and Go are dead code today** — no generator writes `Cargo.toml`, `pom.xml`, or
172+
`go.mod`. The `Registry` interface keeps them cheap to revive; the unreachable extractors go.
173+
- `tools/dep-checker` keeps its own semver logic **nowhere**. `parseSemver`, `isOutdated`,
174+
`updateType`, `stripConstraintPrefix` and `depBumpIsMajor` are deleted in favour of
175+
`internal/versioning`.

0 commit comments

Comments
 (0)