feat(packs): candidate typescript 5 to 6 pack, judged by the target compiler (UPT-56)

.changeset/typescript-six.md

@@ -0,0 +1,5 @@
+---
+'uptide': minor
+---
+
+`check typescript` judges a TypeScript 5 to 6 upgrade with both compilers: every project of the workspace is compiled with the installed TypeScript and with the target's, and what is new at the target is a finding; the bundled compiler never stands in, and a target without a JavaScript API (TypeScript 7) is reported as not compiled. A candidate typescript 5 to 6 pack reports the tsconfig options 6.0 deprecates where they are set, the rootDir default, module-keyword namespaces and import assertions. A peer that rejects the target (typescript-eslint on TypeScript 6) is now reported as a peer conflict even when it is not grouped with the package.

README.md

@@ -68,6 +68,7 @@ The contract: [migration packs](docs/packs.md).
 | `@sentry/nextjs` | 10.x → 11.x | 100% | 100% | 8 | cohstats/coh3-stats, xdoubleu/tools.xdoubleu.com, maximunited/yomu | verified |
 | `@sentry/react` | 10.x → 11.x | 100% | 100% | 2 | owine/youth-activity-scheduler, successible/cleanslate | candidate |
 | `stripe` | 14 and newer | 100% | 90% | 16 | unkeyed/unkey, nextjs/saas-starter | verified |
+| `typescript` | 5.x → 6.x | 100% | 17% | 1399 | m-lyon/recipe, robrechtme/recipes, JosephMaynard/dependency-radar, CroffZ/github-release-notifier, SignalK/signalk-derived-data, shogo82148/typescript-action, Borewit/music-metadata | candidate |
 | `vitest` | 4.x → 5.x | 100% | 50% | 32 | votingworks/vxsuite, vitorvasc/opentelemetry-ecosystem-explorer | verified |
 | `zod` | 3.x → 4.x | 100% | 74% | 85 | ugurkocde/AwesomeIntune, howardyang2009/PATH, band-ai/band-sdk-typescript | verified |
 <!-- packs:end -->

docs/packs.md

@@ -135,7 +135,12 @@ site is reported is part of the contract (`reported`):
 
 - `finding`: `check` lists the site as the pack's own finding (evidence `pack`), found by
   the note's `detect`; `fix` never edits it unasked. Example: a Stripe client created
-  without `apiVersion`, whose API version the SDK bump changes at runtime.
+  without `apiVersion`, whose API version the SDK bump changes at runtime. A pack may
+  instead create the note's findings with `foldFindings`, which runs once the pack's
+  findings are claimed and folds compiler errors that one edit elsewhere resolves into a
+  finding at that edit, as their evidence. The typescript pack folds the "Cannot find
+  name" errors of the 6.0 `types: []` default into one at the tsconfig that leaves
+  `types` unset.
 - `decision`: the pull request lists it under "Decisions for you", with the files and lines.
 - `test-follow-up`: an assertion is updated only after the migrated code made that test
   fail, and the test then has to pass (zod's reworded default messages).
@@ -185,17 +190,21 @@ repository upgraded to.
 
 Where the expected findings come from: two sources that are not Uptide.
 
-1. **The compiler.** The repository's own TypeScript, run on every `tsconfig.json`, with the
-   target version linked where the package is installed and without it. Every diagnostic
+1. **The compiler.** The repository's own TypeScript, run on every project `check` compiles
+   (the TypeScript project scope: each `tsconfig*.json` and its references, configs that are
+   only extended left out), with the target version linked where the package is installed
+   and without it. Every diagnostic
    new at the target is a site: errors, and deprecations (6385, 6387, what an editor strikes
    through). A repository on TypeScript 7, which has no JavaScript API, is compiled with the
    TypeScript of this checkout, and the draft says so. The target is linked with its own
    dependencies at the versions it declares, as an install would leave it, and
    `--also <package>@<version>` links the companions the upgrade commit moved with it (the
    versions in its lockfile): `ai` 7 next to `@ai-sdk/react` 3 is a mix no repository ships,
-   and its errors are not the upgrade's.
+   and its errors are not the upgrade's. The tsconfig's own errors (a deprecated option,
+   TS5101) are listed too, where the compiler points: for an option an extended config sets,
+   that is the extending config, and the site is where the option is set.
 2. **The upgrade commit.** The lines it changed or removed (`git diff -U0 <commit> <upgrade>`,
-   the old side). One site per change: consecutive changed lines of the same change are one
+   the old side), in source files and `tsconfig*.json`. One site per change: consecutive changed lines of the same change are one
    site, at the first line of code (a line that is plainly another rule's starts a new
    one); a run with a diagnostic inside it, or right after a diagnostic of the same rule, is
    that diagnostic's site. Comment-only lines and changes for another package are not sites.
@@ -412,6 +421,17 @@ PR's **Verified by** row. Reverification reruns the required build. Existing pac
 without this declaration retain their current behavior. Next 16 requires this gate
 because custom webpack configuration can fail with the default Turbopack build.
 
+### When the package is the compiler
+
+`typescript` is the one package no overlay can swap: it judges every file, not the ones that
+import it. For it, `check` compiles each of the workspace's projects (the TypeScript project
+scope: every `tsconfig*.json` and its references) twice: with the repository's installed
+TypeScript, and with the target's, loaded from the copy `check` fetched. A diagnostic new at the
+target is a finding. The bundled compiler never stands in for either; a target with no
+JavaScript API (TypeScript 7) is not compiled, and the report says so. Diagnostics on tsconfig
+files are the pack's own findings, from its inventory, in the file that sets the option. The
+typescript pack's RESEARCH.md has the details.
+
 ### Release groups
 
 Use `releaseGroup` for packages published in lockstep with the pack's lead. Like

packages/core/src/adapters/typescript/compile.ts

@@ -21,10 +21,16 @@ import type {
 import { satisfies } from '../../fetch/range.js';
 import { onReset } from '../../shared-state.js';
 import { augmentationCause, findCause, parameterCause } from './cause.js';
-import { type Compiler, repositoryCompiler, unresolvedConfig } from './compiler.js';
+import {
+  type Compiler,
+  consumerCompilerDir,
+  loadCompiler,
+  repositoryCompiler,
+  unresolvedConfig,
+} from './compiler.js';
 import { jsxNamespaceCause } from './config-cause.js';
 import { type CompileProject, projectScope } from './project-scope.js';
-import { type LoadedRepo, loadedRepo, loadRepo, ownsFile } from './repo.js';
+import { inheritedTsconfigOf, type LoadedRepo, loadedRepo, loadRepo, ownsFile } from './repo.js';
 import {
   consumerCopySatisfies,
   type DependencyLinks,
@@ -787,6 +793,9 @@ export async function compileAgainstTargets(
   targets: Target[],
   options: CompileOptions = {},
 ): Promise<CompileSignal> {
+  // The compiler itself is the upgrade: both sides compile whole projects (compileCompilerUpgrade).
+  const compilerTarget = targets.find((t) => t.name === 'typescript');
+  if (compilerTarget) return compileCompilerUpgrade(repoRef, compilerTarget);
   const dir = realpathSync(repoRef.dir);
   const scope = projectScope({ ...repoRef, dir });
   if (scope.projects.length === 0) {
@@ -1129,3 +1138,126 @@ async function compileProject(
     rmSync(overlay, { recursive: true, force: true });
   }
 }
+
+/**
+ * The compiler is the upgrade (`typescript` 5 to 6): no overlay can swap it in, since it judges
+ * every file, not the ones importing it. Each side compiles the workspace's projects whole: the installed side with the repository's own TypeScript, the
+ * target side with the target's, loaded from the copy `check` fetched (the release `fix`
+ * installs). Never the bundled compiler: one the repository neither has nor will have reports
+ * errors nobody sees. A target without a JavaScript API (TypeScript 7) is not compiled, and
+ * the report says so. Diagnostics on the tsconfig files themselves (TS5101, TS5107) are left to
+ * the pack, which reports each where the option is set, in the file that sets it.
+ */
+function compileCompilerUpgrade(repoRef: RepoDir, target: Target): CompileSignal {
+  const dir = realpathSync(repoRef.dir);
+  const skip = (skipped: string, compilers: CompileCoverage['compilers'] = []): CompileSignal => ({
+    diagnostics: [],
+    baselineErrors: 0,
+    skipped,
+    coverage: { compiled: 0, total: 0, skipped: [{ reason: skipped, count: 0 }], compilers },
+    unresolvedInTarget: [],
+    unresolvedFiles: [],
+    linkedDependencies: [],
+    unsatisfiedDependencies: [],
+    timing: { baselineMs: 0, overlayMs: 0, dependenciesMs: 0 },
+  });
+  const installedDir = consumerCompilerDir(dir);
+  if (installedDir === undefined)
+    return skip('not compiled: the repository installs no TypeScript to compare the target with');
+  const before = loadCompiler(installedDir);
+  let after: typeof ts;
+  try {
+    after = loadCompiler(target.dir);
+  } catch {
+    return skip(`not compiled: typescript at ${target.dir} cannot be loaded`);
+  }
+  const compilers = [
+    { version: before.version, own: true },
+    { version: after.version ?? '?', own: true },
+  ];
+  if (typeof after.createProgram !== 'function')
+    return skip(
+      `not compiled: typescript ${after.version ?? 'target'} has no JavaScript compiler API, so its diagnostics cannot be computed; tsconfig findings still come from the pack`,
+      compilers,
+    );
+  // The workspace's projects as the TypeScript project scope finds them (project-scope.ts: every
+  // tsconfig*.json, their references, configs only extended left out): those inside this
+  // workspace, or the one it inherits. A project whose `extends` is not installed is not built.
+  const scope = projectScope({ ...repoRef, dir });
+  const inherited = existsSync(join(dir, 'tsconfig.json')) ? undefined : inheritedTsconfigOf(dir);
+  const configs = scope.projects
+    .map((p) => p.config)
+    .filter(
+      (config) =>
+        (config.startsWith(`${dir}/`) || config === inherited) &&
+        !(repoRef.exclude ?? []).some((x) => config.startsWith(`${x}/`)) &&
+        unresolvedConfig(before, config) === undefined,
+    );
+  if (configs.length === 0) return skip('not compiled: no tsconfig.json', compilers);
+
+  /** The programs of the workspace's projects, built by one compiler. */
+  const programs = (tsc: typeof ts): ts.Program[] =>
+    configs.flatMap((config) => {
+      const parsed = tsc.getParsedCommandLineOfConfigFile(
+        config,
+        {},
+        { ...tsc.sys, onUnRecoverableConfigFileDiagnostic: () => {} },
+      );
+      if (!parsed || parsed.fileNames.length === 0) return [];
+      return [
+        tsc.createProgram({
+          rootNames: parsed.fileNames,
+          options: { ...parsed.options, noEmit: true, skipLibCheck: true },
+          ...(parsed.projectReferences ? { projectReferences: parsed.projectReferences } : {}),
+        }),
+      ];
+    });
+  // The project scope applies to compiler diagnostics as it does for any package: a file no
+  // project covers is never a site. Pack findings (the tsconfig files themselves) are not these.
+  const own = (f: ts.SourceFile): boolean =>
+    !f.isDeclarationFile &&
+    !f.fileName.includes('/node_modules/') &&
+    f.fileName.startsWith(`${dir}/`) &&
+    ownsFile(repoRef, f.fileName) &&
+    scope.covers(f.fileName);
+  const errorsOf = (program: ts.Program, file: ts.SourceFile): ts.Diagnostic[] =>
+    [...program.getSyntacticDiagnostics(file), ...program.getSemanticDiagnostics(file)].filter(
+      (d) => isRepoError(d, dir),
+    );
+
+  const t0 = Date.now();
+  const baseline: ts.Diagnostic[] = [];
+  for (const program of programs(before))
+    for (const file of program.getSourceFiles().filter(own))
+      baseline.push(...errorsOf(program, file));
+  const baselineMs = Date.now() - t0;
+  const known = new Set(baseline.map((d) => keyOf(d, dir)));
+  const knownPlaces = new Set(baseline.map((d) => placeOf(d, dir)));
+  const t1 = Date.now();
+  const fresh = new Map<string, CompileDiagnostic>();
+  const compiled = new Set<string>();
+  for (const program of programs(after))
+    for (const file of program.getSourceFiles().filter(own)) {
+      compiled.add(file.fileName);
+      for (const d of errorsOf(program, file)) {
+        if (known.has(keyOf(d, dir)) || knownPlaces.has(placeOf(d, dir))) continue;
+        const diagnostic = toDiagnostic(d, file, dir);
+        fresh.set(
+          `${diagnostic.file}:${diagnostic.line}:${diagnostic.column}:${d.code}`,
+          diagnostic,
+        );
+      }
+    }
+  return {
+    diagnostics: [...fresh.values()].sort(
+      (a, b) => a.file.localeCompare(b.file) || a.line - b.line || a.column - b.column,
+    ),
+    baselineErrors: baseline.length,
+    coverage: { compiled: compiled.size, total: compiled.size, skipped: [], compilers },
+    unresolvedInTarget: [],
+    unresolvedFiles: [],
+    linkedDependencies: [],
+    unsatisfiedDependencies: [],
+    timing: { baselineMs, overlayMs: Date.now() - t1, dependenciesMs: 0 },
+  };
+}

packages/core/src/adapters/typescript/compiler-upgrade.test.ts

@@ -0,0 +1,100 @@
+import { mkdirSync, mkdtempSync, symlinkSync, writeFileSync } from 'node:fs';
+import { createRequire } from 'node:module';
+import { tmpdir } from 'node:os';
+import { dirname, join } from 'node:path';
+import { describe, expect, it } from 'vitest';
+import { createNpmFetcher } from '../../fetch/npm-fetcher.js';
+import { compileAgainstTargets } from './compile.js';
+import { consumerCompilerDir } from './compiler.js';
+
+/**
+ * `typescript` as the upgraded package: both sides compile the workspace's projects whole,
+ * the installed side with the repository's compiler and the target side with the target's.
+ */
+const devCompiler = dirname(createRequire(import.meta.url).resolve('typescript/package.json'));
+
+function repo(files: Record<string, string>, compiler: string | undefined = devCompiler): string {
+  const dir = mkdtempSync(join(tmpdir(), 'uptide-compiler-upgrade-'));
+  for (const [file, text] of Object.entries(files)) {
+    mkdirSync(dirname(join(dir, file)), { recursive: true });
+    writeFileSync(join(dir, file), text);
+  }
+  if (compiler) {
+    mkdirSync(join(dir, 'node_modules'), { recursive: true });
+    symlinkSync(compiler, join(dir, 'node_modules', 'typescript'), 'dir');
+  }
+  return dir;
+}
+
+const project = {
+  'package.json': JSON.stringify({ name: 'app', devDependencies: { typescript: '5.9.3' } }),
+  'tsconfig.json': JSON.stringify({
+    compilerOptions: { target: 'es2020', module: 'commonjs', baseUrl: '.', noEmit: true },
+    include: ['src'],
+  }),
+  // Nothing imports `typescript`: every file is judged by the compiler all the same.
+  'src/a.ts': 'module Legacy { export const x = 1; }\nexport const y = Legacy.x;\n',
+  // A project the root tsconfig never names is still built.
+  'tools/tsconfig.json': JSON.stringify({
+    compilerOptions: { module: 'commonjs', noEmit: true },
+    include: ['*.ts'],
+  }),
+  'tools/b.ts': 'export function f(a, b?: string) { return b.length + a; }\n',
+  // Outside every project: the project scope (UPT-49) keeps its compiler errors out.
+  'examples/c.ts': 'module Outside { export const z = 3; }\n',
+};
+
+describe('compiling a TypeScript upgrade', () => {
+  it('compiles every project of the workspace, not only files importing the package', async () => {
+    // The same compiler on both sides: nothing is new, and every file was compiled.
+    const signal = await compileAgainstTargets({ dir: repo(project) }, [
+      { name: 'typescript', dir: devCompiler },
+    ]);
+    expect(signal.skipped).toBeUndefined();
+    expect(signal.diagnostics).toEqual([]);
+    expect(signal.coverage.compiled).toBe(2);
+    expect(signal.coverage.compilers?.every((c) => c.own)).toBe(true);
+  });
+
+  it('never falls back to the bundled compiler when the repository installs none', async () => {
+    const dir = repo(project, undefined);
+    const signal = await compileAgainstTargets({ dir }, [{ name: 'typescript', dir: devCompiler }]);
+    // A machine may have a TypeScript in a folder above the temp directory, which Node would
+    // find too; either way it is an installed one, never the bundled compiler.
+    if (consumerCompilerDir(dir) === undefined)
+      expect(signal.skipped).toMatch(/installs no TypeScript/);
+    else expect(signal.coverage.compilers?.every((c) => c.own)).toBe(true);
+    expect(signal.diagnostics).toEqual([]);
+  });
+
+  it('says so, and compiles nothing, when the target has no JavaScript API (TypeScript 7)', async () => {
+    const seven = mkdtempSync(join(tmpdir(), 'uptide-ts7-'));
+    writeFileSync(
+      join(seven, 'package.json'),
+      JSON.stringify({ name: 'typescript', version: '7.0.2', main: 'version.cjs' }),
+    );
+    writeFileSync(join(seven, 'version.cjs'), "module.exports = { version: '7.0.2' };\n");
+    const signal = await compileAgainstTargets({ dir: repo(project) }, [
+      { name: 'typescript', dir: seven },
+    ]);
+    expect(signal.skipped).toMatch(/typescript 7\.0\.2 has no JavaScript compiler API/);
+    expect(signal.diagnostics).toEqual([]);
+  });
+});
+
+describe.skipIf(!process.env.UPTIDE_NETWORK)('compiling against TypeScript 6.0.3', () => {
+  it('reports what the target compiler rejects, from its own diagnostics', async () => {
+    const six = await createNpmFetcher().fetch('typescript', '6.0.3');
+    const signal = await compileAgainstTargets({ dir: repo(project) }, [
+      { name: 'typescript', dir: six.dir },
+    ]);
+    const codes = signal.diagnostics.map((d) => `${d.file}:${d.line} TS${d.code}`);
+    expect(codes).toContain('src/a.ts:1 TS1540');
+    // strict is on by default in 6.0: the nested project's untyped parameter.
+    expect(codes).toContain('tools/b.ts:1 TS7006');
+    expect(codes.some((c) => c.startsWith('examples/'))).toBe(false);
+    // The tsconfig's own deprecation (baseUrl, TS5101) is the pack's finding, not a compile one.
+    expect(signal.diagnostics.some((d) => d.file.endsWith('.json'))).toBe(false);
+    expect(signal.coverage.compilers?.map((c) => c.version)).toEqual(['5.9.3', '6.0.3']);
+  });
+});

packages/core/src/adapters/typescript/compiler.ts

@@ -1,4 +1,4 @@
-import { existsSync } from 'node:fs';
+import { existsSync, readFileSync } from 'node:fs';
 import { createRequire } from 'node:module';
 import { dirname, join, resolve } from 'node:path';
 import { ts } from 'ts-morph';
@@ -26,8 +26,25 @@ export function consumerCompilerDir(dir: string): string | undefined {
   }
 }
 
-function loadCompiler(packageDir: string): typeof ts {
-  return createRequire(join(packageDir, 'package.json'))(packageDir) as typeof ts;
+/** The version a package directory holds now, from its manifest. */
+function installedVersion(packageDir: string): string | undefined {
+  try {
+    return (
+      JSON.parse(readFileSync(join(packageDir, 'package.json'), 'utf8')) as { version?: string }
+    ).version;
+  } catch {
+    return undefined;
+  }
+}
+
+export function loadCompiler(packageDir: string): typeof ts {
+  const require = createRequire(join(packageDir, 'package.json'));
+  // Node keeps a loaded module by path: after `fix` installs another TypeScript at the same
+  // path (the typescript pack upgrades the compiler itself), the old module must not answer.
+  const main = require.resolve(packageDir);
+  const cached = require.cache[main]?.exports as { version?: string } | undefined;
+  if (cached && cached.version !== installedVersion(packageDir)) delete require.cache[main];
+  return require(packageDir) as typeof ts;
 }
 
 /** The consumer's compiler is authoritative; bundles carry ts-morph's compiler as fallback. */
@@ -68,7 +85,8 @@ onReset(() => compilers.clear());
  */
 export function repositoryCompiler(dir: string): Compiler {
   const own = consumerCompilerDir(dir);
-  const key = own ?? '';
+  // Keyed by version too: an install that replaced the compiler is another compiler.
+  const key = own === undefined ? '' : `${own}@${installedVersion(own) ?? ''}`;
   let compiler = compilers.get(key);
   if (!compiler) {
     const loaded = own === undefined ? ts : resolveCompiler(dir, ts, loadCompiler, () => own);