Repository navigation
fix: untracked files written by baseline tests are test artifacts (UPT-105) - #223
Merged
Merged
Conversation
…T-105) A test run that writes a report nobody gitignores (enki's vitest junit reporter writes test-report.xml) stopped fix with "baseline tests modified the working tree". Untracked files are now snapshotted before the baseline tests; new ones are recorded as testArtifacts, hidden from git through the private clone's info/exclude (so no commit, agent diff, follow-up, style step or verificationPending check sees them) and removed before each later test run, uptide verify included. A change to a tracked file during baseline tests still refuses. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Lucas Braum <lucasbraum@gmail.com>
Merged
lucasbraum
pushed a commit
that referenced
this pull request
Oct 18, 2026
This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and publish to npm yourself or [setup this action to publish automatically](https://github.com/changesets/action#with-publishing). If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. # Releases ## uptide@0.15.1 ### Patch Changes - [#220](#220) [`cf86d65`](cf86d65) Thanks [@lucasbraum](https://github.com/lucasbraum)! - With an assisting agent, a root pin Uptide cannot update safely (a nested override such as `overrides > pkg > peerDependencies > react`) no longer stops `fix` with exit 2: the run continues as an agent-led draft that leaves the pin as written and names it, with its exact path and value, at the top of the PR body. The agent-led peer reason now names each blocker's first release that accepts the target, or says none does. An agent-led draft must pass the repository's clean install (`npm ci` or the manager's frozen install) before delivery; if it does not, the PR says so at the top. Without an agent, with `--no-llm`, or on a safety refusal, `fix` still exits 2. - [#232](#232) [`e4a2885`](e4a2885) Thanks [@lucasbraum](https://github.com/lucasbraum)! - `uptide fix` spends less wall time on test runs and dependency copies: - The agent's own `run_tests` check runs only its runner's related tests when the `test` script builds first and then runs a script that is just the runner (`pnpm run build && pnpm run test:unit`). The gate after the agent's edits and the final verification run exactly what they ran before. - The upgrade's install moves the installed `node_modules` out of its temporary worktree instead of copying it. - Each phase of the fix report lists the repository's commands it ran (installs, generators, builds, type-checks, tests, lint, format) with their count and summed wall time. An agent-led run's clean install has its own phase. - [#223](#223) [`2dc2162`](2dc2162) Thanks [@lucasbraum](https://github.com/lucasbraum)! - `fix` no longer stops with "baseline tests modified the working tree" when the test run only writes new untracked files, such as a junit `test-report.xml` that is not gitignored. Those files are recorded as test artifacts (`testArtifacts` in the report), kept out of every commit, the agent's diff and the pending-changes check, and removed before each later test run, `uptide verify` included. A change to a tracked file during baseline tests still refuses the run. - [#215](#215) [`abddcb3`](abddcb3) Thanks [@lucasbraum](https://github.com/lucasbraum)! - `check` no longer lists possible impact (type changes nothing confirmed as breaking) on its first screen: each package says it in one line, `+N possible impact in M files, not counted as breaking`, and `--details` and `--json` still have every site. A package whose types were partly verified no longer repeats its coverage line in the heading, and a monorepo's unbuilt workspace chains are one count on that line. - [#213](#213) [`2de3a18`](2de3a18) Thanks [@lucasbraum](https://github.com/lucasbraum)! - A companion pack that moves with the lead's fix (react beside next) now gets its own scan context and inventory, and the PR body carries a section with its review notes. - [#212](#212) [`733cfbe`](733cfbe) Thanks [@lucasbraum](https://github.com/lucasbraum)! - `pack test --write` records how many breaking sites a finding landed on at any severity and as breaking, out of all breaking sites it scored, and the pack tables show both ("Found, any severity", "Found as breaking") in place of the single recall column. Packs recorded before it show `—`. - [#224](#224) [`09f68df`](09f68df) Thanks [@lucasbraum](https://github.com/lucasbraum)! - `fix` on npm no longer stops when the lock-only install hits ERESOLVE from the old lockfile. It retries without the lockfile entries of the moved packages, their companions and the packages whose peers they satisfy, and only then with the whole lockfile re-resolved; the report says which step worked, and the lockfile check reports the churn as usual. It never uses `--legacy-peer-deps` or `--force`. When every step fails, `fix` ends blocked, naming the conflicting packages from npm's ERESOLVE report. `fix --json` now prints JSON on every exit, a failure included; when the run already has its report (a PR body over GitHub's size limit), that report comes with the error. - [#221](#221) [`ff8de8e`](ff8de8e) Thanks [@lucasbraum](https://github.com/lucasbraum)! - `fix` no longer rejects an agent patch for test files the parent commit already failed, whichever test scope ran them: when the test script builds first and the build failed on the error being fixed, the start had no test results to compare with, so a correct patch was reverted every round. The patch's tests also run with the pack's follow-ups in place (a test asserting the old Stripe API version), which `fix` still applies and reports after the agent. When a PR body cannot fit, the stored report.json the error points to is now written first. `check` and `fix` now also analyze files that reach a package's wrapper through a barrel (`import { getStripeClient } from '../stripe'` with `export * from './client'`). Edited files are formatted with oxfmt and dprint as well as biome and prettier, from the workspace that holds them, with its own configuration, else the root's. The stripe pack adds `request-options`: request options passed in the params slot (`subscriptions.cancel(id, { idempotencyKey })`), which SDK 22 rejects, move to the third argument. It runs assisted until a ground-truth repository exercises it. - [#226](#226) [`448c3cb`](448c3cb) Thanks [@lucasbraum](https://github.com/lucasbraum)! - `uptide fix` judges consumed workspace builds on the final head, not on the tree right after the bump. A build that failed after the bump and passes once the edits land no longer starts a feedback round, adds an "Agent-led draft" reason or shows "Build failed" in the PR body. The builds run again only when the tree changed since their last result, and not at all when the verification build already built those workspaces. The fix report's `buildRuns` gives the number of builds the run executed and their wall time. - [#218](#218) [`13e343b`](13e343b) Thanks [@lucasbraum](https://github.com/lucasbraum)! - `check` folds a compile error into the breaking site it comes from when the compiler traces its value there (an `errors.map(e => e.message)` under a `result.error.errors` that zod 4 no longer types, and the narrowing that depended on it). The rule's line shows `+N downstream errors`, `--details` lists them under their site, and `--json` carries them as that finding's `downstream`. They are no longer separate breaking sites. `fix` still verifies against every error. - [#214](#214) [`195ae10`](195ae10) Thanks [@lucasbraum](https://github.com/lucasbraum)! - A rewrite rule below 80% precision over at least 5 scored ground-truth sites now runs as assisted in `fix` (react-router `dom-imports`); smaller samples (next `middleware-proxy`) stay mechanical and are flagged in the compatibility table. Pack status is unchanged. - [#218](#218) [`13e343b`](13e343b) Thanks [@lucasbraum](https://github.com/lucasbraum)! - pack test runs a ground-truth repository's code generators (prisma generate, local graphql-codegen) after its install, as fix does. It fetches the Prisma engines the locked version needs itself, so generators stay off the network. A repository whose generator still fails is incomplete and fails the pack, unless its ground truth declares `withoutGenerated` - [#227](#227) [`ae175a9`](ae175a9) Thanks [@lucasbraum](https://github.com/lucasbraum)! - A migration whose evidence is larger than GitHub's PR description limit no longer stops `fix` with "Required migration evidence exceeds the PR body budget" and loses the run. Long required lists get a compact form when they do not fit. Lockfile churn shows counts per kind of change and the first packages. Manual sites are grouped by rule, with counts and the first locations. Remaining type errors show the first entries. Every full list stays in report.json and, in the Uptide Action, in the job summary, which the description links to. If even the compact form does not fit, the description is a short summary that says so at the top: the verdict, the checks, the counts and where the full report is. The branch and report.json are always kept. An agent's decisions under "Decisions for you" name the files the agent changed, five at most, and count the rest, instead of every file in the task's scope. A verification-feedback task scoped to the whole repository made each of its decisions a list of hundreds of files. - [#219](#219) [`12dc612`](12dc612) Thanks [@lucasbraum](https://github.com/lucasbraum)! - `fix zod` rewrites `z.string().uuid()` to `z.guid()` by rule without `--include-deprecated`: zod 4's `uuid()` also checks the RFC version and variant, so ids zod 3 accepted stop validating, and `check` now counts the site as breaking. `fix` no longer reports false pre-existing type errors on a TypeScript 7 repository whose `lib` names a library newer than its `target` (`lib: ["ES2023"]` under `ES2022`). Co-authored-by: uptide-release[bot] <338861287+uptide-release[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes UPT-105. Refs UPT-102 (not closed: see the enki rerun below).
What and why
On enki (UpgradeBench, #169), the vitest junit reporter writes
test-report.xml, which the repository does not gitignore. The baseline-tests guard treated any untracked file as a change, sofixstopped with "baseline tests modified the working tree" before doing anything.Now:
untrackedFiles). After the run, a change to a tracked file still refuses, as before. New untracked files are test artifacts (testArtifactsinverify.ts)..git/info/exclude, anchored at the repo top (with the project's subdirectory prefix, and pattern characters escaped). No status, diff,git addorls-files --exclude-standardsees them. That keeps them out of commits, the agent's diff (including tests the agent runs itself), the follow-up and style steps, and theverificationPendingcheck, from one place.clearing()wrapsservices.testsonce, after baseline, so settle, the rules' transactions, the agent gate and the feedback rounds all go through it.testArtifacts).uptide verifyruns in a fresh clone, so it re-applies the exclude and clearing from the stored run; otherwise its final test run would leave the file and fail.pin.ts).fixandverifyalways run in a private clone (isolatedFix/isolatedVerify), so writinginfo/excludenever touches the user's checkout. The comment there names the ceiling if either ever runs in place.Tests
fix.test.ts: the fixture's test script writes an untrackedtest-report.xmland fails if the previous run's file is still there. Baseline and final verification pass,testArtifactsis['test-report.xml'], the file is in no commit and not inchangedFiles, nothing is pending.uptide verifyfrom a fresh clone (no exclude entry, no file on disk) also passes.fix.test.ts: a baseline test run that writes an untracked report and changes a tracked file still refuses.verify.test.ts: a project in a subdirectory, with names containing[ ] # ! space, is hidden and nothing else is.pnpm lint,pnpm typecheck,pnpm test: green locally (core 2268 passed).enki rerun (#169 harness, Uptide arm only, claude-sonnet-5-5, $1 cap)
Run
7f90b41c, from this branch's build (8d1d8bb).@entur/react-component-toggle1.0.0:npm install --package-lock-only→ERESOLVE, exit 2The cause, reproduced outside Uptide: starting from the parent's
package-lock.json,npm install --package-lock-onlyfails with the sameERESOLVEeven with the human upgrade's exact package.json (ed37dd2). Without the old lockfile, the same manifest resolves (exit 0). So Uptide's companion set matches the human's. What blocks is npm's resolution starting from the old lockfile, not the plan. Two defects show up here: the install step has no fallback for this, andfix --jsonprinted nothing on this exit 2. The run ended as a raw npm error instead of a "blocked by" result.UPT-102's done-criterion (a draft that installs, or a clear "blocked by") is not met yet, so this PR does not close it.
🤖 Generated with Claude Code