Skip to content

fix(fix): judge consumed builds on the final head, not after the bump (UPT-104) - #226

Merged
lucasbraum merged 2 commits into
mainfrom
upt-104-final-head-build
Oct 18, 2026
Merged

lucasbraum merged 2 commits into
mainfrom
upt-104-final-head-build

Conversation

@lucasbraum

Copy link
Copy Markdown
Collaborator

Closes UPT-104.

What and why

fix builds the workspaces whose output a consumer reads (prepareWorkspace) right after the bump, before any edit, so the agent's type-check sees fresh declarations. Those results were then reported as the run's build and turned into Build cannot run (…) fallback reasons, and nothing ever built them again. When the edits fixed the build, the run still:

  • forced a feedback round (round === 1 && fallback.length) with nothing left to fix,
  • put an Agent-led draft banner at the top of the PR body,
  • showed Build after the bump … — failed / ❌ build in the PR body.

That's what happened on calmcp (#219) and on dms-saas (#221's final run, uptide-bench fork PR #8).

Change (packages/core/src/fix/run.ts)

  • The consumed builds are judged on the head the edits left. They run again before the verdict and after each feedback round, and the reported verification.build holds those results.
  • Build fallback reasons are recomputed from the latest builds after each build, so a reason the head no longer shows is dropped. The reasons that don't depend on the head (planner refusals, agent-led pins, unknown baseline) stay.
  • The consumed builds run again only when the tree changed since their last result: HEAD's tree plus the diff and untracked names over it, marked ponytail: because an untracked file edited in place goes unseen. A run whose after-bump build passed and that made no later edits does not build again.
  • When the verification build (generic or requiresBuild runs) already built the same workspaces on the head, its results are used instead of a separate build.
  • New report field buildRuns: { count, ms }: every build the run executed (preparation and verification) and their wall time.

Tests

fix.test.ts, consumed workspace builds on the final head (UPT-104):

  1. The after-bump build fails and the fixed head builds: no feedback round, no agentFallback, the agent is never called, the build line says passed, no banner, and 3 builds (baseline, after the bump, edited head).
  2. The after-bump build passes and nothing changes afterwards: no further build (2 builds).
  3. A requiresBuild pack: the consumed workspace is built once on the head, by the verification build.

Tests 2 and 3 each fail with their guard removed. The existing test that counted 2 builds now expects 3, with the third on the edited head. pnpm test, pnpm typecheck and pnpm lint pass; the only lint warnings are the two that predate this change.

Bench (#169 harness, Uptide arm only)

Same harness, model (claude-sonnet-5-5, medium) and $1 cap as #169. Before is origin/main 2dc2162 plus only the buildRuns counter (a local commit, not pushed); after is this branch. One run each, sequential.

Run Rounds Cost Wall Builds (count / time) Fallback banner PR body build line Verdict
calmcp before 1 $0.117 76 s 1 / 0.5 s yes: Build cannot run (.), zodDoc.ts TS2322 . tsc -p tsconfig.json — failed (❌ build) PASS
calmcp after 0 $0.082 58 s 2 / 0.7 s none . tsc -p tsconfig.json — passed (✅ build) PASS
dms-saas before 1 $0.391 300 s 2 / 8.6 s yes: Build cannot run (packages/dms-saas), the stripe errors the agent then fixed packages/dms-saas — failed (❌ build) FAIL
dms-saas after 0 $0.294 256 s 3 / 15.5 s none packages/dms-saas — passed (✅ build) PASS

Phases:

  • calmcp before: analyze 5 s, baseline 3 s, install 6 s, rules 0 s, agent 33 s ($0.089), verify 2 s, feedback 16 s ($0.028)
  • calmcp after: analyze 6 s, baseline 3 s, install 6 s, rules 0 s, agent 35 s ($0.082), verify 2 s
  • dms-saas before: analyze 23 s, baseline 26 s, install 21 s, rules 0 s, agent 120 s ($0.271), verify 26 s, feedback 69 s ($0.120)
  • dms-saas after: analyze 25 s, baseline 30 s, install 24 s, rules 0 s, agent 128 s ($0.294), verify 35 s

The extra build is the head rebuild: +0.2 s on calmcp and +6.9 s on dms-saas. It replaces a feedback round of 16 s / $0.028 and 69 s / $0.120.

How to read the dms-saas rows:

  • The verdicts differ because of the agent's edits, not this change. Before, the agent bumped the API version and a basil-payloads test failed as new. After, 3 failing test files were marked pre-existing and the run passed. Both runs left 0 new type errors.
  • The first after run on dms-saas failed and is not in the table. It exited 2 after 423 s with Required migration evidence exceeds the PR body budget. A run that throws removes its clone, so no report survived, and its spend went unrecorded (no --json output). I reran it once with --keep-temp (no other difference): that's the row above, with a 31 KB PR body. The overflow is the open "PR body overflow" follow-up from UPT-101, and I couldn't attribute it to this change: removing a fallback reason only shortens the body. Nothing here fixes it.

No fork PRs were opened for these runs, and nothing was pushed outside this branch.

🤖 Generated with Claude Code

lucasbraum and others added 2 commits October 17, 2026 18:57
… (UPT-104)

The workspace builds prepareWorkspace runs right after the bump were
reported as the run's build and turned into "Build cannot run" fallback
reasons, even when the edits fixed them. That forced a no-op feedback
round, an "Agent-led draft" banner and "Build failed" in the PR body.

Rebuild them on the head before the verdict and after each feedback
round, and recompute the build fallback reasons from those builds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Lucas Braum <lucasbraum@gmail.com>
…-104)

The final-head rebuild ran even when nothing changed after a passing
after-bump build, and beside a verification build that already built
the same workspaces. Keep the last result with the tree state it was
taken on and build again only when that state changed; take the
verification build's results when it covers those workspaces.

The report's buildRuns counts every build the run executed and their
wall time, so runs can be compared.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Lucas Braum <lucasbraum@gmail.com>
@lucasbraum
lucasbraum merged commit 448c3cb into main Oct 18, 2026
12 checks passed
@lucasbraum
lucasbraum deleted the upt-104-final-head-build branch October 18, 2026 06:36
@uptide-release uptide-release Bot mentioned this pull request Oct 18, 2026
lucasbraum pushed a commit that referenced this pull request Oct 18, 2026
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.


# Releases
## uptide@0.15.1

### Patch Changes

- [#220](#220)
[`cf86d65`](cf86d65)
Thanks [@lucasbraum](https://github.com/lucasbraum)! - With an assisting
agent, a root pin Uptide cannot update safely (a nested override such as
`overrides > pkg > peerDependencies > react`) no longer stops `fix` with
exit 2: the run continues as an agent-led draft that leaves the pin as
written and names it, with its exact path and value, at the top of the
PR body. The agent-led peer reason now names each blocker's first
release that accepts the target, or says none does. An agent-led draft
must pass the repository's clean install (`npm ci` or the manager's
frozen install) before delivery; if it does not, the PR says so at the
top. Without an agent, with `--no-llm`, or on a safety refusal, `fix`
still exits 2.

- [#232](#232)
[`e4a2885`](e4a2885)
Thanks [@lucasbraum](https://github.com/lucasbraum)! - `uptide fix`
spends less wall time on test runs and dependency copies:
  
- The agent's own `run_tests` check runs only its runner's related tests
when the `test` script builds first and then runs a script that is just
the runner (`pnpm run build && pnpm run test:unit`). The gate after the
agent's edits and the final verification run exactly what they ran
before.
- The upgrade's install moves the installed `node_modules` out of its
temporary worktree instead of copying it.
- Each phase of the fix report lists the repository's commands it ran
(installs, generators, builds, type-checks, tests, lint, format) with
their count and summed wall time. An agent-led run's clean install has
its own phase.

- [#223](#223)
[`2dc2162`](2dc2162)
Thanks [@lucasbraum](https://github.com/lucasbraum)! - `fix` no longer
stops with "baseline tests modified the working tree" when the test run
only writes new untracked files, such as a junit `test-report.xml` that
is not gitignored. Those files are recorded as test artifacts
(`testArtifacts` in the report), kept out of every commit, the agent's
diff and the pending-changes check, and removed before each later test
run, `uptide verify` included. A change to a tracked file during
baseline tests still refuses the run.

- [#215](#215)
[`abddcb3`](abddcb3)
Thanks [@lucasbraum](https://github.com/lucasbraum)! - `check` no longer
lists possible impact (type changes nothing confirmed as breaking) on
its first screen: each package says it in one line, `+N possible impact
in M files, not counted as breaking`, and `--details` and `--json` still
have every site. A package whose types were partly verified no longer
repeats its coverage line in the heading, and a monorepo's unbuilt
workspace chains are one count on that line.

- [#213](#213)
[`2de3a18`](2de3a18)
Thanks [@lucasbraum](https://github.com/lucasbraum)! - A companion pack
that moves with the lead's fix (react beside next) now gets its own scan
context and inventory, and the PR body carries a section with its review
notes.

- [#212](#212)
[`733cfbe`](733cfbe)
Thanks [@lucasbraum](https://github.com/lucasbraum)! - `pack test
--write` records how many breaking sites a finding landed on at any
severity and as breaking, out of all breaking sites it scored, and the
pack tables show both ("Found, any severity", "Found as breaking") in
place of the single recall column. Packs recorded before it show `—`.

- [#224](#224)
[`09f68df`](09f68df)
Thanks [@lucasbraum](https://github.com/lucasbraum)! - `fix` on npm no
longer stops when the lock-only install hits ERESOLVE from the old
lockfile. It retries without the lockfile entries of the moved packages,
their companions and the packages whose peers they satisfy, and only
then with the whole lockfile re-resolved; the report says which step
worked, and the lockfile check reports the churn as usual. It never uses
`--legacy-peer-deps` or `--force`. When every step fails, `fix` ends
blocked, naming the conflicting packages from npm's ERESOLVE report.
`fix --json` now prints JSON on every exit, a failure included; when the
run already has its report (a PR body over GitHub's size limit), that
report comes with the error.

- [#221](#221)
[`ff8de8e`](ff8de8e)
Thanks [@lucasbraum](https://github.com/lucasbraum)! - `fix` no longer
rejects an agent patch for test files the parent commit already failed,
whichever test scope ran them: when the test script builds first and the
build failed on the error being fixed, the start had no test results to
compare with, so a correct patch was reverted every round. The patch's
tests also run with the pack's follow-ups in place (a test asserting the
old Stripe API version), which `fix` still applies and reports after the
agent. When a PR body cannot fit, the stored report.json the error
points to is now written first. `check` and `fix` now also analyze files
that reach a package's wrapper through a barrel (`import {
getStripeClient } from '../stripe'` with `export * from './client'`).
Edited files are formatted with oxfmt and dprint as well as biome and
prettier, from the workspace that holds them, with its own
configuration, else the root's. The stripe pack adds `request-options`:
request options passed in the params slot (`subscriptions.cancel(id, {
idempotencyKey })`), which SDK 22 rejects, move to the third argument.
It runs assisted until a ground-truth repository exercises it.

- [#226](#226)
[`448c3cb`](448c3cb)
Thanks [@lucasbraum](https://github.com/lucasbraum)! - `uptide fix`
judges consumed workspace builds on the final head, not on the tree
right after the bump. A build that failed after the bump and passes once
the edits land no longer starts a feedback round, adds an "Agent-led
draft" reason or shows "Build failed" in the PR body. The builds run
again only when the tree changed since their last result, and not at all
when the verification build already built those workspaces. The fix
report's `buildRuns` gives the number of builds the run executed and
their wall time.

- [#218](#218)
[`13e343b`](13e343b)
Thanks [@lucasbraum](https://github.com/lucasbraum)! - `check` folds a
compile error into the breaking site it comes from when the compiler
traces its value there (an `errors.map(e => e.message)` under a
`result.error.errors` that zod 4 no longer types, and the narrowing that
depended on it). The rule's line shows `+N downstream errors`,
`--details` lists them under their site, and `--json` carries them as
that finding's `downstream`. They are no longer separate breaking sites.
`fix` still verifies against every error.

- [#214](#214)
[`195ae10`](195ae10)
Thanks [@lucasbraum](https://github.com/lucasbraum)! - A rewrite rule
below 80% precision over at least 5 scored ground-truth sites now runs
as assisted in `fix` (react-router `dom-imports`); smaller samples (next
`middleware-proxy`) stay mechanical and are flagged in the compatibility
table. Pack status is unchanged.

- [#218](#218)
[`13e343b`](13e343b)
Thanks [@lucasbraum](https://github.com/lucasbraum)! - pack test runs a
ground-truth repository's code generators (prisma generate, local
graphql-codegen) after its install, as fix does. It fetches the Prisma
engines the locked version needs itself, so generators stay off the
network. A repository whose generator still fails is incomplete and
fails the pack, unless its ground truth declares `withoutGenerated`

- [#227](#227)
[`ae175a9`](ae175a9)
Thanks [@lucasbraum](https://github.com/lucasbraum)! - A migration whose
evidence is larger than GitHub's PR description limit no longer stops
`fix` with "Required migration evidence exceeds the PR body budget" and
loses the run. Long required lists get a compact form when they do not
fit. Lockfile churn shows counts per kind of change and the first
packages. Manual sites are grouped by rule, with counts and the first
locations. Remaining type errors show the first entries. Every full list
stays in report.json and, in the Uptide Action, in the job summary,
which the description links to. If even the compact form does not fit,
the description is a short summary that says so at the top: the verdict,
the checks, the counts and where the full report is. The branch and
report.json are always kept.
  
An agent's decisions under "Decisions for you" name the files the agent
changed, five at most, and count the rest, instead of every file in the
task's scope. A verification-feedback task scoped to the whole
repository made each of its decisions a list of hundreds of files.

- [#219](#219)
[`12dc612`](12dc612)
Thanks [@lucasbraum](https://github.com/lucasbraum)! - `fix zod`
rewrites `z.string().uuid()` to `z.guid()` by rule without
`--include-deprecated`: zod 4's `uuid()` also checks the RFC version and
variant, so ids zod 3 accepted stop validating, and `check` now counts
the site as breaking. `fix` no longer reports false pre-existing type
errors on a TypeScript 7 repository whose `lib` names a library newer
than its `target` (`lib: ["ES2023"]` under `ES2022`).

Co-authored-by: uptide-release[bot] <338861287+uptide-release[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant