fix(checkout): fetch PR base branch so origin/<baseBranch> resolves (closes #74) (#96)

Co-authored-by: chrisleekr-bot[bot] <2890262+chrisleekr-bot[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: chrisleekr-bot[bot] <chrisleekr-bot[bot]@users.noreply.github.com>

Author: 3 people

CLAUDE.md

@@ -40,7 +40,7 @@ Single HTTP server (`src/app.ts`) using `octokit` App class. Webhook events arri
 2. Get installation token
 3. Fetch PR/issue data via GraphQL
 4. Build prompt with full context
-5. Clone repo to temp directory, checkout PR/default branch
+5. Clone repo to temp directory, checkout PR/default branch (and supplementally fetch the PR base branch when it differs from head, so `origin/<baseBranch>` resolves for diffs/rebases)
 6. Resolve MCP servers and allowed tools
 7. Run Claude Agent SDK with `cwd` set to cloned repo
 8. Finalize tracking comment (success/error/cost)

IMPLEMENT.md

@@ -1,56 +1,44 @@
-# Implement: issue #66 — paginate GraphQL fetcher + MAX_FETCHED\_\* caps
+# Implement: issue #74 — supplemental fetch of PR base branch in checkout
 
-## Summary
-
-Closes #66 (`fix(pipeline): GraphQL fetcher silently truncates PR/issue context past 100 items`).
-
-`src/core/fetcher.ts` previously issued single-page GraphQL requests with `first: 100` on every connection (issue/PR comments, reviews, the inline comments nested under each review, and changed files). Anything past the first 100 items was silently dropped before the data ever reached the agent — the prompt looked complete but was missing context, which is the failure mode the issue calls out.
-
-The fix:
+Closes #74 (`fix(pipeline): shallow single-branch clone omits origin/baseBranch, breaking PR diffs and auto-rebase`).
 
-1. Every paginated query exposes a single cursor variable named exactly `$cursor` and selects `pageInfo { hasNextPage endCursor }` on its one paginated connection. Both names are part of `@octokit/plugin-paginate-graphql`'s contract: the plugin hard-codes the cursor parameter name (`iterator.js`) and runs a depth-first search for the first `pageInfo` it finds (`object-helpers.js`), so multi-connection / aliased-cursor queries silently truncate or throw. The PR fetch is therefore split into three parallel `paginate(...)` calls (`PR_FIRST_QUERY` for top-level scalars + files, `PR_COMMENTS_QUERY`, `PR_REVIEWS_QUERY`); the issue fetch uses one. The plugin is bundled with `octokit ^5.0.5` and exposed on every `octokit.graphql` instance.
-2. A separate `REVIEW_COMMENTS_QUERY` walks each review's overflow inline comments keyed on the review node ID — the nested per-review pagination cannot ride along on `PR_REVIEWS_QUERY` because of contract-rule #1.
-3. Four new env vars (`MAX_FETCHED_COMMENTS` / `_REVIEWS` / `_REVIEW_COMMENTS` / `_FILES`, default `500` each) cap the merged result. When a cap fires the fetcher emits `log.warn({ connection, fetched, cap })` and sets `FetchedData.truncated.<connection> = true`.
-4. `buildPrompt` in `src/core/prompt-builder.ts` reads `data.truncated` and prepends a `WARNING: pre-fetched context is incomplete…` banner naming the affected connections, so the agent knows to reach for the GitHub CLI when it needs the missing items.
-5. The TOCTOU filter (`filterByTriggerTime`) runs after the paginate merge, so its semantics are unchanged.
-
-## Plan deviations
+## Summary
 
-- **T1 / T2 (install + thread plugin) skipped as no-ops.** `octokit ^5.0.5` already bundles `@octokit/plugin-paginate-graphql` and exposes `octokit.graphql.paginate` on every existing instance. Verified via lockfile inspection and a runtime probe. Threading a shared factory through 11 instantiation sites would have been pure churn for zero behaviour change.
+The pipeline's PR checkout uses `git clone --single-branch --branch=<headBranch>`, which rewrites `remote.origin.fetch` so only the head branch is fetched. The agent prompt then directs the agent to use `origin/<baseBranch>` for diffs, and the auto-rebase handler tells the agent to `git rebase origin/<baseRef>` — both fail because the base branch's remote-tracking ref was never created. This change adds a small additive supplemental fetch in `checkoutRepo` (`git remote set-branches --add` + `git fetch --depth=<cloneDepth>`) that runs only on PR events when `baseBranch` is non-empty and differs from the cloned head branch. Failures are best-effort (warn-and-continue) so a missing base ref never aborts an unrelated request.
 
-## Files changed
+## Files changed (path · one-line rationale)
 
-- `src/core/fetcher.ts` · rewrote both queries to select `pageInfo`, switched to `graphql.paginate`, added `applyCap()` + nested review-comment follow-up; emits structured warn logs and sets `FetchedData.truncated` flags.
-- `src/types.ts` · extended `FetchedData` with `truncated?: { comments?, reviews?, reviewComments?, changedFiles? }`.
-- `src/config.ts` · added 4 zod fields + env wiring (`MAX_FETCHED_COMMENTS` / `_REVIEWS` / `_REVIEW_COMMENTS` / `_FILES`, default `500`).
-- `src/core/prompt-builder.ts` · added `buildTruncationBanner` and injected it into the prompt when any flag is set.
-- `test/core/fetcher.test.ts` · new tests covering pagination merge (length > 100), TOCTOU after merge, cap fire (log + flag), nested review-comment pagination, and banner injection.
-- `test/factories.ts` · extended `makeOctokit` with `graphqlPaginateResponses` (substring-keyed routing on the paginate fn).
-- `docs/operate/configuration.md` · 4 new rows documenting the env vars.
-- `docs/operate/observability.md` · new "Data fetching safety caps" section documenting the warn log shape and prompt banner.
+- `src/core/checkout.ts` · Accept optional `baseBranch` parameter; on PR events with head ≠ base, run `git remote set-branches --add origin <baseBranch>` + `git fetch --depth=<cloneDepth> origin <baseBranch>` after the initial clone. Wraps in try/catch with Pino warn (no abort). Adds a Pino info log anchored on the existing "Cloning repository" line.
+- `src/core/pipeline.ts` · Forward `enrichedCtx.baseBranch` (already populated from GraphQL at L229) into `checkoutRepo` via the new explicit parameter.
+- `src/core/prompt-builder.ts` · Trim now-redundant defensive prose `(NOT 'main' or 'master')` from the PR diff instructions and the IMPORTANT-CLARIFICATIONS line — `origin/<baseBranch>` now resolves first try so the agent doesn't need the warning.
+- `test/core/checkout.test.ts` · New unit test using `git init --bare` + `GIT_CONFIG_COUNT/KEY/VALUE` `insteadOf` redirect to drive `checkoutRepo` against a local fixture. Four cases: PR head ≠ base (both refs present), PR head == base (no duplicate fetch), issue event (no supplemental fetch), PR with no `baseBranch` supplied.
+- `CLAUDE.md` · Pipeline step 5 now notes the supplemental base-branch fetch on PR events.
+- `docs/build/architecture.md` · "One request, one clone" bullet mentions the supplemental fetch with one clarifying sentence.
 
-## Commits
+## Commits (sha · subject)
 
-- `fix(fetcher): paginate GraphQL connections + MAX_FETCHED_* caps (closes #66)` — single commit, branch `fix/issue-66-paginate-graphql`.
+Single conventional-commit on this branch — see `git log main..HEAD --oneline`. Subject: `fix(checkout): fetch PR base branch so origin/<baseBranch> resolves (closes #74)`.
 
-## Tests run
+## Tests run (command · result)
 
-- `bun run typecheck` · clean.
-- `NODE_OPTIONS='--max-old-space-size=4096' bunx eslint .` · 0 errors / 291 warnings (all pre-existing return-type warnings in unrelated files).
-- `bun run format` · clean.
-- `bun test test/core/fetcher.test.ts` · 30 pass / 0 fail / 64 expect calls.
-- `bun run scripts/check-docs-citations.ts` · clean.
-- `bun run scripts/check-docs-versions.ts` · clean.
-- `bun run docs:build` · skipped locally (`mkdocs` not installed in workspace); runs in CI via `.github/workflows/docs.yml`.
-- `bun run test` (isolated runner): 78 files passed, 25 files skipped because Postgres / Valkey are not running in this workspace (pre-existing infrastructure dependency, none of the skipped files were touched by this PR).
+- `bun run typecheck` · pass (clean exit)
+- `NODE_OPTIONS='--max-old-space-size=4096' bun run lint` · 0 errors / 291 pre-existing warnings
+- `bun test test/core/checkout.test.ts test/core/prompt-builder.test.ts` · **33 pass / 0 fail / 94 expect()**
+- `bun test` (full suite) · **530 pass / 153 skip / 194 fail** vs. **530 pass / 153 skip / 195 fail on `main` with this branch's test file present** — net zero new failures (in fact one fewer, since the new checkout test passes here and would fail on `main`). All 194 remaining failures are pre-existing and infra-bound (Postgres / Valkey / removed `finalizeTrackingComment` & `requireDb` test imports) — unrelated to this change.
+- `bun run scripts/check-docs-citations.ts` · OK
+- `bun run scripts/check-docs-versions.ts` · OK
 
 ## Verification
 
-- **Acceptance criterion: PRs/issues with > 100 comments / reviews / changed files no longer truncate silently.** New test "merges paginated issue comments into FetchedData (length > 100)" proves the merge for 250 comments; "merges paginated review comments across nested pageInfo" proves the nested review-comment merge (100 + 50 = 150).
-- **Acceptance criterion: a hard cap protects the prompt window.** `applyCap` clamps to `config.maxFetchedComments` (etc.), emits a structured warn, and sets `truncated.<connection> = true`. Test "logs warn and sets truncated flag when MAX_FETCHED cap fires" asserts all three (length, log fields, flag).
-- **Acceptance criterion: the agent must know when context is incomplete.** `buildPrompt` injects a `WARNING: pre-fetched context is incomplete…` banner naming the affected connections. Test "buildPrompt includes truncation banner when truncated flag is set" asserts the banner.
-- **TOCTOU semantics preserved.** Test "applies filterByTriggerTime AFTER pagination merge" sets `triggerTimestamp` to comment-240 of 600 and asserts comment-239 (newest pre-trigger) survives while comments 240+ are dropped.
-
-### Intentionally NOT done
-
-- T1 / T2 (paginate-graphql install + threaded factory). Plugin is already bundled with `octokit ^5.0.5` and `octokit.graphql.paginate` is present on every existing instance. Documented above under "Plan deviations".
+1. **T1 satisfied** — `src/core/checkout.ts` adds the supplemental `git remote set-branches --add origin <baseBranch>` + `git fetch --depth=<cloneDepth> origin <baseBranch>` after the initial clone, gated on `ctx.isPR && baseBranch !== undefined && baseBranch !== "" && baseBranch !== branch`. Wrapped in try/catch with Pino `warn` carrying `{baseBranch, headBranch, err}` — a missing base ref degrades gracefully to a warn log instead of aborting checkout. Pino info log added before the supplemental fetch with `{baseBranch, headBranch, depth}`.
+2. **T2 satisfied** — `src/core/pipeline.ts` now passes `enrichedCtx.baseBranch` as the third argument to `checkoutRepo`. `EnrichedBotContext.baseBranch` is required (typed in `src/types.ts:166`), populated from GraphQL at `src/core/pipeline.ts:229`.
+3. **T3 satisfied** — `test/core/checkout.test.ts` builds a real bare upstream with `main` + `feat/x`, redirects the hardcoded `https://github.com/...` URL via `GIT_CONFIG_COUNT/KEY_0/VALUE_0` `insteadOf`, and asserts `git branch -r` against the workDir. Cases:
+   - PR head ≠ base → both `origin/feat/x` and `origin/main` resolvable via `git rev-parse`.
+   - PR head == base → only `origin/main` (no duplicate fetch attempted).
+   - Issue event (`isPR=false`) → only `origin/main` (supplemental fetch gated on `isPR`).
+   - PR with `baseBranch` undefined → only `origin/feat/x` (gate also covers absence).
+4. **T4 satisfied** — `(NOT 'main' or 'master')` removed from `src/core/prompt-builder.ts:56` and L140; the actual `git diff origin/<baseBranch>...HEAD` directives are intact.
+5. **T5 satisfied** — `test/core/prompt-builder.test.ts` had no assertions matching the trimmed substring (verified by grep); 29/29 prompt-builder tests still pass.
+6. **T6 satisfied** — `CLAUDE.md` Pipeline step 5 + `docs/build/architecture.md` "One request, one clone" bullet both note the supplemental base-branch fetch in one sentence each.
+7. **Manual smoke not run** — the live-PR trigger check requires the deployed daemon (out of scope for the implement workflow). Verification 3 in the plan is covered structurally by the new unit test, which exercises the exact code path with a real `git clone --single-branch` + supplemental fetch against a real local upstream.
+8. **No new dependencies, no schema changes, no API surface changes** — checkout signature gained one optional parameter; all existing callers (triage handler, plan handler) continue to work unchanged because they don't pass it.

docs/build/architecture.md

@@ -45,7 +45,7 @@ flowchart TD
 
 - **Async processing.** The webhook handler responds within ten seconds, so the router fires `processRequest` with fire-and-forget semantics after the 200 OK is queued. Every box downstream of `ACK` runs after the HTTP response is on the wire.
 - **Two-layer idempotency.** The fast path is an in-memory `Map` keyed by `X-GitHub-Delivery`. The durable path (`isAlreadyProcessed` in `src/core/tracking-comment.ts`) scans GitHub issue/PR comments for the hidden delivery marker the bot embeds in the tracking comment, so duplicate deliveries are detected across pod restarts, OOM kills, and crash loops — this works **without** `DATABASE_URL`. `DATABASE_URL` is required only to persist execution / dispatch history across restarts.
-- **One request, one clone.** Each delivery clones the repo into a unique temp directory under `CLONE_BASE_DIR` **on the daemon host**. Claude operates on local files via `cwd`. The directory is removed in the pipeline's `finally` block regardless of outcome.
+- **One request, one clone.** Each delivery clones the repo into a unique temp directory under `CLONE_BASE_DIR` **on the daemon host**. Claude operates on local files via `cwd`. On PR events the checkout supplementally fetches `origin/<baseBranch>` (when it differs from the head ref) so the agent's `git diff origin/<baseBranch>...HEAD` and `git rebase origin/<baseBranch>` directives resolve first try. The directory is removed in the pipeline's `finally` block regardless of outcome.
 - **The webhook server never runs the pipeline.** Only daemons execute `runPipeline`. The webhook server is the orchestrator — it enqueues jobs and optionally spawns ephemeral daemons.
 - **Every orchestrator runs a queue worker.** `src/orchestrator/queue-worker.ts` polls `queue:jobs` via `LMOVE` into a per-instance processing list (`queue:processing:{instanceId}`), offers the job to a locally-connected daemon, and atomically re-queues it to the head when no local daemon can take it. Multi-orchestrator HA: `LMOVE` grants exactly-once claim across instances; the offer/accept round-trip stays in-process. Crash recovery is handled by each orchestrator draining its own processing list at startup, plus a cross-instance reaper (`src/orchestrator/valkey-cleanup.ts`) draining processing lists owned by instances whose `orchestrator:{id}:alive` liveness key has expired.
 - **MCP servers.** Tracking-comment updates, inline PR reviews, scoped review-thread resolves, daemon-capability reports, repo-memory, and (optionally) Context7 library docs are exposed as MCP servers the agent can call. Git changes are made via the Bash tool against the cloned repo, not through a dedicated MCP server.

docs/operate/observability.md

@@ -29,6 +29,8 @@ If you add a new secret-bearing config field to `src/config.ts`, add its propert
 | `cost_usd`                           | Agent-reported total cost from the SDK.                                                               |
 | `workflowRunId`, `workflowName`      | UUID of the `workflow_runs` row + workflow name. Stable per run.                                      |
 | `intentWorkflow`, `intentConfidence` | Intent-classifier verdict and confidence for comment triggers.                                        |
+| `branch`, `depth`                    | Initial clone target — emitted by `checkout.ts` on the `Cloning repository` line.                     |
+| `baseBranch`, `headBranch`           | PR base + head ref — emitted on `Fetching PR base branch` (info) and the matching warn on fetch fail. |
 
 ## Ship workflow log fields
 

src/core/checkout.ts

@@ -21,6 +21,7 @@ import type { BotContext, CheckoutResult } from "../types";
 export async function checkoutRepo(
   ctx: BotContext,
   installationToken: string,
+  baseBranch?: string,
 ): Promise<CheckoutResult> {
   // Ensure base directory exists. Path is Zod-validated at startup (not user input).
   // eslint-disable-next-line security/detect-non-literal-fs-filename
@@ -67,6 +68,32 @@ export async function checkoutRepo(
     await $`git -C ${workDir} config user.name ${botName}`;
     await $`git -C ${workDir} config user.email ${botEmail}`;
 
+    // PR events: --single-branch above narrowed remote.origin.fetch to the head ref
+    // only, so origin/<baseBranch> is absent. The agent prompt and the auto-rebase
+    // directive both reference it for diffs / rebases — a missing ref burns recovery
+    // turns or silently falls back to `git diff HEAD`. Widen the refspec and pull
+    // the base ref in. Best-effort: a missing base ref shouldn't break the request.
+    if (ctx.isPR && baseBranch !== undefined && baseBranch !== "" && baseBranch !== branch) {
+      log.info(
+        { baseBranch, headBranch: branch, depth: config.cloneDepth },
+        "Fetching PR base branch",
+      );
+      try {
+        await $`git -C ${workDir} remote set-branches --add origin ${baseBranch}`;
+        // Both sides of the diff are bounded by CLONE_DEPTH. If head/base diverge by
+        // more than that on long-lived branches, the merge base may not be reachable
+        // locally and `git diff origin/<base>...HEAD` silently widens to the closest
+        // shallow-boundary commit — bump CLONE_DEPTH or `git fetch --unshallow` if
+        // an agent reports a noisy diff for a long-history base.
+        await $`git -C ${workDir} fetch --depth=${config.cloneDepth} origin ${baseBranch}`;
+      } catch (err) {
+        log.warn(
+          { baseBranch, headBranch: branch, err },
+          "Failed to fetch PR base branch — agent diff/rebase commands may fail",
+        );
+      }
+    }
+
     log.info("Repository checked out and git configured");
 
     return {

src/core/pipeline.ts

@@ -239,7 +239,11 @@ export async function runPipeline(
       return { success: true, durationMs: 0, costUsd: 0, numTurns: 0, dryRun: true };
     }
 
-    const { workDir, cleanup } = await checkoutRepo(enrichedCtx, installationToken);
+    const { workDir, cleanup } = await checkoutRepo(
+      enrichedCtx,
+      installationToken,
+      enrichedCtx.baseBranch,
+    );
     overrides.onWorkDirReady?.(workDir);
 
     try {

src/core/prompt-builder.ts

@@ -53,7 +53,7 @@ export function buildPrompt(
   const diffInstructions =
     ctx.isPR && data.baseBranch !== undefined
       ? `
-   - For PR reviews: The PR base branch is 'origin/${data.baseBranch}' (NOT 'main' or 'master')
+   - For PR reviews: the PR base branch is 'origin/${data.baseBranch}'
    - To see PR changes: use 'git diff origin/${data.baseBranch}...HEAD' or 'git log origin/${data.baseBranch}..HEAD'`
       : "";
 
@@ -137,7 +137,7 @@ ${ctx.repoMemory.map((m) => `[id:${m.id}] [${m.category}]${m.pinned ? " [pinned]
 Your task is to analyze the context, understand the request, and provide helpful responses and/or implement code changes as needed.
 
 IMPORTANT CLARIFICATIONS:
-- When asked to "review" code, read the code and provide review feedback (do not implement changes unless explicitly asked)${ctx.isPR ? "\n- For PR reviews: Your review will be posted when you update the comment. Focus on providing comprehensive review feedback." : ""}${ctx.isPR && data.baseBranch !== undefined ? `\n- When comparing PR changes, use 'origin/${data.baseBranch}' as the base reference (NOT 'main' or 'master')` : ""}
+- When asked to "review" code, read the code and provide review feedback (do not implement changes unless explicitly asked)${ctx.isPR ? "\n- For PR reviews: Your review will be posted when you update the comment. Focus on providing comprehensive review feedback." : ""}${ctx.isPR && data.baseBranch !== undefined ? `\n- When comparing PR changes, use 'origin/${data.baseBranch}' as the base reference` : ""}
 - Your console outputs and tool results are NOT visible to the user
 - ALL communication happens through your GitHub comment - that's how users see your feedback, answers, and progress. Your normal responses are not seen.