Skip to content

feat(profile): collect content counts, batch augment, size blockers - #22

Merged
cvega merged 3 commits into
mainfrom
feat/profile-counts
Jun 15, 2026
Merged

cvega merged 3 commits into
mainfrom
feat/profile-counts

Conversation

@cvega

@cvega cvega commented Jun 14, 2026

Copy link
Copy Markdown
Owner

What

Addresses the profiler gaps you spotted: it gathered only the per-feature presence signals, one repo per request, and never the actual content volume. This collects real metadata, batches the second pass, and turns that data into an org-wide scale rollup and the first working blocker.

Changes

1. Content-volume counts in discovery (free)

totalCount is a cheap connection aggregate, and discovery already pages the org's repos 100 at a time. So the discovery query now also pulls each repo's issues, pull requests, branches, and tags counts — gathered "from multiple repos at a time" with zero extra requests, exactly the insight you raised.

2. The augment pass is now batched

The one thing that can't fold into a discovery node is the branch-protection rule detail (node-level flags). That pass was one request per repo; it's now an aliased batch — r0: repository(...) {…} r1: repository(...) {…} — profiling 25 repos per request. An org of N repos drops from N requests to ~N/25. The default-branch commit count (history.totalCount, the one count with timeout risk at 100-wide) rides in this smaller-fanout pass too.

It's partial-failure aware: GraphQL returns a null alias + an errors entry when one repo in a chunk is inaccessible (surfaced by the client as a throw carrying partial data). That repo degrades to zeroed signals from the recovered data instead of failing the whole chunk; the run continues.

3. First working blocker

analyze wires the git-archive-size blocker off diskUsage (flagged ≥ 2 GiB as an estimate to confirm with git-sizer). So blockers is no longer always 0 — genuinely large repos now surface.

4. Migration-scale rollup

getProfileDetail derives an org-wide scale (issues, PRs, commits, branches, tags, releases, total size) by summing the persisted signals on read — no schema change, mirroring the insights pattern — surfaced as a "Migration scale" section on the run page.

Performance

Before After
Requests for N repos discovery (N/100) + augment (N) discovery (N/100) + augment (N/25)
Content volume collected none issues, PRs, commits, branches, tags, releases, size

Testing

  • 79 profile tests pass. New/updated coverage: discovery content-count mapping; the batched augment (single request for a chunk, input-order, owner/name-per-alias, null-alias degradation, partial-data recovery on error, rethrow when no recoverable data, empty chunk); runner chunking incl. completed chunks persist when a later chunk fails; the git-archive-size blocker (oversized → blocker, under-threshold → clear, null diskUsage → clear, not a false positive); and the scale rollup summation.
  • Full gate suite green (ok: true); coverage 86.58% / 84.65%.
  • Runtime smoke: seeded a 2-repo run → /api/profile/:id returns the correct scale (issues:15, commits:150, …), the size-blocker persists, and the page renders the Migration scale tiles.

Design notes / deliberately deferred

  • Chunk size 25 balances GitHub's per-request node budget (≤ 25 × 100 rules) and query complexity against the request count; easy to tune.
  • Commits via history.totalCount is the one count with timeout risk at scale, so it's in the smaller-fanout batch, not the 100-wide discovery page.
  • Size detection beyond the disk-usage proxy (release-asset sizes, git-sizer for commit/file/ref limits) remains a separate heavier pass — still honestly reported as indeterminate, per your earlier "defer git-sizer" call.

Note

Two ProfileRun field comments still say "gaps" (the gap → consideration rename leftover I flagged on the docs PR) — left for a separate tidy.

cvega added 3 commits June 14, 2026 16:36
The profiler only gathered the per-feature signals the consideration
checks need, one repo per request, and never the actual content volume.
This adds real repo metadata, batches the second pass, and turns the
content/size data into a scale rollup and the first working blocker.

- Discovery now also pulls each repo's content-volume totalCounts (issues,
  pull requests, branches, tags) — they ride along free in the 100-repo
  page already being fetched.
- The augment pass is batched: one aliased GraphQL request profiles a
  chunk of 25 repos (rule details + the default-branch commit count)
  instead of one request per repo, so an org of N repos costs ~N/25
  requests. It's partial-failure aware — an inaccessible repo in a chunk
  degrades to zeroed signals via the error's partial data instead of
  failing the whole chunk.
- analyze wires the git-archive-size blocker off diskUsage (a proxy,
  flagged for git-sizer confirmation), so blockers are no longer always 0.
- getProfileDetail derives an org-wide migration-scale rollup (issues,
  PRs, commits, branches, tags, releases, total size) by summing the
  persisted signals on read — no schema change — surfaced as a "Migration
  scale" section on the run page.
Each repository row on the run page is now an expandable toggle: clicking
it reveals a detail panel with that repo's individual signal counts —
issues, pull requests, commits, branches, tags, releases, discussions,
projects, environments, stars, watchers, protection rules, and size —
plus a facts line (visibility, default branch, last push, archived/fork/
empty). The per-repo signals were already loaded with the run, so this is
purely client-side; expansion is keyed on nameWithOwner so it survives a
live refresh.
@cvega
cvega merged commit f0e20b5 into main Jun 15, 2026
2 checks passed
@cvega
cvega deleted the feat/profile-counts branch June 18, 2026 08:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant