Repository navigation
feat(profile): cheaper per-repo crawl (REST commits + signals), API-call counting, detail-page polish - #28
Merged
Merged
Conversation
Commit count was gathered with GraphQL `history.totalCount`, which walks the entire commit graph and is the field most likely to trip GitHub's ~10s query timeout at org scale. Replaced it with the standard REST pagination trick: request one commit per page and read the `rel="last"` page number from the Link header — an exact count in a single cheap request, no graph walk. - core/github.ts: add `countByPagination(client, route, params)` — a reusable single-count helper over any paginated REST collection (forces per_page=1, parses the Link header's rel=last page; falls back to the returned item count for 0/1-item collections). - profile/commits.ts: `countRepoCommits(rest, repo)` over the commits endpoint, scoped to the default branch; empty repos / no default branch / errors degrade to 0. - runner: commits are now a third best-effort pass (its own worker pool, wider concurrency since each call is tiny), merged onto the recorded signals. Dropped `history.totalCount` from the GraphQL details fragment, removing the most timeout-prone field from the batched query. Verified with the real Octokit client against a local fake (rel=last page=4096, single-commit fallback, 409-empty → 0). All 11 gates green; coverage 88.38/87.06.
…g, risk ordering Closes 8 of the 11 "not yet evaluated" considerations and adds visibility into the crawl's API cost. New per-repo REST signals (rest-signals.ts), gathered in the per-repo pass alongside commits — each a single cheap request that degrades to 0/false for a read-only token: - webhooks (Link-header count) — migrate disabled; secrets not carried. - Pages (200/404 probe) — settings migrate but need reconfiguring. - code scanning (alerts present) — alert history/states not migrated. Org-level coverage: the 5 secrets/runners/custom-property considerations are already gathered once per run (run.orgResources); mark them evaluated (ORG_EVALUATED_CONSIDERATION_IDS) so the summary stops listing them as "not yet evaluated". Net: the not-yet-evaluated list drops from 11 to 3 (commit-size, file-size, git-ref-length — all need git-sizer / a deep ref scan). API-call counting: getSourceClients installs an Octokit `before` request hook that tallies every REST + GraphQL request (incl. retries and each pagination page); the runner persists the total (profile_runs.api_calls, additive column) at completion. Surfaced in the run header. Deep-scan-first ordering: the counts pass orders repos by disk usage (the only pre-counts size vector); the expensive per-repo passes then order by a composite risk score (issues + PRs + disk + releases), so a run cut short by rate limits or time still covered the heaviest, most limit-prone repos. UI reuse: the profile header now renders the shared AuthPill (source rate-limit quota) from the layout's auth-pill context — the same component the Migrate pages use — since a crawl spends source quota. New per-repo tiles for webhooks / Pages / code scanning. All 11 gates green; coverage 88.63/87.25. Verified with the real Octokit client against a local fake (Pages 200/404, code-scanning presence, webhook Link count) and a seeded run (API-call total, reused pill, not-yet-evaluated 11 → 3).
The REST signals pass probed GET /repos/{owner}/{repo}/pages per repo, which 404s on every repo without Pages — one wasted call + one noisy (but expected) 404 per repo. GitHub already returns has_pages on the org repo listing (minimal-repository), alongside has_wiki/has_issues which discovery reads, so Pages is free: hasPages moves onto the discovery spine and the per-repo /pages call is removed. The REST pass now makes one fewer request per repo (just webhooks + code-scanning).
The remaining 404/403 responses (code-scanning) are expected and correctly handled — they mean the feature isn't enabled (404) or GHAS/scope is absent (403), both degrading to 'no history to lose' (false), never failing the crawl. Verified with the real Octokit client: discovery reads has_pages, and the REST pass issues zero /pages calls. All gates green; coverage 88.63/87.25.
…rators Two table polish fixes: (1) the per-repo metadata line glued '· ' onto each label so the dots had uneven spacing (gap on the left, none on the right). Switched to the same ::before separator pattern the migrate card uses — each item after the first gets a '·' pseudo-element with even margins and a dimmer gray-600, so spacing is symmetric. (2) the repo cell showed the full owner/repo on every row, but every repo shares the org (shown in the header), so 'owner/' was redundant noise — now shows just the repo name, with the full owner/repo on hover (title).
…pty/archived %) The summary header repeated information: 'Repos with results' was identical to 'Repositories profiled' (both the repo count), and the 'Blockers'/'Warnings' totals echoed the Migration-summary rollup's 'Repos with blockers'/'Repos with warnings'. The org-level Insights section, meanwhile, only counted empty/stale/archived repos as pills. Reframed the top row as organization composition — Repositories profiled, plus Stale / Empty / Archived as a percentage of the org (with counts) — reusing the existing per-repo insight ids (empty-repo / stale-confirm / archived-move-now), which are mutually exclusive per repo, so the percentages never double-count. Severity now lives only in the Migration-summary readiness rollup, and per-repo insight chips stay in the table. Removed the now-redundant org-level Insights section. Same tile look and feel; no duplicated numbers.
Adds an Active tile to the composition row — repos that are none of stale/empty/archived (recently pushed, has content, not archived). Since those insight buckets are mutually exclusive per repo, active is the remainder, so the four percentages partition the org and sum to 100%. Widened the row to 5 tiles (grid-cols-2 sm:grid-cols-3 lg:grid-cols-5) and recolored Archived from green to sky so it doesn't clash with Active's green (active = healthy/green, stale = amber, empty = gray, archived = sky).
The source rate-limit pill read 'Source: PAT 5K/5K' with the quota jammed against the auth mode, and the pill sat flush against the '← All runs' link.
AuthPill (shared with the Migrate pages): separate the quota from the label with a subtle left divider (current color at 20%) and padding, set the digits tabular for alignment, and use a middot between label and mode ('Source · PAT │ 5K/5K') to match the app's other metadata lines. Profile header: add a thin vertical divider between the pill and the back link so status and navigation read as distinct.
The org rulesets warning and the 'Organization-level resources to recreate' box were two separate panels for the same idea — org-scoped things that aren't migrated and must be recreated on the target (the registry literally routes rulesets to 'Re-create rulesets on the target'). They were split only by history. Folded rulesets in as the first tile of the resources box, with an amber accent so its extra severity isn't lost — unlike the other resources, a ruleset can fail the migration outright. The full 'can fail the migration' caveat is on the tile's title and in the box footnote. Removed the standalone yellow banner.
Multi-word chips like 'Empty repository' wrapped across two lines, breaking the rounded-pill shape (icon centered against two text rows). Added whitespace-nowrap to the consideration and insight chips and shrink-0 to the insight icon, so each chip stays on a single line and the flex-wrap container moves whole chips to the next line instead. Verified the 'Empty repository' pill is single-line (22px) down to a 900px viewport.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Makes the Profiler's per-repo crawl cheaper and more complete, adds API-cost visibility, and tidies the profile detail page. Eight commits, each green (
bun run gates— all 11 gates; coverage 88.63 func / 87.25 line).Crawl: cheaper, more complete, observable
f508f12— count commits via the RESTLinkheader (rel="last"page number) instead of GraphQLhistory.totalCount, which walks the whole commit graph and is the field most likely to trip GitHub's ~10s query timeout. Adds a reusablecountByPaginationhelper and drops the timeout-prone field from the batched query.b324038— new per-repo REST signals (webhooks, code-scanning) + API-call counting (an Octokitbeforehook tallies every REST + GraphQL request; persisted on the run and shown in the header) + deep-scan-first ordering (counts by disk; expensive passes by an issues + PRs + disk + releases risk score, so a run cut short still covered the heaviest repos) + reuses the sharedAuthPill(source rate-limit quota) on the profile header. Marks the 5 org-level considerations evaluated, dropping "not yet evaluated" from 11 → 3.f853b35— read Pages for free from discovery'shas_pagesand drop the per-repo/pagesprobe (it 404'd on most repos). The remaining code-scanning 404/403s are expected (feature off / GHAS absent) and correctly degrade to "no history to lose".Profile detail page: tidy + dedup
83e7317— repo table: drop the redundantowner/prefix (shown in the header; full name on hover), and fix the metadata-line dot separators to the app's shared::beforepattern.0f262a7— dedup the summary header.Repos with resultsduplicatedRepositories profiled; the topBlockers/Warningsechoed the Migration-summary rollup. Reframed the top row as organization composition — share of the org that's stale / empty / archived — reusing the existing per-repo insight ids (mutually exclusive, so they never double-count). Removed the redundant org-level Insights section.92980f5— add Active % to that row; since the buckets partition the org, the four percentages sum to 100%.a4e1596— refine the rate-limit pill (Source · PAT │ 5K/5K— divider + tabular digits) and header spacing.bcf4093— merge the standalone org-rulesets warning into the "Organization-level resources to recreate" box as an amber-accented tile (rulesets are an org resource that needs recreating; the "can fail the migration" caveat stays on the tile title + footnote).Net effect on per-repo API cost
Before: a commit-graph GraphQL walk +
/pages+/hooks+/code-scanningper repo. After: a single Link-header commit count +/hooks+/code-scanning— fewer calls, none timeout-prone, and the run's total cost is now measured and shown.Verification
bun run gates— all 11 gates green; coverage 88.63 / 87.25.rel="last"counting; discoveryhas_pages+ zero/pagescalls; webhooks/code-scanning degrade paths.AuthPill; composition row sums to 100%; merged org-resources box with the amber rulesets tile; table short-names + even metadata separators; refined pill spacing.Still deferred
The remaining 3 considerations (commit-size, file-size, git-ref-length) need a clone + git-sizer or a full ref-pagination scan. Also noted for a follow-up: code scanning could move to a single org-level call instead of one per repo.