Speed up main page load and show explicit loading/error states#661
Merged
Conversation
The main page fetched the full ~30 MB registry/kgs.jsonld on every visit and parsed it before rendering anything, while the count badges were hardcoded to "0 resources / 0 KGs". For several seconds the registry looked empty, with no indication that data was still loading. A failed load reset the badges to 0 as well, so an error was nearly indistinguishable from both "still loading" and "registry is empty". Load speed: the table only reads ~13 fields per resource, so add util/make_summary_json.py to generate registry/kgs-summary.json with just those fields, and fetch that instead. 28.5 MB -> 0.67 MB raw, 1.75 MB -> 0.16 MB gzipped (what GitHub Pages actually serves). Fetch + parse drops from ~880 ms to ~30 ms on localhost. The full kgs.jsonld is untouched and still offered for download; custom.js falls back to it if the summary is missing, e.g. on a deployment built before this change. Loading state: the badges and table area now ship a "Loading registry data..." spinner in the static markup, so it is visible from first paint rather than only after the fetch resolves. Counts are only shown once data has actually loaded, and "No resources found" now appears only after a real filter returns nothing. Error state: a failed load shows a distinct error alert naming the cause, explaining that this is a load failure rather than an empty registry, and offering a Retry button. Badges go to "-" with bg-danger instead of "0", so failure can never be mistaken for loading or emptiness. A failed taxon_mapping.yaml fetch now shows a dismissible warning that taxon filtering is degraded, instead of only logging to the console. Also fixes two problems found while investigating: - The <noscript> fallback iterated site.ontologies, which does not exist (the key is site.resources), so it rendered an empty table with OBO-style links irrelevant to this registry. It now shows a short notice with data download links. Kept deliberately small: <noscript> content is downloaded by every browser, so rendering 1000+ rows there would add page weight for everyone. - jQuery was loaded twice with different versions (3.6.1 in index.html, then 3.6.0 again in _includes/scripts.html). custom.js is the only jQuery consumer and index.html loads its own, so drop the duplicate. Verified by driving the real custom.js against the real registry data over HTTP in jsdom: 31 checks covering the loading state, successful render (1012 rows), domain/type/taxon/search filters, the empty-result path, the error state, Retry recovery, the kgs.jsonld fallback, and the degraded-taxon warning. Refs #660 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
caufieldjh
added a commit
that referenced
this pull request
Jul 15, 2026
After #661 removed the 30 MB download, the main page still took ~2.3s from navigation to a rendered table. Profiling showed that was not our JS (a CPU profile over the whole load found 1151 ms idle, 799 ms "(program)", and only 98 ms of JS) but two structural problems. 1. The data request was blocked behind the CDN stylesheets. A parser-inserted <script> must wait for all pending stylesheets before it executes. custom.js sits at the end of <body> and runs inside jQuery(document).ready(), so the fetch could not start until the CDN CSS and jQuery had both resolved. Measured: registry-data.js finished downloading at 107 ms, but the fetch did not start until 3917 ms, behind bootstrap.min.css. The fetch needs neither jQuery nor CSS, so it now lives in a dependency-free assets/js/registry-data.js loaded from the TOP of <head>, above the stylesheet links, where nothing blocks it. The request now goes out at ~76-105 ms, in parallel with the CDN assets. custom.js awaits the in-flight promise. Retry still issues a fresh request rather than replaying the settled one. 2. Building 1012 rows of DOM cost ~600 ms, and was repaid on every filter. Isolating the phases: building the row HTML string for 1012 rows takes 16 ms, but inserting it (~1977 KB of HTML, 8000+ cells) takes 597 ms; the first 50 rows cost 47 ms. So the cost is DOM construction, and the fix is to not build nodes until they are needed. (content-visibility: auto was tested first and only gave 1.2x, since it skips layout but not construction.) The table now renders 60 rows and appends more as an IntersectionObserver sentinel scrolls into view. Because only part of the table is in the DOM, sorting can no longer reorder rows the way sorttable.js did -- it now sorts the dataset and re-renders from the top, preserving the aria-sort and chevron behavior. sorttable.js has no remaining references and is removed; nothing else used it (no CSS, no other tables). Also removed as part of this: the 50-row setTimeout chunking (it split a 16 ms string concat across ~21 macrotasks and inserted the DOM only at the end, so it bought nothing), the MutationObserver that re-ran initSortableTables on every mutation, and the dead $tableMain/#table-main no-ops. renderTable no longer shows a spinner on re-render, which fixes the flicker on every keystroke noted in #662. Measured on a local Jekyll build, navigation -> first row (median of 3): localhost 5 Mbps/40ms before #661 2863 ms 49416 ms after #661 2340 ms 3303 ms after this change 556 ms 2150 ms A filter change from 1012 -> 153 rows drops from ~263 ms to ~125 ms. Verified in Chromium against the built site: 17 checks covering the fetch ordering, batch rendering, full-count badges, scrolling to all 1012 rows, whole-dataset sorting in both directions (descending correctly yields "zwd", which DOM-only sorting could not), license sorting through the nested object, aria-sort state, filters, search, the empty-result path, and absence of spinner flicker. Known tradeoff: in-page Ctrl-F only finds rows that have been scrolled into existence. The search box searches the whole dataset and is unaffected. Refs #662 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This was referenced Jul 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #660.
The main page fetched the full ~30 MB
registry/kgs.jsonldon every visit and parsed it before rendering anything, while the count badges were hardcoded to0 resources / 0 KGs. For several seconds the registry looked empty, with no indication that data was still loading. A failed load reset the badges to0too, so an error looked almost identical to both "still loading" and "registry is empty".This addresses all three items in #660.
1. Improve data loading speed
The table only reads ~13 fields per resource (id, name, description, category, domains, collection, taxon, activity_status, evaluation_page, homepage_url, license, publications) — the rest of the 30 MB is contact/product/publication detail the main page never renders.
New
util/make_summary_json.pygeneratesregistry/kgs-summary.jsonwith just those fields, wired into theMakefilenext to the existingkgs.jsonldtarget. The main page fetches that instead:kgs.jsonld(before)kgs-summary.json(after)~11x less to download, and the slow synchronous
JSON.parseof a 30 MB string is gone. The fullkgs.jsonldis untouched and still offered for download / API use;custom.jsfalls back to it automatically (with a console warning) if the summary is missing, e.g. on a deployment built before this change.2. Loading indicator
The badges and table area now ship a
Loading registry data…spinner in the static markup, so it's visible from first paint rather than only after the fetch resolves (the oldLoading resources...only appeared after the fetch already succeeded, so it never covered the actual slow part). Counts appear only once data has loaded, andNo resources foundnow shows only when a real filter returns nothing.3. Make errors obvious vs. still-loading
A failed load now shows a distinct error alert that names the cause, states explicitly that this is a load failure rather than an empty registry, and offers a Retry button. Badges go to
—withbg-dangerinstead of0, so failure can't be confused with loading or emptiness.A failed
taxon_mapping.yamlfetch (previously console-only, invisible) now shows a dismissible warning that taxon filtering is degraded to exact matches; the table itself is unaffected.Also fixed (found while investigating — the "subtle error" from #660)
<noscript>fallback was broken:_includes/resource_table.htmliterated{% raw %}{% for ont in site.ontologies %}{% endraw %}, but the data key issite.resources, so no-JS users got an empty table plus OBO-style links (purl.obolibrary.org, Ontobee) irrelevant to this registry. Replaced with a short notice + data download links. Deliberately kept small:<noscript>content is downloaded by every browser even with JS enabled, so rendering 1000+ rows there would add page weight for everyone and undercut the point of this PR.index.html, then 3.6.0 again in_includes/scripts.html.custom.jsis the only jQuery consumer in the repo andindex.htmlloads its own, so the duplicate is dropped.Verification
I drove the real
custom.jsagainst the real registry data over HTTP in jsdom (real jQuery 3.6.1, real DOM, realsorttable.js). 31/31 checks pass, covering:01012 resources/153 KGs)—badges withbg-danger, Retry button presentkgs.jsonldI also diffed the summary against
kgs.ymlfield-by-field across all 1012 resources: identical values, and zero rendering differences for the publication button (the 111 resources whosepublicationscount differs all have empty lists, which render the same disabled button either way).Not covered: no local Jekyll install, so the full site build isn't exercised here — but
resourcesis confirmed as a top-level key in_config.yml, so the{% raw %}{{ site.resources | size }}{% endraw %}in the new noscript block resolves.🤖 Generated with Claude Code