feat: sphereql-vis crate — offline 3D visualization (+ configurable radial)#34
Draft
bkahan wants to merge 25 commits into
Draft
feat: sphereql-vis crate — offline 3D visualization (+ configurable radial)#34bkahan wants to merge 25 commits into
bkahan wants to merge 25 commits into
Conversation
Introduce `sphereql-vis`, a pure (`sphereql-core`-only) emitter that owns a serializable `Scene` (point cloud + `SceneStats` + a `#[non_exhaustive]` `Overlay` set) and one hardened Three.js renderer. `to_html()` inlines the Three.js + OrbitControls runtime so files open offline (the old Python viewer claimed "self-contained" but loaded from a CDN); `to_html_cdn()` emits a smaller CDN-backed file. Fixes at the source: - projection-aware stats label (no more hardcoded "PCA variance" on a UMAP/Laplacian pipeline) - reduce-based min/max (fixes a latent large-N RangeError crash) - single-place non-finite filtering, deterministic per-category decimation, centralized </script> escaping, empty-scene handling Draw the structure the engine computes: 9 overlay kinds (centroids, classified bridges, slerp geodesic paths, Voronoi caps, antipodes, coverage maps, domain-group spokes, globs, manifold slices) with legend/overlay toggles, OrbitControls, and click-to-inspect. Consumers consolidated onto the shared crate: - sphereql-python `visualize`/`visualize_pipeline`: API byte-identical, now offline + projection-aware; old viz_template.html removed - new CI-safe `visualize_corpus` example (the true e2e viz demo) - `e2e_transformer` migrated off its duplicated inline template - umbrella `sphereql --features vis` (does not leak into --no-default-features) - shared example helpers extracted into a testable sphereql-examples lib Tests: sphereql-vis (11 + doctest), a CI-run overlay-join smoke test, XSS, NaN, offline-URL-absence, CDN-mode, radius-alignment, and a geodesic no-duplicate-vertex regression. CI/docs wired: crate tables, dependency edge list, publish order, feature matrix, CHANGELOG. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reworks sphereql-vis/src/template.html into a celestial-observatory console (fully offline; no web fonts/assets): - Atmosphere: nebula gradient + film grain + vignette; a (θ,φ)-aligned lat/long graticule globe with a backdrop shell; crisp solid-disc point shader with a bright core. - Instrument chrome: hairline panels with HUD corner brackets, monospace typography, a wordmark + projection-kind pill badge, a hover reticle, and a staggered load reveal (prefers-reduced-motion aware). - Tabbed control deck — Domains / Overlays / Settings — with select all/none. - Settings: Scale (default 5×, up to 50×; the model spreads apart on screen so overlapping blobs separate), Radial spread (thickens the r shell), Domain spread (angular de-clumping), point size (down to 0.5), a Reference globe toggle, auto-rotate, and Reset to defaults. - 2D sky-chart minimap (equirectangular θ→φ) with a live camera reticle and click-to-aim navigation. - Reliable click-to-select via pointerdown/up + movement threshold (the `click` event is unreliable under OrbitControls); auto-rotate defaults off. All data/placeholder/escaping contracts preserved; 11 tests + doctest green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- UI scale slider (0.70×–1.60×) — zooms the HUD panels independently of the 3D scene (CSS zoom). - Color schemes — Aurora / Spectral / Viridis / Sunset / Ice, switchable live (recolors points, legend, minimap, selected point). - Nearest-chord neighbor rows tinted with the neighbor's domain color. - Pick threshold now scales with the model (maxR · 0.07 · scale) so points stay clickable at any Scale (incl. 50×). - Clickable "© bkahan/sphereQL" credit linking to the public GitHub repo. - All covered by Reset to defaults. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…s nav - Default Scale 12× (max 100×); camera frames the 12× model on load. - Zoom-speed control (slider; default 0.5) — less sensitive scroll zoom. - Save/Load view settings as a TOML file (scale, zoom, radial, domain spread, point size, UI scale, color scheme, globe, auto-rotate). - Anchored, domain-colored label pinned on the selected sphere; the hover reticle now snaps onto the actual point so the click target is unambiguous. - Click a point to auto-center the orbit on it; Esc / empty-click returns to the main sphere (smooth target tween). - Spread-aware bridges: endpoints follow the spread/radial transform. Adversarial review fixes: robust behind-camera reject for screen labels (getWorldDirection in-front test), clear hover on select, re-apply selection highlight in place on palette change (no panel flicker), and surfaced config load errors (console + button feedback) instead of swallowing them. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…mbed Bridges: - Tie each bridge to its exact source-point index (full-precision coord match) so its endpoint follows that sphere precisely under spread/scale, instead of a nearest-centroid approximation. - Highlighting a sphere now draws its bridges from the sphere to the connected domains (classification-colored) regardless of the global toggle, with a `bridges: N` count in the info panel. Navigation / input: - Zoom-to-cursor: intercept the wheel and dolly while keeping the world point under the pointer fixed (OrbitControls r128 has no zoomToCursor); touch pinch still uses the built-in zoom. Touch / mobile / embed: - canvas touch-action:none; panels are position:fixed and the page is locked to the viewport (overflow:hidden, overscroll-behavior:none). - Responsive layout ≤720px: header across the top, Domains/Info share the upper row, stats above a full-width filter bar pinned to the bottom; minimap + hints drop out; a ⛶ hide-UI toggle collapses all panels. - visualize_corpus prints a ready-to-paste <iframe> embed snippet. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…xed)
The radial coordinate r was a faithful pass-through of the raw embedding
magnitude, which on sparse/uniform corpora clusters in a narrow near-maximum
band (e.g. HandCrafted/UMAP r ∈ [0.95, 2.13]). Add a way to remap it.
- `PipelineConfig.radial` (`RadialConfig` { mode, lo, hi, percentile }) with
`RadialMode::{Magnitude (default), Fixed, Stretch}`.
- `RadialStrategy::MagnitudeStretch { m_lo, m_hi, r_lo, r_hi }` — affine map of
the embedding magnitude from a corpus source band onto [r_lo, r_hi], clamped.
- The fit path resolves the mode once (two-pass percentile scan of corpus
magnitudes) and bakes the bounds into the strategy, so corpus points AND
later queries get consistent r. UmapGraph now carries per-point magnitudes
so the graph-reuse (tuner) fit path can resolve bounds too.
- PCA's volumetric mode (which overrides any RadialStrategy) is auto-disabled
when a non-default radial mode is chosen, so Stretch/Fixed take effect.
- `visualize_corpus --radial lo:hi`; the example prints the resulting r range.
Default is unchanged (Magnitude pass-through, identical to before). Verified:
`--radial 0.2:1.9` on HandCrafted/UMAP gives r ∈ [0.200, 1.900] with the same
winning projection + score. Tests for the stretch math, the percentile bounds,
mode resolution, and a pipeline integration check.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- DOM labels anchored above centroid / antipode / domain-group markers, with a connector tick. Projected each frame, scaled by camera distance (0.7–1.7×), hidden behind the globe (front-hemisphere occlusion) and when the matching overlay layer is off. - Clickable: click any label to tween-focus the camera on it; clicking a domain (centroid) label solos that domain (others dimmed; click to restore). - Settings → "Labels": one checkbox per label kind present in the scene (Centroids / Antipodes / Domain groups), each toggling independently. Covered by Reset. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
refactor(vis): extract viewer.js + add ScenePoint.id and Scene::to_json Cross-cutting foundation for the visualization roadmap (decouple viewer from generator so it can rebuild from any Scene at runtime). - ScenePoint gains optional `id` (stable identity for cross-scene k-NN highlight and morph; viewer will key on id, not array position) plus a `with_id` builder. Serialized only when present. - Scene::to_json / from_json: canonical JSON wire form (doctested) for the drag-drop loader and the future WASM bridge. - Viewer runtime moved out of template.html into src/viewer.js, inlined at a new /*__SPHEREQL_VIEWER__*/ placeholder, so the baked HTML and the upcoming WASM studio share one implementation and cannot drift. Still a single self-contained offline page. - Re-anchor tests/emit.rs::embedded_payload: the data line is no longer adjacent to `const pts=D.points` (now in viewer.js); read the single `const D=` line directly. All 11 emit tests + 2 doctests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> @
Split the monolithic viewer init into persistent setup + a rebuild(sc) / teardown() core so the viewer can swap to any Scene at runtime (a dropped file, or a live WASM pipeline) without reloading the page — the foundation for the generalize/studio/compare roadmap. - Persistent (created once): renderer/scene/camera/controls/lights/ raycaster and every static DOM event listener. Per-scene state moved to module-level lets that rebuild() reassigns and listeners read live (never a stale snapshot). - teardown() disposes the prior scene's GPU geometries/materials (points, globe, lines, every overlay group), removes them from the THREE scene, and clears generated DOM; only ever called synchronously at the top of rebuild() so animate() never sees half-torn state. - rebuild(sc) reads the Scene shape (points/overlays/stats/title/ surface_radius/show_axes) and resets all view settings to defaults, so a swapped scene starts clean. Added null guards (getHovered/search/ drawMinimapBase) for the transient null window. - Baked page just calls rebuild(D) once at boot. - Re-anchor tests/emit.rs::embedded_payload on the payload script's opening (the const pts=D.points line it keyed on moved into rebuild()). Verified: node --check passes; end-to-end emission unchanged (1097 KB, self-contained, 0 external scripts); adversarial diff-review vs the pre-refactor extraction found no behavioral regressions, leaks, TDZ, or stale-state carryover. All 11 emit tests + 2 doctests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The viewer is now a general 3D sphere/embedding explorer: drop a Scene JSON anywhere on the page (or use Settings ▸ Open Scene JSON…) and it teardown()s the current scene and rebuild()s around the new one — no reload, no regenerate. - parseScene(obj): normalizes arbitrary input into the Scene shape. Accepts the full Scene::to_json shape OR a bare points array, and tolerates minimal points — each needs only finite x/y/z OR finite r/theta/phi; the missing pair is derived with the sphereql-core convention (x=r·sinφ·cosθ …; θ=atan2(y,x)∈[0,2π), φ=acos(z/r)). surface_radius falls back to the median ‖xyz‖, stats/overlays/title get sensible defaults. Non-placeable points are dropped; empty or shapeless input throws a human-readable message. - Drag-drop overlay (#dropzone) bound on window with a file-drag guard and an enter/leave depth counter; hidden #scene-file input + an Open Scene button reuse the soft-fail (✓/✗) feedback pattern. - Documented the public Scene JSON runtime contract in the crate docs. Verified with a headless THREE+DOM harness booting the real viewer.js: 20 checks incl. full-scene/bare-array/spherical-only/mixed/error inputs, a clean second-scene rebuild() (the drag-drop swap path, recomputing categories), and a real 775-point emitted scene round-tripping through parseScene. Rust: all sphereql-vis tests + doctests green; doc-snippets gate green; node --check clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three pure-client explorer tools, added as header buttons: - Ruler (⟀): click two points; reports the great-circle angle (deg + rad) and chord, and draws the connecting geodesic on the shell via slerp (same arc construction as overlay.rs geodesic_path). Esc clears. - PNG (⤓): preserveDrawingBuffer renderer → canvas.toDataURL → download. - Share (🔗): copies a link whose #v= hash is base64(JSON) of camera pose + view settings + tool toggles (NEVER scene data); applyViewHash() restores it once after the boot rebuild, reading only validated numbers/known keys. Folds in fixes from an adversarial review workflow (8 confirmed of 21): - MAJOR: ruler slerp blew up for (near-)antipodal picks (1/sin(om)→∞, arc collapsed through the globe). Now special-cased: coincident → short segment; antipodal → clean great semicircle about a ⟂ axis; else slerp. Arc verified on-shell for all cases. - MAJOR: stored XSS via stats.sampled_from / dropped_nonfinite on the drag-drop path — now coerced to numbers in parseScene AND escHtml-ed at the render chokepoint. - MAJOR: rulerOn wasn't reset on scene swap — teardown now fully disarms the tool (flag + button + picks). - MINOR: dropped the deprecated escape/unescape from the hash codec (encodeURIComponent/decodeURIComponent round-trip verified). - MINOR: malformed overlays in a dropped scene no longer abort rebuild (per-overlay try/catch + kind filter in parseScene). - Documented the median-shell parity with Scene::surface_radius_for and the preserveDrawingBuffer tradeoff (kept for cross-browser PNG). Verified via the headless THREE+DOM harness (24 checks: ruler 0/90/180° + on-shell antipodal arc, PNG download, hash round-trip, ruler-reset, stats coercion, overlay filtering) plus all sphereql-vis Rust tests + doctests; page stays fully offline (1.1 MB, 0 external scripts). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Decouples the viewer's data source: a fitted pipeline (and free text) can now produce a sphereql-vis Scene entirely in the browser, feeding the same viewer as the baked HTML and drag-drop. - New optional deps behind features `scene` (sphereql-vis) and `lingua` (sphereql-lingua, implies `scene`), both added to `default` so `cargo --all-features` and the wasm jobs build them. - Pipeline.buildSceneJson(title) → JSON: maps exported_points → ScenePoint (stable id + certainty/intensity), category centroids, classified bridges, Voronoi territory caps, and domain-group spokes, with projection-kind/evr stats. Colors use the viewer's aurora palette + sorted-category assignment so overlays match the point colors. - LinguaStudio.process(text) → JSON: runs LinguaPipeline in-browser; each placed concept becomes a point (domain → category, surface form → label, normalized → id), each typed relation a geodesic path, with mean salience as the headline stat. - Both return JSON strings (not tsify): a Scene is deep and the viewer parses JSON anyway, keeping the generated .d.ts small. Sizes (measured): raw .wasm 1.5M → 2.3M after wasm-bindgen (no wasm-opt; wasm-pack's -Oz trims ~30-40%, gzip far more) — separate-file packaging is the right default; single-file base64 stays opt-in (Phase 5). Verified: native --all-features build + clippy -D warnings clean; new native tests (build_scene_json shape + ids/quality/overlays/stats; LinguaStudio smoke); wasm32 release build + wasm-bindgen confirm buildSceneJson / LinguaStudio.process are exported. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A live, in-browser projection studio under sphereql-wasm/studio/: paste prose and sphereql-lingua places each concept on the sphere as you type, or switch to Corpus JSON mode to run the full embedding pipeline and pick a projection — all client-side via WASM, re-projecting live. - Shared viewer, zero drift: the studio shell is emitted from Scene::to_html() (three.js + the SAME viewer.js inlined, empty initial scene) by examples/build_studio.rs, with the studio chrome injected after <body> and studio.js before </body>. studio.js drives the viewer's global rebuild()/parseScene(). - Off-thread compute: a classic Web Worker (worker.js) importScripts the --target no-modules wasm glue and runs LinguaStudio.process / Pipeline.buildSceneJson, posting Scene JSON back. Requests are debounced (~320ms) and id-tagged so the freshest paste wins; pre-init requests are queued (newest only) and replayed on ready. - build.sh assembles dist/ (self-contained index.html + worker/driver + pkg/*.wasm); prefers wasm-pack, falls back to wasm-bindgen + wasm-opt. Separate-file packaging is the default; single-file (base64 wasm) is a documented future opt-in. dist/ is gitignored. Headless-verified: studio shell emits with all chrome IDs + inlined viewer globals + empty scene + offline three; studio.js/worker.js pass node --check; a worker-protocol harness (stubbed wasm) covers queue-before-ready, ready, lingua/corpus dispatch, newWithConfig, and the unknown-kind error path; build.sh assembles dist end-to-end and the no-modules glue exposes wasm_bindgen/LinguaStudio/Pipeline/buildSceneJson. Remaining: a manual browser smoke-test (paste → worker → render). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Corpus mode gains a nearest-neighbor query: enter a query vector, the
studio runs pipeline.nearest off-thread and the viewer highlights the
hits by stable id.
viewer.js:
- rebuild() builds an idToIndex map (stable id → point index).
- highlightByIds(ids) emphasizes the matches (closest largest), dims the
rest, and fans geodesics from the nearest match to the others on the
shell. Accepts raw ids or {id,…} objects (NearestOut). Returns the
resolved count; [] clears. Persistent queryGroup wired into scalables
and cleared on teardown / re-query.
- Factored the robust great-circle builder out of the ruler into a shared
shellArc(a,b) (coincident → segment, antipodal → ⟂-axis semicircle,
else slerp), now reused by the query fans.
studio (worker.js / studio.js / chrome.html):
- The worker keeps the last corpus Pipeline alive and adds a `query`
handler → pipeline.nearest(query, k) → posts neighbors; a query before
any corpus fails gracefully.
- A query row (vector input + k + Find, Enter to submit) shows in corpus
mode; onmessage now branches scene-rebuild vs neighbor-highlight.
Headless-verified: a viewer harness covers id resolution (raw + object
ids), the geodesic-fan count, unknown-id skipping, clear, and
idToIndex/queryGroup reset on rebuild; the worker harness covers
corpus-keeps-pipeline → query → neighbors (honoring k) and the
query-before-corpus error. sphereql-vis Rust tests + doctests green; the
studio rebuilds with the query chrome. No Rust source changed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two ways to compare projections of the same corpus, both id-aligned. viewer.js: - Morph: setMorphTarget(sceneB) keys B's points by stable id; applyMorph(t) slerps each A-point's direction toward its id-matched B-direction and lerps the radius by t∈[0,1] (t=0 = A, t=1 = B). Unmatched points stay; t=0 falls back to the normal spread/radial view. Cleared on teardown. - Compare embedding (opt-in via #embed): the viewer accepts an injected scene + a synced camera over postMessage and broadcasts its own camera moves to the parent. The broadcast is epsilon-gated (not a bare flag), so the OrbitControls damping that follows an applied update can't start a feedback storm. Fully inert without #embed (baked viewer unaffected). studio: - A morph control (corpus mode): pick a second projection → the worker builds it as a morph target → a slider drives applyMorph. - compare.html / compare.js: two embed.html#embed iframes fed two projections by one worker; the parent relays each pane's camera to the other for synced orbiting. build_studio.rs now also emits embed.html (a plain, chrome-less viewer for the panes); build.sh copies the compare assets. Headless-verified: morph endpoints exact (t=0→A, t=1→B, t=0.5 on the geodesic with lerped radius), unmatched points untouched, clear/rebuild reset; embed sync covers scene injection, camera broadcast, the epsilon gate, the echo guard (no feedback storm), and inertness without #embed. All 6 JS harness suites + sphereql-vis Rust tests green; clippy clean. Remaining: a manual browser smoke-test of the live studio panes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two isolated viewer additions, both persisted in the TOML config. - Density shading: rebuild() bins each point's (original) direction into a θ×φ grid (the minimap's scheme) and stores its bin's normalized count as a per-point `density` attribute. A "Density shading" toggle flips a shader uniform that recolors the cloud as a cold→hot heatmap (and fades sparse points), so dense regions of the sphere read at a glance. - Pins: a 📍 pin-mode tool — click the globe shell to drop an annotated (θ,φ) marker with a floating, depth-culled label (clickable to remove). Pins survive scale/zoom (angular coords, re-placed on the current shell). Esc exits pin mode; rebuild clears pins; reset clears them. - Persistence: both ride the existing flat TOML — `density` as a bool and `pins` as a single base64-JSON value (not array-of-tables), encoded with the same encodeURIComponent codec as the share hash. Loaded pins are validated (finite θ/φ) and rendered via textContent (no innerHTML), so a hostile config can't inject markup. Headless-verified: density attribute carries correct normalized bin counts (clustered=1.0, lone=0.25), the toggle flips the uniform; pins add markers, keep explicit/auto(ASCII) labels, round-trip through the TOML base64 key, and reset on rebuild. All 7 JS harness suites + sphereql-vis Rust tests green; the studio rebuilds with the new controls. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The studio opened on an empty sphere (then auto-ran a tiny lingua example); it now opens on the rich 775-point HandCrafted corpus — the same auto-tuned, fully-overlaid scene visualize_corpus renders — because it simply looks better as a landing view. - New sphereql_examples::demo_scene(CorpusId) factors the load → embed → auto-tune (deterministic seed) → build_corpus_scene flow so it can be reused without duplicating the pipeline setup. - build_studio bakes demo_scene(HandCrafted).to_html() as the studio's (and the compare panes') opening scene; sphereql-examples + sphereql- corpus are dev-deps (native example only — not compiled into the wasm cdylib, so the shipped bundle is unchanged). - studio.js no longer auto-runs the lingua example on boot, so the baked scene stays until the user acts (type / example / corpus / morph). Verified: build_studio emits index.html + embed.html with 775 points / 1020 overlays (umap_sphere); sphereql-wasm bridge tests pass; clippy + fmt clean on the changed crates. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…default
Two studio fixes:
- The studio now OPENS in Corpus JSON mode (lingua is the alternate),
matching the baked 775-point demo scene that's already on screen.
- Switching to lingua and back to corpus no longer runs the tiny sparse
6-point example (the bug that replaced the 775). Corpus mode now
restores the baked demo scene via rebuild(D) — instant, no recompute —
so the 775 is always the corpus default.
Mechanics: per-mode input buffers (corpus starts empty for your own
{categories,embeddings}; lingua seeds the prose example); switching modes
saves/restores each buffer; entering corpus mode calls restoreDemoScene()
(rebuild of the baked global D); the debounced live re-projection now only
runs in lingua mode (corpus JSON runs explicitly via Run/example). The
demo corpus itself (2.1 MB of dense 128-dim vectors) is intentionally NOT
dumped into the textarea — the box is for pasting your own corpus, and the
'example' button drops a small editable template.
node --check clean; build.sh assembles dist with the 775 baked + Corpus
mode active.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…se one
Addresses three studio issues:
1. Remove the sparse corpus entirely. The 6-point CORPUS_EXAMPLE is gone;
the "example" button is now lingua-only (hidden in corpus mode), so
there's no path back to the sparse scene.
2. Return from the side-by-side: compare.html gains a "← studio" link.
3. Compare did nothing because it had no corpus to project. The demo
corpus is now shipped as a fetched dist/demo-corpus.json (emitted by
build_studio via demo_corpus_json) — NOT dumped in any textarea. The
worker can now re-project / compare / morph it:
- compare.js: empty box → compare the demo corpus (projA vs projB),
so it works out of the box; robust injection waits for each pane's
new `sphereql-embed-ready` signal (no lost scene if a pane is still
loading); also relays cameras as before.
- studio.js: run / morph fall back to the demo corpus when the box is
empty, so they act on the 775-point demo.
- viewer.js: the #embed block posts `sphereql-embed-ready` to the
compare host once its message listener is live.
The textarea stays empty for your own paste (per the earlier choice);
the 2.1 MB corpus lives only in the fetched file + the worker.
Verified: all 7 JS harness suites pass (incl. embed sync with the new
ready ping); build.sh assembles dist with demo-corpus.json (2.1 MB),
both drivers fetching it, the back link present, and no CORPUS_EXAMPLE;
clippy + fmt clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Query needs a worker pipeline, so it previously failed on the default
demo until you ran a corpus. The studio now primes the demo pipeline in
the background on boot — without touching the textarea or the displayed
scene:
- worker.js: new `load` kind builds the pipeline (kept for queries) but
returns no scene, so the baked demo scene on screen is untouched.
- studio.js: primeDemo() sends `load` with the fetched demo corpus once
wasm + the corpus are ready (config:null → fast PCA build). Because
nearest() works in embedding space, the projection used for priming
doesn't matter — query highlights the baked scene correctly by id.
Guarded by demoPrimed + corpusRan so it never overwrites a corpus the
user has run/morphed themselves.
Net: open the studio → query immediately finds neighbors in the 775-point
demo; paste your own corpus + Run and query switches to it. The textarea
stays empty (no 2.1 MB dump).
Verified: worker harness covers the new load kind (load → {ok,loaded},
then query works); all query/morph/embed harness suites still pass;
build.sh assembles dist with the priming wired in.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bridge classification was gated on the outer projection's EVR, but for UmapSphere `explained_variance_ratio()` returns a kNN-recall score (neighbourhood preservation) orthogonal to relation validity, and ~0.09 on real corpora -- so it suppressed EVERY bridge to Weak (0 Genuine wormholes). Remove the gate; classify from projection-independent full-dim signals, implementing the documented balanced_affinity_quantile intent: Genuine when min(affinity_to_source, affinity_to_target) clears the home-affinity quantile AND the two category centroids are distinct (new overlap_separation_quantile floor). territorial_factor stays a soft discount; min_evr_for_classification deprecated (kept for serde). Replaces the misleading low-EVR test with one asserting classification is independent of the deprecated gate. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…pane locks
Four UI fixes:
1. Compare bar no longer shifts on "compare": #status had a variable
width (margin-left:auto) so its text changes reflowed the flex bar,
pushing the controls right. It's now a fixed-width, right-aligned,
ellipsised cell. Also reflowed the compare page to a flex column (bar
auto-height + panes fill the rest) so a wrapping bar can't overlap the
panes.
2. SphereQL header fits the screen: #hdr gains max-width:calc(100vw-32px)
+ box-sizing, and the subtitle flexes/shrinks (min-width:0), so the
centered header never overflows the viewport.
3. Resizable minimap: #mini gets CSS `resize:both` (min/max bounded); a
ResizeObserver keeps the drawing buffer matched to the element size
(MW/MH are now mutable) so the sky chart stays crisp at any size.
4. Independent orbit / zoom locks in side-by-side compare: two toggles
("⟲ orbit", "⊙ zoom") broadcast a `sphereql-lock` message to both
panes; the viewer's #embed block maps them to controls.enableRotate /
enableZoom + a wheel-zoom `zoomLocked` flag. Locks are re-sent to a
pane when it (re)loads.
Verified: all 7 JS harness suites pass (incl. a new lock-message test:
lockRotate→enableRotate=false independently of lockZoom→pinch off +
wheel zoomLocked); sphereql-vis Rust tests green; build.sh assembles the
dist with all four wired in.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…dio failure modes)
From the adversarial production-readiness audit:
- BLOCKER (correctness): parseScene computed the median surface_radius
over ALL norms, but Rust Scene::surface_radius_for filters finite &&
>0. A dropped scene containing origin points landed on a different
shell than baked. JS now applies the same filter — exact parity.
- BLOCKER (robustness): a missing/failed demo-corpus.json fetch was
swallowed silently while the status still claimed "demo corpus",
leaving query broken with a misleading message. Now tracked
(demoFailed) in studio.js + compare.js: honest status, and run/find/
morph give a clear "demo unavailable — paste/run your own" instead of
a confusing failure.
- MAJOR (robustness): a wasm load failure ({type:'fatal'}) left the UI
interactive but inert (pendingRun stuck, no recovery signal). Now a
wasmFailed flag short-circuits run/find/morph/build with "wasm
unavailable — reload the page" (studio.js + compare.js).
- MINOR (security): the #embed viewer accepted scene/cam/lock postMessage
from any window. It now requires e.source === parent (the compare
host); scene data was already parseScene-sanitized, so this closes
camera/lock spoofing.
- MINOR (correctness): morph's antipodal branch used lerp+normalize,
which collapses to the origin at the midpoint. Replaced with the
perpendicular-axis great-circle sweep (same as shellArc) so the path
stays on the sphere. Also corrected the "upper-median" comment.
- MINOR (robustness): compare shows "✓ compared" only once BOTH panes
receive a scene (was set on the first); build.sh errors clearly if
neither wasm-pack nor wasm-bindgen is installed.
Verified: all 7 JS harness suites pass (added a surface_radius-parity
test + a non-parent-source rejection test); sphereql-vis Rust tests +
doctests green; node --check + bash -n clean. The earlier deterministic
sweep (fmt/clippy -D/4 CI gates/wasm32 build) was already green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The viewer runtime (sphereql-vis/src/viewer.js) and the studio worker are browser JS with no Rust around their logic, so the Rust suite never covered them — the harness that caught real bugs during development lived only in a scratch dir. This commits it as js-tests/ and runs it in CI. - js-tests/harness.cjs boots the real viewer.js in a Node vm against thin THREE + DOM stubs and exposes its internals for assertions. - 7 suites (01-parse-scene … 07-worker) cover parseScene + surface_radius Rust-parity, the ruler/PNG/share tools, semantic query, morph, the compare embed-sync (incl. the echo guard + non-parent rejection), density+pins, and the worker message protocol. - run-all.cjs runs each suite in its own process and exits non-zero on any failure; a new `js-tests` CI job runs `node js-tests/run-all.cjs`. - Paths are repo-relative; the one integration check that needs a real emitted page is env-gated (SPHEREQL_EMIT_HTML) and skipped otherwise, so CI needs no Rust artifacts. Verified: `node js-tests/run-all.cjs` → all 7 suites pass; check-docs OK (js-tests is not a crate, no table/floor impact). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ates The rust-api audit flagged Scene::to_json's `.expect` as a potential panic on NaN/Inf. It isn't: serde_json renders non-finite floats as JSON `null` rather than erroring, so a built — or hand-constructed — Scene with non-finite coordinates serializes without panicking (the WASM bridge's buildSceneJson relies on this to never trap the worker). This adds a regression test that constructs a Scene with NaN/±Inf via the public fields, asserts to_json doesn't panic and yields valid JSON with nulls, and documents that Rust from_json won't read those nulls back (serde null↛f64) — fine, since the runtime consumer is the viewer's JS parseScene, which coerces null→0. No code change — the audit finding was a false positive; this verifies and pins the behavior. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
sphereql-vis, a pure (sphereql-core-only) crate that turns pipeline output into a self-contained, offline, interactive 3D sphere viewer — making visualization first-class in Rust (it was Python-only and structurally weak before). Consumed by the Python bindings (API unchanged), a newvisualize_corpusexample, the migratede2e_transformer, and the umbrellasphereqlcrate behind avisfeature.This is the foundation PR; a follow-on roadmap (drag-drop arbitrary scenes, a WASM lingua-live studio, compare/morph, query box, ruler, shareable URLs, density/pins) builds on it.
What's here
Scene(point cloud +SceneStats+ a#[non_exhaustive]Overlayset — centroids, classified bridges, slerp geodesic paths, Voronoi caps, antipodes, coverage, domain groups, globs, manifold slices) + a hardened HTML emitter. Three.js + OrbitControls inlined → opens offline;to_html_cdn()for a smaller file.PipelineConfig.radial:Magnitude/Fixed/Stretch) — fixesrclustering in a narrow near-max band on sparse corpora.<script>-breakout escaping, large-N min/max crash fix, NaN filtering, decimation, projection-aware stats labels.Tests
sphereql-visemit/XSS/NaN/offline/radius-alignment suite + doctest; a CI-run overlay-join smoke test; embed radial-stretch tests; Pythontest_viz(incl. projection-aware label + offline). All four gate scripts, clippy-D warnings, and fmt green.Draft — opening as a checkpoint ahead of the roadmap work.
🤖 Generated with Claude Code