Add a VAME view - #418
Draft
h-mayorquin wants to merge 16 commits into
Draft
Conversation
|
Preview deployment ready! https://create-vame-view.neurosift.pages.dev Built from e4523bf |
# Conflicts: # src/pages/NwbPage/externalVideoUtils.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds a viewer for VAME motif segmentation. When it finds a VAME object, the view reads the per-frame ndx-vame
MotifSeriesand run-length-encodes the motif sequence in the browser (no change to the file) into the in-memory equivalent of anEthogramBoutstable: each run of a single motif becomes one(start_time, stop_time, label)row, i.e. the same dynamic-table /TimeIntervalsshape a stored bouts table has. It then renders that as a montage over a session ethogram strip: pick a motif in the sidebar (or click its lane) and the montage plays several example clips of it side by side, looping in sync, cropped from the linked video with the pose keypoints and skeleton overlaid; when a file has pose but no video the clips play pose-only on a black canvas.Rather than building this only for VAME, the montage-and-strip renderer is tool-agnostic: it consumes labeled behavioral bouts (the published
ndx-ethogramEthogramBoutstype, one row per bout with a single textlabel), and VAME is a thin adapter that produces those bouts from aMotifSeries. Because the renderer makes no VAME-specific assumptions, the same view serves any segmentation or classification tool that emits bouts, which is what lets it work directly on the public DANDI 001878 ("Standardized behavioral segmentation datasets") with no extra code. That dandiset stores each tool's output asEthogramBouts, including a VAME table, so the VAME motif output and several other tools all render through the one view: