Annotations fail to load in DIVE when importing video (.mp4) via GCS Import

[BrendaTran]

Describe the bug
When importing a folder containing a video file (e.g., .mp4) and its corresponding annotations.csv via Girder GCS Import, the DIVE Launch Viewer fails to display the annotation bounding boxes. The video payload loads and plays successfully, but the annotations are completely missing. (Note: This issue appears specific to video imports).

To Reproduce
Steps to reproduce the behavior:

1. Stage a folder in GCS containing a .mp4 video and an annotations.csv file.
2. Execute a Girder GCS Import for the staged folder.
3. Launch the imported dataset via the DIVE viewer.
4. See error: Video loads successfully, but no annotation bounding boxes are rendered.

Related data
The issue is not data-specific and can be easily reproduced with any standard .mp4 and accompanying annotations.csv imported via GCS. I can provide a dummy sample dataset upon request if needed.

Expected behavior
The GCS import process should correctly link the accompanying annotations.csv file to the video payload so that DIVE parses and renders the bounding boxes automatically upon launch.

Desktop (please complete the following information):

• OS: Windows 11
• Browser Chrome

Additional context
Deployment: DIVE Web, deployed on GCP (Google Cloud Platform)
Import Source: GCS (Google Cloud Storage)
Component: Girder Assetstore Import
Observation: I noticed that the annotations.csv doesn't end up in an auxiliary folder after import. I'm not sure if DIVE requires that specific structure, but I wanted to note it in case the file is just being routed to the wrong place during the video import process.

cc: @csbrown-noaa
cc: @kelseylmartin

[kelseylmartin]

@BryonLewis please make this issue as well as issue 1671 (which I see you're already working on) a priority. For context, Brenda and the rest of the team working on this only have a few weeks left on their contract.

[BryonLewis]

A dummy CSV would help for debugging purposes. Also, I believe the matching for videos relies on the video and the csv having the same name. I.E underwater_example_1.mp4 and underwater_example_1.csv should match and import. I'll create a video demo showing how this works with some of my test data.

[BryonLewis]

This video explains how the import works and shows some of my local testing process.
I use MinIO, which can mimic the protocols and behavior of S3/GCP for importing.

20260602-DIVE-AssetStore-Import_small.mp4

This video shows that VIAME-CSV files can be imported as long as they have the same filename as the video file:

20260602-DIVE-AssetStore-Import-CSV_small.mp4

[BrendaTran]

Hi @BryonLewis, thanks for the demo videos! I have validated the behavior you showed and will make sure to name the output csv or json files to match the video files on my end.

[csbrown-noaa]

@BryonLewis this seems to be a symptom of a more general issue we are having where DIVE anticipates hard-coded relative paths to images/videos. We have a labelstudio instance as well and it has a path field:

"""
{
"id": 332,
"data": {
"image": "gs://nmfs-dev-uc1-pifsc/SFM03/AIML/training_imagery/ESA_AGLO/raw/OCC-TUT-001_2023_27.JPG"
},
"annotations": [
{..........
"""

Common formats like COCO have a few fields that could be useful. VIAME CSV has a file field, as well, at least for images (the 2nd column, I think).

How difficult do you think it would be to un-hard-code this behavior, so that we could specify a relative or absolute URL for the images/videos? Right now, we are faced with the prospect of having to maintain multiple copies of rather large datasets to work around this. I think that the kwcoco data structure supports this sort of thing, so that might be useful here.

[BryonLewis]

just confirming, the earlier issue was mentioning .csv files during GCS Import, this is talking about COCO JSON import. So there may be a mix of COCO and CSV files imported when using the GCS import.

For the COCO import should want to rely solely on the filename and not the image path because you have GCP storage bucket paths. I can make it so it only looks at the filename and assumes it is relative to the COCO json file location.

[csbrown-noaa]

I can create a new issue if you like. Let me describe the actual use case and we can go from there:

We have GCP buckets full of data. The data are not VIAME/DIVE-specific. They are just videos and images of fish or coral or whatever. The data aren't set up to specifically back a DIVE instance, they are just data. We also have annotations, generated through some means, either manual, with a VIAME model, or with another model. At annotation creation time, we take note of the URL of the video/image, and we log that into the annotations. The annotations are actually not stored near the data, but in a separate bucket. This is because a single video may have multiple sources of annotations - e.g. from multiple models that we want to compare, from a human, etc.

When we import the data into DIVE, it seems that DIVE expects the video/imagery and the annotations to be near each other in some particular fashion. This seems to apply whether we are importing CSV data, or COCO.json data, imagery or videos - although the particular mechanism is different in each case, the general need to arrange our files in a particular fashion seems to be a common thread.

Perhaps we are missing something, and you can advise? How can we have one bucket full of data, another bucket with annotations, and load these into DIVE through GIRDER without having to move them around?

[csbrown-noaa]

Opening a new issue.