Serve small text-based files inline instead of redirecting to S3

[kabilar]

Problem

GET requests for assets such as .json, .yaml, .tsv, .md, and .bidsignore files are redirected to S3, which serves them with a Content-Type: application/octet-stream. Browsers download these files rather than displaying them inline.

Fix

dandidav now fetches small text-based files (≤ 10 MiB) itself and serves them with a browser-displayable content type instead of redirecting:

• JSON — any .json file, plus Zarr metadata files .zattrs/.zarray/.zgroup (which have no extension) — served as application/json
• YAML — .yaml/.yml — served as text/yaml, matching the virtual dandiset.yaml files
• Other text — .tsv/.csv/.md/.txt/.bidsignore — served as text/plain, which browsers reliably display inline (more specific types like text/tab-separated-values tend to be downloaded)

Files larger than 10 MiB continue to be redirected to S3 as before, to avoid buffering large amounts of data in memory. This also covers Zarr entries served via both the Archive and the Zarr manifests.

In the HTML collection view, links for these files now point to dandidav's own path (rather than directly to the S3 download URL) so that clicking them displays the file inline.

Verification

• Added inline_content_type unit tests covering JSON / YAML / plain-text / Zarr-metadata / non-inline cases (case-insensitive extensions included)
• cargo test (367 passing), cargo clippy, and cargo fmt --check all clean
• Tested end-to-end against real assets in dandisets/001412: dataset_description.json → application/json and .bidsignore → text/plain, both 200 with no Content-Disposition (display inline), versus the current 307 → S3 → application/octet-stream (download)

🤖 Generated with Claude Code

[kabilar]

cc @satra @ayendiki @balbasty

[kabilar]

Hi @yarikoptic, for context I would like to be able to directly display text files (zarr.json, tsv, etc.) in the browser rather than downloading. I tested the functionality locally and it works, but I relied on claude to generate the code since I don't have much experience with rust. Thanks.

[kabilar]

Here is a dandiset with several of those file types (tsv, json, yaml, .bidsignore) that can be used for testing: https://webdav.dandiarchive.org/dandisets/001412/draft/

[yarikoptic]

I think it might be just a matter of providing extra url options like we do on dandiarchive.org already: e.g. on https://dandiarchive.org/dandiset/001412/draft/files?location= you can

1. open in the browser -- https://api.dandiarchive.org/api/assets/7d298d77-4bca-4a3f-91e3-ab81b22ebd48/download?content_disposition=inline
2. download -- https://api.dandiarchive.org/api/assets/7d298d77-4bca-4a3f-91e3-ab81b22ebd48/download/

so I would have just added that [view] or just 👁 which would have the URL with the URL with ?content_disposition=inline to the download API endpoint. Or am I missing the goal here?

[kabilar]

Hi @yarikoptic, that's a good idea. Using content_disposition=inline worked for the .json file as you showed (and yaml currently displays inline with webdav), but it doesn't work for .bidsignore, .tsv, zarr.json. The asset still downloads. Would I need to change the DANDI download API endpoint to display inline for these asset types? Thanks.

• tsv: https://api.dandiarchive.org/api/assets/e4a12fcd-3361-4e93-a976-bbb55fc5ed72/download?content_disposition=inline

• bidsignore: https://api.dandiarchive.org/api/assets/037a862c-6230-45e7-8d0d-8e1941b24275/download?content_disposition=inline

• zarr.json: https://api.dandiarchive.org/api/assets/d89d1e80-5280-4d91-9fc7-f86569db9f95/download?content_disposition=inline

  

[yarikoptic]

sure -- as much as possible to centralize/formalize instead of ad-hoc solutions on webdav level. Here worth double checking (I forgot) how it is interacting with encodingFormat -- see discussion in elderly

• "Open in browser" on dataset_description.json results in Download request dandi-archive#2027 (comment)

and potentially just give it current assessment etc. Related (might be addressed by working on this)

• .tsv files must be previewed natively in the browser dandi-archive#2405