Add hdf5-meta extension for an optional HDF5 metadata sidecar#355
Closed
daltoncass wants to merge 2 commits into
Closed
Add hdf5-meta extension for an optional HDF5 metadata sidecar#355daltoncass wants to merge 2 commits into
daltoncass wants to merge 2 commits into
Conversation
Defines the hdf5-meta v1.0.0 extension: an OPTIONAL HDF5 sidecar file (recording.sigmf-meta.h5) that stores a columnar duplicate of a Recording's metadata for faster, smaller column-oriented access. The .sigmf-meta JSON remains complete and authoritative; the sidecar is a derived cache. Tools without HDF5 support ignore the extension and read the JSON unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Clarify that hdf5-meta is a metadata-only performance cache: it is not a sample-data container, does not represent multiple channels or arrays, and does not change the SigMF data model. In particular it leaves both multichannel mechanisms (core:num_channels and SigMF Collections) untouched and composes with Collections per-Recording. Pre-empts a reviewer question on the spec PR. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
Author
|
Moving this to the community-extensions repo per your guidance that extensions start there and migrate into core if adopted by multiple entities: sigmf/community-extensions#2 The extension text is unchanged aside from a new Scope and Non-Goals section (§0.1) addressing the multichannel question — it clarifies that |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed in #354.
Summary
Adds a canonical
hdf5-metaextension namespace (v1.0.0) defining an OPTIONAL HDF5 sidecar file —recording.sigmf-meta.h5— that stores a columnar duplicate of a Recording's metadata for faster, smaller, column-oriented access on Recordings with largecaptures/annotationsarrays.Design
.sigmf-metaJSON file remains complete, authoritative, and SigMF Compliant. The HDF5 file is a derived cache and is never required to read a Recording."optional": trueincore:extensions.global→ HDF5 attributes;captures/annotations→ compound (structured) datasets, one column per field, dot-notation field names (core.sample_start), gzip SHOULD.additional_content.md: extensions "MAY define new top-level SigMF Objects, key/value pairs, new files, ...").Extension fields (global)
hdf5-meta:file.sigmf-metahdf5-meta:versionNotes
Compliance
A Recording with this extension is compliant when (1) the JSON file is a valid SigMF Recording without the sidecar, (2)
hdf5-metais listed incore:extensionswithhdf5-meta:fileset, and (3) if present, the sidecar matches the structure in the extension doc and is content-equivalent to the JSON.