Repository navigation
feat(ir): virtualize MP3 files through the Rust IR, as the MP3 profile (§17) - #91
Merged
Merged
Conversation
…e (§16) Add the MP3 profile (spec/virtualize/mp3.md, §16 of spec/virtualize.md): an MPEG-1, MPEG-2 or MPEG-2.5 Layer III file, chosen by `ID3` or a frame header at byte 0, becomes a plain Zarr array `samples` of int16 [time, channel], the first non-image convention. The parser reads the whole file, walks the frames (a chain from each frame's end, a confirmed resync after junk), finds the ID3v2, ID3v1 and APE tags and the Xing, Info or VBRI frame, and rejects Layers I and II, free format and a stream whose parameters change. Each chunk is 256 frames, referenced with a 4-byte sample count and a lead-in: the two frames before the chunk and the fewest before them holding the version's largest bit reservoir, so that a decoder decoding the chunk's bytes in one pass gives the stream's samples; the bytes between frames are left out of the ranges. The root keeps the stream's parameters, the info frame's LAME fields and the ID3v1 tag; refkv_source is the file's IR mirror. The `mp3` Zarr codec (python/src/refkv/zarr/codecs.py) decodes a chunk with miniaudio, the new `audio` extra, keeping the last `keep` samples. Fixtures (fixtures/mp3, 21 accepted and 12 rejected, encoded by LAME and ffmpeg, with ffmpeg's samples as expected values) are checked by python/tests/ir/test_mp3.py (chunked decoding equals miniaudio's sequential decoding exactly, and ffmpeg's within rounding) and js/test/mp3/verify.py; the generator needs lame and ffmpeg, so `just fixtures` skips it as it does NDPI's. corpus_mp3.txt lists a LibriVox recording. Assisted-by: ClaudeCode:claude-fable-5-1 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…alization-9618b8 # Conflicts: # demo/index.html # docs/virtualize/README.md # js/src/virtualize/index.ts # js/src/virtualize/ir/run.ts # js/src/virtualize/profiles.ts # python/src/refkv/virtualize/profiles.py # rust/refkv-ir/profiles.json # rust/refkv-ir/src/run.rs # spec/conventions.md # spec/virtualize.md # spec/virtualize/profiles.json
d-v-b
changed the title
This was referenced Oct 23, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤖 AI text below 🤖
An MP3 file (MPEG-1, MPEG-2 or MPEG-2.5 Layer III) is now virtualized as a plain Zarr array
samplesof int16[time, channel], the first convention whose source is not an image: chunks of 256 frames reference the frames plus the lead-in a decoder needs (the bit reservoir and the filter bank state), and the newmp3codec decodes a chunk with miniaudio when it is read. The root keeps the stream's parameters, the Xing/Info frame's LAME fields and the ID3v1 tag, andrefkv_sourceis the file's IR mirror, so the file is rebuilt byte for byte.What changed
src/mp3.rs(parser),schema/mp3.json(source model),project/mp3.rs(projection); the format is wired into the run, the Python bindings, the CLI and the browser host;_set_chunk_framesis a test knob like_set_copy_threshold.spec/virtualize/mp3.md(convention and profile §17, after main's QPTIFF §16), the §1.2 detection row, the registry entry and its derived tables, the schema, aTECHNIQUES.mdrow.audioextra (miniaudio), themp3codec, the sniff;python/tests/ir/test_mp3.py.js/test/mp3/verify.py.fixtures/mp3(21 accepted, 12 rejected) fromfixtures/generators/mp3/write_fixtures.py, which needslameandffmpeg, sojust fixturesskips it like NDPI's;expected/*.npyare ffmpeg's decodes.docs/virtualize/mp3/README.md,corpus_mp3.txt(a LibriVox chapter on archive.org).Verified (after merging main): the Rust core's tests,
pytest(1972 passed before the merge; the MP3, QPTIFF, conventions and refstore tests after it), the Node tests (638),just sections,js/test/mp3/verify.py,just cli-testandjust cli-identical --only mp3(170 conversions identical).Noted, not changed: the archive reader sends no
User-Agent, so reading a virtualized file hosted on Wikimedia Commons gets HTTP 403 while virtualizing it (the planner, which sends one) works; the corpus uses archive.org instead.Stack bookkeeping
Independent of other open PRs;
origin/main(the QPTIFF profile) is merged in, which moved MP3 from §16 to §17.🤖 Generated with Claude Code