Migrate music-metadata-browser to music-metadata

[Borewit]

Changes:

• Migrate music-metadata-browser to music-metadata.
• Close web stream, when fetching metadata from URL
• Pass Content-Type and Content-Length to music-metadata
• Deprecate polyfills:
  • process
  • buffer
• Adds ability to decode Ogg/FLAC radio streams

Related:

• Migrate music-metadata-browser to music-metadata #88
• Convert more functions to async/await #91

[hvianna]

@Borewit So I went to have a look at my previous migration attempt and remembered why I didn't go through with it..

If I understand it correctly, parseWebStream() requires that response.body is a readable byte stream, which is still unsupported on Safari (as per MDN documentation). It also requires at least Chromium 116, and I've been trying to keep audioMotion running on Chrome 87, because I use an old media box which is stuck on that version.

I don't know if it would be possible to have like a polyfill to provide compatibility, or if there is any alternative for parseWebStream() that wouldn't have this requirement.

[Borewit]

I think I have now handled the fallback for Safari. Yet I have no Safari to test with.

[Borewit]

I think actually the default (non byte) web stream also works: Borewit/peek-readable#786

Note that module has been merged (with the fix) with https://github.com/Borewit/strtok3

Let me know if you see any remaining issue on your old media box @hvianna .

[hvianna]

Made some more observations. I hope to have time to test this more throughly over the weekend.

[hvianna]

On web server mode it now reads a large part of the files to get the metadata and then stalls after some time. 😟 These are big 5.1 FLAC files ranging from ~50MB to ~500MB each. It seems to work fine on file system mode though.

BTW, my web parser function is not working with the directory listings generated by webpack-dev-server, so you may need to use npm start to test this.

[Borewit]

On web server mode it now reads a large part of the files to get the metadata and then stalls after some time. 😟 These are big 5.1 FLAC files ranging from ~50MB to ~500MB each. It seems to work fine on file system mode though.

BTW, my web parser function is not working with the directory listings generated by webpack-dev-server, so you may need to use npm start to test this.

I like to load a portion of my library (I have more then 1 TB) into the queue, but how do I move /drag multiple folders into the playlist queue?

I have fairly large playlist as well, and I noticed some tracks did not resolved (dev branch). Not sure if this is caused by the obscure fetch implementation or of it has something to do with file encoding.

[hvianna]

I like to load a portion of my library (I have more then 1 TB) into the queue, but how do I move /drag multiple folders into the playlist queue?

Adding folders is not supported. You'll need to enter each folder and add all files (double arrow button).

[Borewit]

I did a test, using the server fastify implementation #101, using a script running in Node.js traversing my music library.
Where the files are stored on NAS.

import {parseWebStream} from 'music-metadata';

const rootFolder = 'http://localhost:8080/music';

let numberOfFilesScanned = 0;

function increaseCounter() {
  ++numberOfFilesScanned;
  if (numberOfFilesScanned % 100 === 0) {
    const delta = performance.now() - startTime;
    const timePerFile = Math.round(100 * delta / numberOfFilesScanned) / 100;
    console.log(`Scanned ${numberOfFilesScanned} audio files, in ${timePerFile} ms per file`);
  }
}

async function scanFolder(folder) {
  // console.log(`Scanning folder: ${folder}`);
  const response = await fetch(`${folder}/`);
  if (response.ok) {
    const folderListing = await response.json();
    for (const file of folderListing.files) {
      if (isAudioFile(file.name)) {
        const url = `${folder}/${encodeURIComponent(file.name)}`;
        // console.log(`Fetching ${url}...`);
        const response = await fetch(url);
        if (response.ok && response.body) {
          const mm = await parseWebStream(response.body);
          // await response.body.cancel();
          increaseCounter();
        }
      }
    }
    for (const subFolder of folderListing.dirs) {
      await scanFolder(`${folder}/${encodeURIComponent(subFolder.name)}`); // Recursion
    }
  }

}

function isAudioFile(filename) {
  const fileExt = filename.split('.').pop()?.toLowerCase();
  return ['flac', 'mp3', 'wav', 'm4a', 'opus'].indexOf(fileExt) !== -1;
}

const startTime = performance.now();
scanFolder(rootFolder);

Which outputs:

Scanned 100 audio files, in 45.31 ms per file
Scanned 200 audio files, in 46.18 ms per file
Scanned 300 audio files, in 45.79 ms per file
Scanned 400 audio files, in 45.22 ms per file
Scanned 500 audio files, in 46.6 ms per file
Scanned 600 audio files, in 50.2 ms per file
Scanned 700 audio files, in 51.1 ms per file
Scanned 800 audio files, in 51.32 ms per file
Scanned 900 audio files, in 50.32 ms per file
Scanned 1000 audio files, in 49.85 ms per file

These are all FLAC files. So it scans a FLAC file 50 ms, I think that performance is not to bad.

[Borewit]

On web server mode it now reads a large part of the files to get the metadata and then stalls after some time. 😟 These are big 5.1 FLAC files ranging from ~50MB to ~500MB each. It seems to work fine on file system mode though.

BTW, my web parser function is not working with the directory listings generated by webpack-dev-server, so you may need to use npm start to test this.

Nailed the degradation in performance here: #103

[Borewit]

Restored mostly the original metadata load limitation mechanism, as I made things worse.