Media Juggler

Scripts to tag, convert, and upload my music, audiobooks, ebooks, and comics to my self-hosted media services.

Table 1. Self-Hosted Services

Media	Service
Audiobooks	Audiobookshelf
Books, Comics, Manga	Kavita
Music	Jellyfin

ℹ️

I’m currently putting in a lot of work to add metadata for audiobooks, books, and comics to several data sources to make managing your digital media easier, these include open data sources such as BookBrainz, MusicBrainz, and Wikidata, as well as {Open Library} and Hardcover. I’ve also contributed metadata for video games to PCGamingWiki, which can be used by to manage your save data with Ludusavi.

DRM-Free Media Sources

Digital rights management or DRM, restricts your ability to use and manage content that you purchase. DRM traps you within the confines of the limited implementation and ecosystem completely subject to the whims of the distributor. It’s effectively a tool for vendor lock-in. You want to read a book you purchased from Amazon on your Kobo ereader? Too bad. DRM prevents that. In many jurisdictions, circumventing DRM in any way is criminalized. This is the case in the United States thanks to Section 1201 of the Digital Millennium Copyright Act, often abbreviated as the DMCA, which makes DRM circumvention illegal, even for fair use purposes.

Thus, for self-hosting media, it’s important to vote with your wallet and purchase DRM-Free media. It’s not always straightforward to find where you can purchase legitimate, DRM-Free content and many sellers don’t describe things in detail. Below are lists of my favorite sources for DRM-Free media. For music, DRM isn’t so much an issue, so I’ve listed my favorite sources of high-quality music. I’ve include a list of DRM-Free sources of video games, even though I don’t have scripts that import video games.

DRM-Free Audiobooks

• Libro.fm

• Humble Bundle

• Dragonsteel

DRM-Free Books

• Dragonsteel

• eBooks.com (Not all titles are drm-free)

• Humble Bundle

• Fanatical

DRM-Free Comics

• Humble Bundle

• Fanatical

DRM-Free Manga

• Humble Bundle

• Fanatical

• Denpa

• J-Novel Club

High-Quality Music

• HDtracks

• 7digital

DRM-Free Video Games

• itch.io

• GOG

How It Works

You purchase DRM-Free media, then point these scripts at it to automatically tag, convert, and upload it the correct location on your media server. Files are automatically removed after being imported. The scripts also perform lossless optimization to reduce file sizes where applicable. Comics in the EPUB format are converted to CBZ, which can lose information such as chapters, but provides a better reading experience on KOReader. There are scripts for each type of media, which can be found under overlays/media-juggler/. Metadata is retrieved from various sources, as detailed in the Metadata Sources table.

Table 2. Metadata Sources

Media	Metadata Source
Audiobooks	MusicBrainz
Books, Light Novels	Currently, a bunch of sources through Calibre plugins, including Hardcover and Wikidata.
Comics, Manga, Manhwa	Comic Vine, Wikidata
Music	MusicBrainz

Several tools are used to get metadata for different media, which are documented in the Metadata Tools table.

Table 3. Metadata Tools

Media	Tool
Audiobooks	No external tools
Books, Light Novels	Calibre
Comics, Manga, Manhwa	No external tools
Music	Beets

Once the media is uploaded on the server, it’s automatically picked up by the pertinent service. The services for each media type are documented in the Self-Hosted Services table.

Metadata Handling

Metadata is embedded directly in the media files when possible. EPUBs and audio files support natively embedding metadata. Audiobookshelf can’t read the ISBN from the embedded metadata in M4B files, so a sidecar metadata.opf file containing the ISBN is placed in the same directory as the audiobook. For comics and the like, ComicInfo and MetronInfo metadata files are incorporated in the CBZ. While PDFs technically support embedding metadata, it’s not well supported in practice. PDFs are embedded with the limited fields supported by the ebook-meta utility from Calibre. For comics in PDF formats, I sidecar ComicInfo and MetronInfo metadata files with the PDFs, even though sidecar metadata is not supported by Kavita.

Optimizations

I make use of several utilities to losslessly decrease file sizes. I only optimize images, ebooks, and comics. I don’t really mess with audio although I have some experimental options for messing around with that. Images are optimized with image_optim using its default settings followed by Efficient Compression Tool, which can typically gain a small improvement for PNGs. For comic book archives and EPUBs, this is easy enough as these are just ZIP archives of files. I also optimize the ZIP archive compression with Efficient Compression Tool. For PDFs, I use minuimus and pdfsizeopt, with a couple tweaks to use oxipng with max settings followed by Efficient Compression Tool. This often leads to significant space savings for manga from Fanatical and Humble Bundle, usually 25-35%, but it takes a long time on my crappy laptop, like 16 hours for a 200 MiB PDF.

Usage

This repository provides a collection of scripts to make it easier to add various digital media to my collection. This includes music, audiobooks, ebooks, and comics. When purchasing media, it often needs to be renamed, converted, tagged with correct information, losslessly optimized to reduce files size, and finally uploaded to the appropriate location on my media server. The utilities here aim to automate this as much as possible. Ultimately, the workflow should be as simple as downloading the media to the corresponding import directory on my computer. There’s quite a bit of initial set up that needs to be completed first, though, so that’s where we’ll begin.

❗	Be sure to increase the maximum_cover_size tweak or Calibre will shrink cover images when embedding metadata. ~/.config/calibre/tweaks.json { "maximum_cover_size": [ 999999999999, 999999999999 ] }

Use this Nix flake in your Home Manager configuration to incorporate these scripts in your project. Here’s an example of how to incorporate this repository’s flake as part of your project.

1. Add media-juggler to flake.nix and the provided Home Manager module to the modules of your configuration.

   flake.nix

   {
     inputs = {
       flake-utils.url = "github:numtide/flake-utils";
       home-manager = {
         url = "github:nix-community/home-manager";
         inputs.nixpkgs.follows = "nixpkgs";
       };
       media-juggler.url = "github:jwillikers/media-juggler";
       nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
     };
     outputs =
       {
         self,
         flake-utils,
         media-juggler,
       }:
       flake-utils.lib.eachDefaultSystem (
         system:
         let
           pkgs = import nixpkgs {
             inherit system;
           };
         in
         {
           homeConfigurations.jdoe = home-manager.lib.homeManagerConfiguration {
             inherit pkgs;
   
             modules = [
               media-juggler.hmModules.media-juggler
             ];
           };
         }
       );
   }

2. Apply the updated Home Manager configuration.

   home-manager --flake .#$(id --name --user)@$(hostname --short) switch

3. Create an Adobe account in order to use Adobe Digital Editions. It is recommended to use a dedicated account for this in the event that the account is banned by Adobe. Be sure to read the Setup section of the README for the Calibre ACSM Input plugin to understand all of the implications.

4. Sign up for a Comic Vine account here.

5. Get your Comic Vine API token from here. This will be used later to configure some of the tools.

6. Set the environment variable COMIC_VINE_API_KEY to your Comic Vine API key.

   set --export --universal COMIC_VINE_API_KEY <Comic Vine API key>

7. Sign up for a Hardcover account here.

8. Get your Hardcover API token from here.

9. Open Calibre.

   setsid calibre &>/dev/null &

10. Go to Preferences › Plugins › Get new plugins.

11. Install the Hardcover, KFX Input, and Barnes & Noble plugins.

12. Restart Calibre to finish installing the plugins.

13. Go to Preferences › Plugins.

14. Select the Hardcover plugin.

15. Push the Customize plugin button.

16. Add your Hardcover API Key, everything past the Bearer word, to the API Key field.

17. Hit OK.

18. Go to Preferences › Plugins.

19. Select the DeACSM plugin.

20. Push the Customize plugin button.

21. Add keys for Adobe Digital Editions for the DeACSM plugin if needed. To import a previously exported activation, go to Preferences › Plugins › DeACSM › Customize plugin › Import existing activation backup (ZIP). For a new activation, go to Preferences › Plugins › DeACSM › Customize plugin › Link to ADE account. After a new activation export the activation using Preferences › Plugins › DeACSM › Customize plugin › Export account activation data and back up this file.

22. Hit Apply.

23. Hit Close.

1. Set the environment variable WIKIDATA_USERNAME to your user’s Wikidata username. I use the fish shell, thus the syntax below.

   set --export --universal WIKIDATA_USERNAME '<username>'

2. Create a Wikidata API token according to the Setting up OAuth 2.0 instructions.

   1. Create a Wikidata account.

   2. Log in to Wikidata.

   3. Register a new OAuth client here.

      1. Include the work test in the application name to make a temporary token that will last 30 days and not require approval.

      2. Check the box This consumer is for use only by <username>.

      3. For Applicable project, select www.wikidata.org.

      4. For Applicable grants, the default permissions should suffice.

      5. Check the final box to agree to the terms.

      6. Hit Propose consumer.

3. Set the environment variable MEDIA_JUGGLER_WIKIDATA_ACCESS_TOKEN to the access token for your API OAuth2 token created in the previous step.

   set --export --universal MEDIA_JUGGLER_WIKIDATA_ACCESS_TOKEN '<wikidata-access-token>'

4. Set up an API token for Hardcover.

   1. Create a Hardcover account.

   2. Find your API key from here.

   3. Set the environment variable MEDIA_JUGGLER_HARDCOVER_API_TOKEN to the API key from the previous step.

      set --export --universal MEDIA_JUGGLER_HARDCOVER_API_TOKEN '<hardcover-api-key>'

      ℹ️	The Hardcover API key will expire in one year and reset on January 1st.

5. Enable debugging by setting the log level for Nushell.

   $env.NU_LOG_LEVEL = "DEBUG"

6. The rsync utility is recommended for transferring files over the network. Install rsync on the server and pass the --use-rsync flag to the script.

7. Be sure to backup any files before attempting to import them. The import process automatically deletes files after import, so if it’s your only copy, it may be deleted when an import fails or converted in a way that is irreversible.

8. Now, the provided scripts can be run as follows.

   ℹ️	SSH-style paths like server:/path/to/file are supported. In this case, the media is copied to the machine before being imported.

   import-audiobooks.nu

   Providing a MusicBrainz release ID is recommended for books containing audio files longer than about 9 hours, as AcoustID doesn’t yet support fingerprints for such tracks.

   import-audiobooks.nu --use-rsync file.m4b --musicbrainz-release-id 2cfe3579-56f3-4360-8165-d02b6b2252bb

   import-comics.nu

   The import-comics.nu script requires a Comic Vine issue ID. It’s possible to supply other IDs, such as a Wikidata, BookBrainz, or ISBN-13 with CLI flags. If the Comic Vine issue ID can be gleaned from Wikidata, then it isn’t required on the CLI. If the comic matches a hash on Wikidata, no external identifiers are required. If identifiers already exist in the metadata of the file, then those IDs are not requires. The script also attempts to extract the ISBN from within the pages of the comic. The script defaults to preferences for Manga. Pass --manga No for comics and --manga Yes for manga read from left-to-right. The default value of --manga is YesAndRightToLeft.

   import-comics.nu --use-rsync file.epub --isbn 9781234561234

   import-ebooks.nu

   Providing an ISBN-13 identifier with CLI flag is recommended.

   import-ebooks.nu --use-rsync file.epub --isbn 9781234561235

   import-music.nu

   The import-music.nu script can take a list of files, a zip archive, or a directory as its argument. Files should be organized in folders for each album or single.

   import-music.nu --use-rsync album.zip

Importing Multiple Items

The scripts only support importing a single item at a time when providing an identifier. To batch import files, the command can be called for each file and identifier from with a script. The command below does this for each file in the current directory, using Nushell. It provides the Wikidata edition ID for each comic.

ls | get name | sort --natural | zip [Q139382000 Q139398001 Q139398002] | each {|| overlays/media-juggler/import-comics.nu --use-rsync $in.0 --wikidata-edition-id $in.1}

Develop

The nix develop command can be used to enter or run commands in an environment with all of the necessary dependencies. For convenience, direnv can be used to automatically load this environment when entering the project’s directory. The mkhl.direnv VSCode extension integrates this environment in VSCode for development. Nix also generates the configuration for pre-commit, which automates formatting and various checks when committing changes. Follow the instructions here to set up your development environment using Nix.

1. Install an implementation of Nix, such as Lix used here.

   curl -sSf -L https://install.lix.systems/lix | sh -s -- install

2. Clone this project’s repository.

   git clone https://github.com/jwillikers/media-juggler.git

3. Change into the project directory.

   cd media-juggler

4. Install direnv for your system according to the direnv installation instructions.

   sudo rpm-ostree install direnv
   sudo systemctl reboot

5. Integrate direnv with your shell by following the instructions on the direnv Setup page.

6. Permit the direnv configuration for the repository.

   direnv allow

7. Use nix build to run the appropriate target.

   nix build .#calibre-plugins.acsm

Future Work

Currently, the large task I’d like to complete is adding support for other metadata sources besides Comic Vine for metadata, such as BookBrainz, Wikidata, and possibly Hardcover. This migration will greatly reduce the complexity of the manga import process which is ridiculously convoluted right now.

A similar change for eBooks, where I embed data directly in EPUBs without Calibre, may also be possible. This would remove reliance on Calibre plugins and make it possible to incorporate metadata from BookBrainz.

I’d also like to incorporate chapters from the Table of Contents section of editions in Open Library for incorporating chapters in CBZs somehow.

import-audiobooks.nu Todo

• ❏ Incorporate Hardcover and Wikidata ids in the metadata.

import-comics.nu Todo

• ❏ Add a flag to embed cover from file.

• ✓ Use the Comic Vine API directly instead of using ComicTagger.

• ❏ Cache Comic Vine API responses.

• ❏ Switch to using the BookBrainz API, Wikidata, and the Hardcover API, in addition to Comic Vine. Comic Vine’s data is not open and it’s a pain to contribute to.

• ✓ Automatically look up Hardcover, Wikidata, BookBrainz, and Open Library IDs by ISBN.

• ✓ Automatically fetch missing ids from Wikidata, BookBrainz, and Open Library.

• ❏ Remove the residual files and directory exported by Calibre.

Contributing

Contributions in the form of issues, feedback, and even pull requests are welcome. Make sure to adhere to the project’s Code of Conduct.

Code of Conduct

Refer to the project’s Code of Conduct for details.

License

This repository is licensed under the MIT license.

© 2026 Jordan Williams

Authors

Jordan Williams