v1.4: FLAC/ALAC/AIFF support, USB drive mode, crate selector, base64 fix

.gitignore

@@ -0,0 +1,4 @@
+.DS_Store
+__pycache__/
+*.pyc
+*.xml

README.md

@@ -1,9 +1,20 @@
-# serato2rekordbox v1.3
+# serato2rekordbox v1.4
 
-This command line tool converts your Serato DJ Pro library (playlists, tracks, metadata, beatgrids, and hot cues) into a Rekordbox XML file that can be imported into Rekordbox DJ (tested on 6.8.5 and should work on older/newer versions) and can be exported easily to a USB or just used in Rekordbox in HID mode. 
+This command line tool converts your Serato DJ Pro library (playlists, tracks, metadata, beatgrids, and hot cues) into a Rekordbox XML file that can be imported into Rekordbox DJ (tested on 6.8.5 and should work on older/newer versions) and can be exported easily to a USB or just used in Rekordbox in HID mode.
 
 ## Changelog
 
+### v1.4:
+
+- **FLAC hot cues and beatgrids** -- Serato writes hot cues and beatgrids as double-base64-encoded Vorbis comments (`SERATO_MARKERS_V2`, `SERATO_BEATGRID`) in FLAC files. The extractor now correctly decodes these, matching the upstream staddle/serato2rekordbox implementation. ID3 GEOB frames are tried as a fallback for third-party tools
+- **USB drive / path remapping support** -- point at a USB drive containing both `_Serato_/` and your music files with a single path; the script auto-discovers crates and remaps track paths from the original library location to the USB
+- **Interactive crate selector** -- pick which crates to convert before processing (curses-based on macOS/Linux, text-based on Windows)
+- **New format support** -- `.flac`, `.alac`, `.aiff` files are now fully supported (metadata, hot cues, beatgrids)
+- **CLI arguments** -- `--serato`, `--music`, `--output`, `--skip-crate-selection` for full control without editing code
+- **Missing beatgrid is no longer fatal** -- tracks without Serato beatgrid data are still included in the XML (Rekordbox will analyse on import)
+- **FLAC metadata fix** -- ID3 tags are now loaded correctly (previously crashed on all `.flac` files)
+- **Code cleanup** -- removed unused imports, consolidated format dispatch, improved error messages
+
 ### v1.3:
 
 - Fixed issue where subcrates inside subcrates were not processed at all
@@ -38,17 +49,19 @@ I couldn't find any other free software that could convert it so I decided to ma
 *   **Track Metadata:** Transfers essential metadata including Title, Artist, BPM, and Key.
 *   **Hot Cue Transfer:** Extracts and transfers hot cues.
 *   **Accurate Beatgrids:** Extracts the Serato beatgrid data directly from the audio files to extract the *first beat position* from the audio file's beatgrid data and includes it in the XML. This tells Rekordbox exactly where the first beat is, allowing it to correctly align the entire beatgrid without needing to re-analyse it itself.
-*   **Automatic Serato Folder Detection:** Automatically attempts to find your Serato `_Serato_` folder on standard Windows, macOS and Linux locations.
+*   **USB Drive Support:** Run the script against a USB drive that contains both the Serato metadata folder and your music files. Track paths stored in the original library location are automatically remapped to the USB via a fast filename index.
+*   **Interactive Crate Selector:** Choose which crates to convert via an in-terminal checkbox UI (macOS/Linux) or numbered menu (Windows). Skip with `--skip-crate-selection`.
+*   **Automatic Serato Folder Detection:** Automatically attempts to find your Serato `_Serato_` folder on standard Windows, macOS and Linux locations, or from within a USB drive root.
 *   **Detailed Error Reporting:** Collects and reports errors (missing files, unsupported formats, processing errors, crate reading issues) in a clear, grouped summary at the end. Failed tracks are excluded from the output XML.
-*   **File Support:** Supports conversion for `.mp3`, `.m4a` and `.wav` audio files found in your Serato library.
+*   **File Support:** Supports conversion for `.mp3`, `.m4a`, `.alac`, `.wav`, `.flac`, and `.aiff` audio files.
 *   Normal crates and subcrates are supported.
 
 ## Prerequisites
 
 Before running the script, ensure you have the following:
 
-1.  **Python 3:** The script is written in Python 3. You can download it from [python.org](https://www.python.org/downloads/).
-2.  **Serato DJ Pro:** Serato must be installed and run at least once on the computer where you run this script, as the script needs to access the Serato `_Serato_` folder and the music files.
+1.  **Python 3:** The script is written in Python 3. You can download it from [python.org](https://python.org/downloads/).
+2.  **Serato DJ Pro:** Serato must be installed and run at least once on the computer where you run this script (or your Serato metadata folder must be available on a USB drive).
 3.  **Rekordbox DJ:** You will need Rekordbox DJ (version 6 recommended) to import the generated XML file.
 
 ## Installation
@@ -60,56 +73,94 @@ Before running the script, ensure you have the following:
     ```
     Or download the ZIP file and extract it.
 
-2.  **Install Dependencies:** Open your terminal or command prompt, navigate to the script's directory, and install the required Python package:
+2.  **Install Dependencies:** Open your terminal or command prompt, navigate to the script's directory, and install the required Python packages:
     ```bash
     pip install tqdm mutagen
     ```
 
 ## Usage
 
-1.  **Open Terminal/Command Prompt:** Navigate to the directory where you saved the script files.
-2.  **Run the Script:** Execute the script using Python 3:
-    ```bash
-    python3 serato2rekordbox.py
-    ```
-3.  **Let the magic happen:** The script will attempt to auto-detect your Serato folder, read your crates, process your tracks, structure playlists, and finally write the XML file. It shouldn't take longer than a few seconds to complete unless you have a huge library.
-4.  **Review Output:** After completion, the script will print a summary of successful/unsuccessful conversions and list any items that could not be processed, grouped by the type of error.
+### Quick start (auto-detect everything)
+
+If your Serato library is in the default location and music files are where Serato expects them:
+
+```bash
+python3 serato2rekordbox.py
+```
+
+### USB drive (one command)
+
+Your USB drive contains `_Serato_/` (metadata) plus your music folders:
+
+```bash
+python3 serato2rekordbox.py /Volumes/YourUSB
+```
+
+The script will:
+1.  Find `_Serato_/` inside the USB for crate files
+2.  Index all music files in sibling folders (skipping `_Serato_`, `_Serato_Backup`, system folders)
+3.  Remap crate paths from the original library location to the USB via filename lookup
+4.  Open the crate selector so you can pick what to convert
+
+### Separating Serato metadata from music files
+
+```bash
+python3 serato2rekordbox.py --serato ~/Music/_Serato_ --music /Volumes/USB/Music
+```
+
+### Command-line options
+
+| Flag | Description |
+|---|---|
+| `root` | (positional) Path to a USB drive or folder containing both `_Serato_/` and music files |
+| `--serato PATH` | Path to the Serato library folder (contains `subcrates/`). Auto-detected when using positional `root`; otherwise defaults to `~/Music/_Serato_` |
+| `--music PATH` | Root path of the music files. Auto-detected when using positional `root`; otherwise defaults to the Serato folder |
+| `--output PATH` | Output XML file path. Defaults to `serato2rekordbox.xml` beside the script |
+| `--skip-crate-selection` | Skip the interactive crate selector and import all crates |
+
+### Crate selector
+
+On macOS and Linux a full-screen checkbox interface opens:
+- **Arrow keys** navigate, **Space** toggles, **A** selects all, **N** deselects all, **Enter** confirms, **q** cancels
+
+On Windows (or when curses is unavailable) a numbered menu appears:
+- Type comma-separated numbers to toggle crates (e.g. `1,3,5`), **a** for all, **n** for none, **Enter** to confirm, **q** to cancel
+
+Press **Enter** immediately to accept all crates and skip selection.
 
-The output file `serato2rekordbox.xml` will be generated in the same directory as the script.
+### Output
 
-**Note:** There are no configuration variables (`base_dir`, `serato_folder_path`) to change at the top of the script anymore, as it attempts to find the Serato folder automatically.
+After completion the script prints a summary of successful / unsuccessful conversions. The output file `serato2rekordbox.xml` is generated in the script's directory (or at the path specified by `--output`).
 
 ## Importing into Rekordbox
 
 Once `serato2rekordbox.xml` is generated:
 
 1.  Open Rekordbox DJ.
-2.  Go to Settings (Gear icon at top) > Advanced > rekordbox xml and select the generated XML file.
-3.  In the sidebar, drag the imported playlists into your USB and wait for Rekordbox to finish processing.
+2.  **Add your tracks first** -- drag the music folders from your USB into Rekordbox so the tracks appear in the library. This is required because Rekordbox only resolves playlist references for tracks it already knows about.
+3.  Go to Settings (Gear icon at top) > Advanced > rekordbox xml and select the generated XML file.
+4.  In the sidebar, the imported playlists will appear under the **Collection** tab. Drag them onto your USB export and wait for Rekordbox to finish processing.
 
-Rekordbox will import the playlists and tracks. The tracks should appear with their correct metadata (Title, Artist, BPM, Key), Hot Cues, and the accurate Beatgrids based on the first beat position provided in the XML. Rekordbox may still perform some background analysis (like waveform drawing), but it should respect the imported beatgrid and cue data.
+Rekordbox will import the playlists with their track ordering, Hot Cues, and the accurate Beatgrids based on the first beat position provided in the XML. Rekordbox may still perform some background analysis (like waveform drawing), but it should respect the imported beatgrid and cue data.
 
 ## Limitations
 
 *   This script primarily transfers playlists, basic metadata, hot cues, and the **first beat position** for the beatgrid. Other Serato-specific data like loops, specific track flags (e.g., played status) etc. may not work.
 *   Some tracks may not have the correct beatgrid data or key.
 *   Smart crates are not supported.
 *   Some beatgrids in Rekordbox appear to be slightly off beat even though perfectly on beat in Serato.
-*   You may have to manually adjust the Serato folder path in the code if it isn't auto detected.
 *   There may be occasional tracks which are not correctly processed for some unknown reason, however across my library of almost 4000 tracks there's only been a handful which have been problematic for me.
 
 ## Notes
 
 *   The script has been tested on my own Serato library which contains almost 4000 tracks and performs as intended. It was able to process the entire library in around 20 seconds and rekordbox exporting to my USB (HFS+) only took around 15 minutes.
-*   Rekordbox 6.8.5 is recommended at the moment as Rekordbox 7 is reported to export to USB much slower. 
+*   Rekordbox 6.8.5 is recommended at the moment as Rekordbox 7 is reported to export to USB much slower.
 *   `HFS+` seems to be the fastest USB file format but `FAT32` is more compatible with older Pioneer hardware.
-*   `.wav` files have not been extensively tested but appear to work fine.
 
 ## Future improvements
 
-- Support other file formats eg. `.flac`, `.alac`, `.aiff` - only `.mp3`, `.m4a` and `.wav` are supported at the moment.
 - Make a GUI?
-- I thought about trying to reverse engineer the USB export structure so the program could directly export to a USB itself without needing Rekordbox at all, however the USB structure (analysis, database etc) is extremely complex, would require alot of effort and likely wouldn't be as reliable.
+- I thought about trying to reverse engineer the USB export structure so the program could directly export to a USB itself without needing Rekordbox at all, however the USB structure (analysis, database etc) is extremely complex, would require a lot of effort and likely wouldn't be as reliable.
 
 ## Contributing
 

extract_aiff.py

@@ -0,0 +1,186 @@
+import io
+import logging
+import struct
+
+import mutagen
+from mutagen.aiff import AIFF
+
+from utils import convert_key_to_camelot
+
+
+def extract_metadata(file_path: str) -> dict:
+    """Extract metadata, hot cues, and beatgrid from an AIFF file.
+
+    Serato embeds ID3v2 tags in AIFF files (same GEOB structure as WAV/FLAC).
+    """
+    results = {
+        "metadata": {
+            "title": "Unknown",
+            "artist": "Unknown",
+            "bpm": 0.0,
+            "key": "Unknown",
+            "duration_sec": 0.0,
+            "sample_rate": 0,
+        },
+        "hot_cues": [],
+        "beatgrid": {"markers": {"non_terminal": [], "terminal": None}},
+    }
+
+    try:
+        audio = AIFF(file_path)
+    except Exception as e:
+        logging.error("Failed to open AIFF file '%s': %s", file_path, e)
+        return results
+
+    # Audio info
+    if audio.info:
+        results["metadata"]["duration_sec"] = round(audio.info.length, 3)
+        results["metadata"]["sample_rate"] = audio.info.sample_rate or 0
+
+    # Serato writes ID3 tags to AIFF — same structure as WAV
+    tags = audio.tags
+    if not tags:
+        # Try loading as generic mutagen file for ID3
+        try:
+            tagfile = mutagen.File(file_path)
+            if tagfile and tagfile.tags:
+                tags = tagfile.tags
+        except Exception:
+            pass
+
+    if tags:
+        # Standard metadata
+        tit2 = tags.get("TIT2")
+        if tit2 and hasattr(tit2, "text") and tit2.text:
+            results["metadata"]["title"] = str(tit2.text[0]).strip()
+
+        tpe1 = tags.get("TPE1")
+        if tpe1 and hasattr(tpe1, "text") and tpe1.text:
+            results["metadata"]["artist"] = str(tpe1.text[0]).strip()
+
+        tbpm = tags.get("TBPM")
+        if tbpm and hasattr(tbpm, "text") and tbpm.text:
+            try:
+                results["metadata"]["bpm"] = float(str(tbpm.text[0]).strip())
+            except (ValueError, TypeError):
+                pass
+
+        tkey = tags.get("TKEY", tags.get("initialkey"))
+        if tkey and hasattr(tkey, "text") and tkey.text:
+            results["metadata"]["key"] = convert_key_to_camelot(str(tkey.text[0]).strip())
+
+        # Hot cues and beatgrid from GEOB frames
+        from mutagen.id3 import GEOB
+
+        for tag in tags.values():
+            if isinstance(tag, GEOB):
+                desc = getattr(tag, "desc", "")
+                if desc == "Serato Markers2":
+                    try:
+                        results["hot_cues"] = parse_serato_hot_cues(tag.data)
+                    except Exception as e:
+                        logging.warning("Error reading hot cues from '%s': %s", file_path, e)
+                elif desc == "Serato BeatGrid":
+                    try:
+                        results["beatgrid"] = parse_beatgrid(tag.data)
+                    except Exception as e:
+                        logging.warning("Error reading beatgrid from '%s': %s", file_path, e)
+
+    return results
+
+
+def parse_serato_hot_cues(base64_data: bytes) -> list:
+    """Parse Serato Markers2 GEOB payload (shared with MP3/WAV/FLAC)."""
+    import base64
+    import re
+
+    if isinstance(base64_data, str):
+        base64_data = base64_data.encode("utf-8")
+
+    clean_data = re.sub(rb"[^a-zA-Z0-9+/=]", b"", base64_data)
+    padding_needed = 4 - (len(clean_data) % 4)
+    if padding_needed != 4:
+        clean_data += b"=" * padding_needed
+
+    try:
+        data = base64.b64decode(clean_data)
+    except Exception:
+        return []
+
+    hot_cues = []
+    idx = 0
+    total = len(data)
+
+    while idx < total:
+        nxt = data[idx:].find(b"\x00")
+        if nxt == -1:
+            break
+
+        entry_type = data[idx : idx + nxt].decode("utf-8", errors="replace")
+        idx += nxt + 1
+
+        if idx + 4 > total:
+            break
+
+        entry_len = struct.unpack(">I", data[idx : idx + 4])[0]
+        idx += 4
+
+        if entry_type == "CUE" and idx + entry_len <= total:
+            raw = data[idx : idx + entry_len]
+            if len(raw) >= 12:
+                cue_index = raw[1]
+                position_ms = struct.unpack(">I", raw[2:6])[0]
+                r, g, b = raw[7:10]
+                color = "#{:02X}{:02X}{:02X}".format(r, g, b)
+                label = raw[12:].rstrip(b"\x00").decode("utf-8", errors="replace")
+
+                hot_cues.append({
+                    "index": cue_index,
+                    "position_ms": position_ms,
+                    "color": color,
+                    "name": label,
+                })
+
+        idx += entry_len
+
+    return hot_cues
+
+
+def parse_beatgrid(raw_data: bytes) -> dict:
+    """Parse Serato BeatGrid binary struct from a GEOB frame."""
+    result = {"markers": {"non_terminal": [], "terminal": None}}
+
+    fp = io.BytesIO(raw_data)
+    version = struct.unpack("BB", fp.read(2))
+    if version != (0x01, 0x00):
+        logging.warning("Unsupported beatgrid version: %s", version)
+
+    num_markers = struct.unpack(">I", fp.read(4))[0]
+    markers = []
+
+    for i in range(num_markers):
+        pos = struct.unpack(">f", fp.read(4))[0]
+        data = fp.read(4)
+        if i == num_markers - 1:
+            bpm = struct.unpack(">f", data)[0]
+            markers.append(("terminal", pos, bpm))
+        else:
+            beats = struct.unpack(">I", data)[0]
+            markers.append(("non_terminal", pos, beats))
+
+    fp.read(1)  # trailing byte
+
+    if not markers:
+        return result
+
+    non_terminal = []
+    terminal = None
+    for m in markers:
+        if m[0] == "terminal":
+            terminal = {"position": m[1], "bpm": m[2]}
+        else:
+            non_terminal.append({"position": m[1], "beats_till_next_marker": m[2]})
+
+    result["markers"]["non_terminal"] = non_terminal
+    result["markers"]["terminal"] = terminal
+    return result