diff --git a/docs/contracts.md b/docs/contracts.md index 7c63b3c..d23c9e9 100644 --- a/docs/contracts.md +++ b/docs/contracts.md @@ -229,7 +229,7 @@ Documents written by particular stages (artifacts of no object: their ids are `< | `nnnotes.catalog-index/1` | `catalog.index` | [catalog-index.schema.json](https://github.com/MetaSekaiLab/nnnotes/blob/main/src/nnnotes/schemas/catalog-index.schema.json) | | `nnnotes.census/2` | `unity.census` | [census.schema.json](https://github.com/MetaSekaiLab/nnnotes/blob/main/src/nnnotes/schemas/census.schema.json) | | `nnnotes.scripts/1` | `link.scripts` | [scripts.schema.json](https://github.com/MetaSekaiLab/nnnotes/blob/main/src/nnnotes/schemas/scripts.schema.json) | -| `nnnotes.addresses/2` | `link.addresses` | [addresses.schema.json](https://github.com/MetaSekaiLab/nnnotes/blob/main/src/nnnotes/schemas/addresses.schema.json) | +| `nnnotes.addresses/3` | `link.addresses` | [addresses.schema.json](https://github.com/MetaSekaiLab/nnnotes/blob/main/src/nnnotes/schemas/addresses.schema.json) | | `nnnotes.view/1` | `view.` | [view.schema.json](https://github.com/MetaSekaiLab/nnnotes/blob/main/src/nnnotes/schemas/view.schema.json) | | `nnnotes.objects/1` | `pipeline.assemble` (a product's object table) | [objects.schema.json](https://github.com/MetaSekaiLab/nnnotes/blob/main/src/nnnotes/schemas/objects.schema.json) | diff --git a/docs/products.md b/docs/products.md index 366e81d..25e114a 100644 --- a/docs/products.md +++ b/docs/products.md @@ -14,12 +14,22 @@ objects: `link.addresses`; - the objects of the Resources.Load paths it gives (`resources`): paths of the ResourceManager in the APK's built-in files, such as the materials a runtime loads by name, through the same table and compared ignoring case; +- the objects of the built-in selectors it gives (`builtins`): objects of the APK's built-in files by class + (`GraphicsSettings`) or a MonoBehaviour by its script's class (`MonoBehaviour:`), such as settings assets + the game reads without a path, through the same table; - the object ids the stage names besides (`root_objects`). +A product stage may also read cue sheets: the catalog keys `cue_sheets` gives (each also one of its keys). A cue +sheet is the ACB its key loads, held in a bundle (the `acb` file of an object of the key) or a raw file of the +catalog that `cri.audio` decoded. The task's inputs then hold each ACB (`acb:`) and, as `sheets`, the files +`cri.audio` made of it (`{key: {acb, status, reason, files: {name: {sha256, size, ext}}}}`); it depends on those +`cri.audio` tasks, and a product file may be one of those files as stored, without copying its bytes. A stage that +reads files of the APK names them in `APK`: inputs `apk:`, from the fact `apk`. + All are read from the planning facts alone, so a command can ask for what its products need before anything runs: -`assemble.request(stages, products, env)` gives the fact `roots` (`{"keys", "resources", "objects"}`), and the request of -`unity.export` then holds the units of those objects and every unit reachable from them through the census `refs` -([units](stages.md#units)), and nothing else of their bundles. +`assemble.request(stages, products, env)` gives the fact `roots` (`{"keys", "resources", "builtins", "objects"}`), and +the request of `unity.export` then holds the units of those objects and every unit reachable from them through the +census `refs` ([units](stages.md#units)), and nothing else of their bundles. What a task reaches: the units holding its roots, then every unit that an item of a reached unit references (the item's `refs`), through the `unity.export` results. A sprite whose atlas is in another bundle has its `image` from the @@ -29,7 +39,7 @@ object that a reached object references and no census holds is listed as `missin | Part of the task | | |---|---| | inputs | `objects`: the object table of what it reaches (`nnnotes.objects/1`); the stage's own inputs (master tables, data its parameters name) | -| depends | `link.addresses` (when it has keys or resource paths), the `unity.export` and `sprite.crop` tasks whose results it read | +| depends | `link.addresses` (when it has keys, resource paths or built-in selectors), the `unity.export` and `sprite.crop` tasks whose results it read, the `cri.audio` tasks of its cue sheets | | artifacts | `#`: the product's own files; `#objects`: the object table of the objects its files use (the whole table unless the stage narrows it to some objects, or to some files of an object: a shader's programs for one graphics API) | | item | one for the subject: `exported`, or `unsupported` with a reason code | @@ -190,3 +200,47 @@ null `focusWorld`. | `spine/.json` or `.skel` | the bytes of the SkeletonDataAsset's `skeletonJSON` (JSON when they start with `{`) | | `spine/.atlas` | the bytes of each SpineAtlasAsset's `atlasFile`, as stored (an atlas two skeletons share is written once) | | `spine/` | for each page of the atlas, the PNG of the `_MainTex` of the atlas asset's material of the same index | + +## Live sounds + +What a live plays besides its chart is two products of `nnnotes.pipeline.live`, each shared by every live that plays +it: the BGM of a music (`live.music`) and the sounds of an option set (`live.sounds`). Their roots are the objects of +the keys of their cue sheets, they read the sheets' `cri.audio` files, and neither keeps objects. A sound entry, of +either, is + +| Field | | +|---|---| +| `row` | the `MasterSound` row | +| `sheet`, `cue` | its cue sheet and cue name | +| `category`, `categories` | the row's category and the CRI categories of the cue (ACB command 65) | +| `volume`, `busSends` | the cue's volume (with the sheet's) and bus sends (ACB commands 146 and 111) | +| `lengthMs`, `other` | the cue's length, its other commands as stored | +| `layers` | one per track of the cue: `file` (`audio//`), the stream's `stream`, `awbId`, `sampleRate`, `channels`, `samples`, `loopStart`, `loopEnd`, the waveform's `loopFlag`, the track's and synth's `volume` and `busSends`, `other` | + +`live.music`: its subject is a music id (`MasterLiveMusic._id`), its manifest `music/.json` and its input +`master` the music's rows of `MasterLiveMusic`, `MasterSound` and `MasterSoundCueSheet`. Its files are `music.json` +(`format` 1, `musicId`, `music`: the BGM's `soundId`, `volume` and `startTimeSec`, and `sounds`: the BGM's entry) +and each layer's file in the web format of the parameter `format`: `aac` (AAC-LC 160 kbit/s in MP4, the default), +`opus`, `vorbis`, `mp3`, or `flac`, the decoded FLAC itself. An AAC layer records `encoderDelay`, the priming samples +of the MP4 edit list, which a decoder that does not apply the edit list outputs first. The encode is bit-exact; the +atom `audio.encode` names the ffmpeg build. + +`live.sounds`: its subject names the note sound sets offered besides the default, `default` or +`NoteSePatternId=,,...` (the default set among them), and its manifest is `sounds/.json`. Its roots +include the game's sound settings, the `SoundVolumeSettings` and `AppConfigDefaultData` MonoBehaviours of the APK's +built-in files, and it reads the ACF of the APK (`assets/Cri/Sound/Sirius.acf`). Its input `master` holds the +option defaults of preset 1 (`MasterOptionDefault`), `MasterLiveNoteSe`, `MasterLiveSe` and the `MasterSound` and +`MasterSoundCueSheet` rows of its sounds. Its files are each sound's decoded FLAC and `sounds.json`: + +| Field of sounds.json | | +|---|---| +| `format` | 1 | +| `sounds` | the entry of each sound by `MasterSound` id: the note sounds of the default options and of every set offered, and the live sounds of the start and finish cheers and of the finish direction of every result | +| `categories` | the CRI category volumes of a fresh profile: each category's default volume, and the live categories' config volumes | +| `acf`, `react` | the ACF's `name`, `buses` and `categories`; its REACT (ducking) entries | +| `noteSe` | the note sounds of option preset 1: `types` (note sound type -> sound id), `volumes`, `mutes`, `patternId`, `useIndividualNoteSe`, `gekisouTraceVolume`; with sets offered `groups` (set id -> type -> sound id) | +| `liveSe`, `liveSeNames` | live sound type -> sound id, and the types' names | +| `timeline` | the intro and end timings of the cheers and the start voice | + +The `product` layout makes the `live.music` products of the musics whose BGM key the selection holds, and the +default `live.sounds` when the selection holds every cue sheet of its sounds; both need the master data. diff --git a/docs/schemas.md b/docs/schemas.md index 6ffa241..fa3c1f9 100644 --- a/docs/schemas.md +++ b/docs/schemas.md @@ -14,7 +14,7 @@ schemas.load("plan") # the schema as a dict | Schema | Format | Document | |---|---|---| -| `addresses` | `nnnotes.addresses/2` | the address table of a catalog version and the censuses of its bundles | +| `addresses` | `nnnotes.addresses/3` | the address table of a catalog version and the censuses of its bundles | | `artifact` | `nnnotes.artifact/1` | the content, provenance and semantics of one output file | | `catalog-diff` | `nnnotes.catalog-diff/1` | the difference between two catalog versions (`catalog diff`) | | `catalog-index` | `nnnotes.catalog-index/1` | the index of a catalog pair: remote and APK catalogs, every location in catalog order | diff --git a/docs/stages.md b/docs/stages.md index 6e26c6f..3415322 100644 --- a/docs/stages.md +++ b/docs/stages.md @@ -13,8 +13,8 @@ The asset export is built from three layers: ## Stages Every stage is at version 1, except `unity.census` (version 4: it lists what every object references, the -hierarchy holding it and the Resources.Load paths of a ResourceManager), `link.addresses` (version 2: the -Resources.Load paths), `live2d.model` (version 2: assembled from the exported objects), `unity.export` (version 2: one task per unit), `cri.movie` (version 2: every stream kind, +hierarchy holding it and the Resources.Load paths of a ResourceManager), `link.addresses` (version 3: the +Resources.Load paths and the built-in selectors), `live2d.model` (version 2: assembled from the exported objects), `unity.export` (version 2: one task per unit), `cri.movie` (version 2: every stream kind, codec and channel of a USM) and `catalog.index` (version 2: gzip input and Japanese remote asset placeholders). | Stage | Subject | Inputs | Outputs | @@ -29,7 +29,9 @@ codec and channel of a USM) and `catalog.index` (version 2: gzip input and Japan | `live2d.model`, `live2d.cubism` | a Live2D model key (`Character/Live2D///model/`) a command asks for | `objects`: the object table of what the model reaches ([products](products.md#live2d-models)); depends on `link.addresses` and the `unity.export` tasks it reads | `model.json` (the player's model) and the object table; the Cubism SDK files | | `room.gltf` | a spot background key (`Spot//Background/`) | `objects`: the object table of what the background reaches ([products](products.md#spots)) | `room.glb` and `room.json` | | `spot.files` | a spot situation key (`Spot///`) | `objects`; `master`: the spot's master record | `spot.json` and the Spine files | -| `link.addresses` | global | catalog index, censuses (the built-in files' too) | key -> objects, Resources.Load path -> objects, serialized file -> bundle or built-in file, bundle -> keys | +| `live.music` | a music id (`MasterLiveMusic`) | `objects`; `acb:` and `sheets`: the BGM's cue sheet and its `cri.audio` files ([products](products.md#live-sounds)); `master`: the music's master record; parameter `format` (`aac`) | `music.json` and the BGM in the web format | +| `live.sounds` | the note sound sets offered: `default` or `NoteSePatternId=` | `objects` (the sound settings of the APK's built-in files); `acb:` and `sheets` of each cue sheet; `apk:assets/Cri/Sound/Sirius.acf`; `master` | `sounds.json` and the decoded sounds | +| `link.addresses` | global | catalog index, censuses (the built-in files' too) | key -> objects, Resources.Load path -> objects, built-in selector (class, MonoBehaviour script class) -> objects, serialized file -> bundle or built-in file, bundle -> keys | | `view.` | a catalog (`main`) | `master:` per master table it reads, `rules` (the view's rule and the resolvers it uses), `addresses` (the part of the address table it read: the keys looked up and the prefixes listed); no parameters, atoms or context | `view.:#view` (facts: rows, entries, gaps, unreferenced) | | `view.voices` | a catalog (`main`) | `master:
` per master table its rules read, `rules`, `addresses`, `sheets` (each cue sheet the `cri.audio` results are known by that a voice names or the rules' prefixes list: its task and ACB content id), `acb:` (the ACB of each decoded one), `episodes` (per story episode and part, the `unity.export` artifact holding the object its key names), `adv:` (those contents); context `audio` (the keys of those `cri.audio` results); runs after `unity.export` and `cri.audio` | `view.voices:#view` (facts: rows, entries, gaps, unreferenced streams) | @@ -112,7 +114,7 @@ several serialized files the file's tag follows the bundle name, as in `link.add bundle's content changes, so a unit keeps its task across catalog versions. A run exports the units holding an object of the selected bundles, the units of the objects the fact "roots" names -(catalog keys, Resources.Load paths and object ids: a product's roots, [products](products.md)) and every unit reachable +(catalog keys, Resources.Load paths, built-in selectors and object ids: a product's roots, [products](products.md)) and every unit reachable from those through what their objects reference (the census `refs`, across the censuses of the run); with `--only-class` only the units whose root object is of those classes. The selection is not part of any key: a unit's task is the same whichever request reaches it. @@ -174,13 +176,13 @@ whose atlas is in its own bundle. `cri.audio` and `cri.movie` take their subjects by content: a task is `cri.audio:` or `cri.movie:`, so an ACB or a USM stored both as a raw file and in a bundle, or in several bundles, -is decoded once. Raw files are told apart by their first bytes (`@UTF` ACB, `AFS2` AWB, `CRID` USM). Both read the -HCA keycode from the `boot` input when the task runs; the key itself is never part of a task, a record or a log. -Each task has one item (object: the task id; class `ACB` or `USM`): `exported`, or `unsupported` with a reason code. -Their artifacts belong to no object; the `original` layout places them under each name the stage gives the content: -the catalog keys that load a raw file, `Cri/Sound/` for a cue sheet held in a bundle (the sheet's name -when the catalog has no such key), the keys of its bundle that end in the movie asset's name for a movie held in a -bundle. +is decoded once. Raw files are told apart by their first bytes (`@UTF` ACB, `AFS2` AWB, `CRID` USM), read once per +content and recorded in the depot, so a later plan reads no raw file it has seen. Both read the HCA keycode from the +`boot` input when the task runs; the key itself is never part of a task, a record or a log. Each task has one item +(object: the task id; class `ACB` or `USM`): `exported`, or `unsupported` with a reason code. Their artifacts belong +to no object; the `original` layout places them under each name the stage gives the content: the catalog keys that +load a raw file, `Cri/Sound/` for a cue sheet held in a bundle (the sheet's name when the catalog has no +such key), the keys of its bundle that end in the movie asset's name for a movie held in a bundle. | Stage | Parameters | Atoms | Artifacts (`#`) | |---|---|---|---| diff --git a/src/nnnotes/cli/assets.py b/src/nnnotes/cli/assets.py index ae71088..28c40a9 100644 --- a/src/nnnotes/cli/assets.py +++ b/src/nnnotes/cli/assets.py @@ -31,6 +31,7 @@ products {product stage: subjects} (with a product stage in the pipeline: the `product` layout makes each stage's products of the keys the selected bundles hold, held_keys; `export live2d` asks for one) roots the root objects of those products (pipeline.assemble.request) + apk {path in the APK: Input "apk:"} of the APK files the product stages read (their `APK`) Selection (--select, repeatable; the union; none: everything): `group:G` the bundles of an Addressables group (the stable name up to `_assets_`, else up to its first `_`), `key:PREFIX` the bundles and raw files the catalog keys @@ -119,6 +120,8 @@ def matches(self, stage: str) -> bool: StageSource("live2d.cubism", "nnnotes.pipeline.live2d:CubismStage", output=True, product=True), StageSource("room.gltf", "nnnotes.pipeline.spot:RoomStage", output=True, product=True), StageSource("spot.files", "nnnotes.pipeline.spot:SpotStage", output=True, product=True), + StageSource("live.music", "nnnotes.pipeline.live:MusicStage", output=True, product=True), + StageSource("live.sounds", "nnnotes.pipeline.live:SoundsStage", output=True, product=True), StageSource("view.", "nnnotes.pipeline.views:stages", output=True), StageSource("view.voices", "nnnotes.pipeline.voices:VoiceStage", output=True), ) @@ -745,11 +748,15 @@ def facts(self, pipeline: list[str], fetch: bool, background: bool = False) -> d else: # without master data, the products that read it have no subjects (as the views) held = held_keys(index, sel.selected) chosen = { - n: [] if n in readers and facts["master"] is None else self.installed.stages[n].select(held) + n: [] + if n in readers and facts["master"] is None + else self.installed.stages[n].select(held, Env(self.depot, facts)) for n in products } facts["products"] = chosen facts["roots"] = assemble.request(self.installed.stages, chosen, Env(self.depot, facts)) + members = {m for n in products if chosen[n] for m in self.installed.stages[n].APK} + facts["apk"] = builtins.apk_files(self.depot, self.apk, members) if self.products is not None or not set(self.layouts) & set(OBJECT_LAYOUTS): facts["selected"] = [] # products alone: what they reach if any(n in pipeline for n in RAW_STAGES): diff --git a/src/nnnotes/cli/site.py b/src/nnnotes/cli/site.py index 6b9629a..1849734 100644 --- a/src/nnnotes/cli/site.py +++ b/src/nnnotes/cli/site.py @@ -5,8 +5,8 @@ import sys from pathlib import Path +from ..formats.encoders import DEFAULT_AUDIO_FORMAT, WEB_AUDIO from ..site.compress import DEFAULT_ENCODING, ENCODINGS -from ..site.webaudio import DEFAULT_AUDIO_FORMAT, WEB_AUDIO from ..source.config import ConfigError, use from ..source.session import Session from .common import ( diff --git a/src/nnnotes/content/liveaudio.py b/src/nnnotes/content/liveaudio.py index 07e7e32..b491023 100644 --- a/src/nnnotes/content/liveaudio.py +++ b/src/nnnotes/content/liveaudio.py @@ -1,47 +1,78 @@ -"""Live sounds for one music: SE / cheer / voice cue sheets, their CRI cue data and the live-audio index. +"""Live sounds: the cue structure of the live's cue sheets, the game's sound settings and the live-audio documents. -`extract(cat, master, player, music_id, out_dir)` decodes every cue sheet the live plays in the preview slice -(auto play at Perfect, fresh profile, default options) with `cri.decode` into `/audio//` -and writes `/audio/live-audio.json`: +What a live plays besides its chart, from the master data (MasterSound, MasterSoundCueSheet, MasterLiveMusic, +MasterLiveNoteSe, MasterLiveSe, MasterOptionDefault), the cue sheets' ACBs and their decoded streams (cri.decode or +the cri.audio stage), the ACF of the APK (ACF_IN_APK) and two MonoBehaviours of the APK's built-in files +(SETTINGS_SCRIPTS). A sound entry (sound_entry): - sounds MasterSound id -> {row, sheet, cue, categories, volume, busSends, layers[{file, sampleRate, samples, - loopStart, loopEnd, loopFlag, volume, busSends}], lengthMs}; the cue structure (sequence -> tracks -> - synth -> waveform) is read from the ACB (nnnotes.formats.acb; command 65 categories, 146 volume, 111 bus - send) + {row, sheet, cue, category, categories, volume, busSends, lengthMs, other, layers[{file, stream, awbId, + sampleRate, channels, samples, loopStart, loopEnd, loopFlag, volume, busSends, other}]} + +the cue structure (sequence -> tracks -> synth -> waveform) read from the ACB (acb_cues: command 65 categories, 146 +volume, 111 bus send), each layer the decoded stream `audio//`. + +music_doc (the live.music product, format MUSIC_FORMAT): `music` {soundId, volume, startTimeSec} the BGM sound, and +`sounds` {its id: its entry}, each layer in the product's audio format. + +sounds_doc (the live.sounds product, format SOUNDS_FORMAT), the sounds of the default options (auto play at Perfect, +fresh profile) and of the note sound sets offered: + + sounds MasterSound id -> entry: the note SE of the default preset and of every offered set, and the live + SE of LIVE_SE_SOUNDS (the start and finish cheers and the finish direction of every result: the one a + live ends with depends on how it is played) categories CRI category volumes for a fresh profile (SoundVolumeSettings x AppConfigDefaultData) - react the ACF's REACT (ducking) entries between live categories - music the BGM sound id and PlayMusic volume + acf, react the ACF's buses and categories, its REACT (ducking) entries between live categories noteSe LiveNoteSeType -> sound id / volume / mute for the default preset (LiveSettingCreator.CreateSESettings); - with liveoptions.LiveOptions offering note sound sets also `groups`: set id -> LiveNoteSeType -> - sound id (MasterLiveNoteSe), every offered set's sounds decoded as well - liveSe LiveSeType -> sound id (MasterLiveSe); `sounds` has those of LIVE_SE_SOUNDS: the start and finish - cheers and the finish direction of every result (the one a live ends with depends on how it is played) + with sets offered also `groups`: set id -> LiveNoteSeType -> sound id (MasterLiveNoteSe) + liveSe LiveSeType -> sound id (MasterLiveSe) timeline intro / end timings the runtime needs - voice the start / finish character voice rule and the voice character the caller chose (none by default) -Cue sheets come through `cri` (any of its layouts; the note SE / live SE / voice sheets are embedded ACBs); -each cue layer maps to its vgmstream stream by `streams.json` (memory AWB order). A sheet already decoded into -`/audio//` (the BGM by score.py) is reused. +A live.sounds subject names the sets offered: `default`, or `NoteSePatternId=,,...` (sounds_subject). + +extract() writes the same for one music into a live directory (audio/live-audio.json: the fields of both documents +in one, the BGM sound among the sounds) from a Catalog, decoding each cue sheet with cri.decode into +`/audio//` (a sheet already decoded there, the BGM by score.py, is reused). """ from __future__ import annotations import struct from pathlib import Path +from typing import TYPE_CHECKING from ..core.jsonio import write_json -from ..formats import cri from ..formats.acb import commands as _commands from ..formats.acb import tables as _tables from ..formats.acb import u16s as _u16s from ..formats.encoders import FLAC_LEVEL from ..source.apkset import ApkSet -from ..source.catalog import Catalog -from ..unity.player import PlayerData -from .liveoptions import LiveOptions -from .score import master_table + +if TYPE_CHECKING: + from ..source.catalog import Catalog + from ..unity.player import PlayerData + from .liveoptions import LiveOptions ACF_IN_APK = "assets/Cri/Sound/Sirius.acf" +MUSIC_FORMAT, SOUNDS_FORMAT = 1, 1 # the "format" of music_doc / sounds_doc +MUSIC_TABLES = ("MasterLiveMusic", "MasterSound", "MasterSoundCueSheet") +SOUND_TABLES = ("MasterOptionDefault", "MasterLiveNoteSe", "MasterLiveSe", "MasterSound", "MasterSoundCueSheet") +# the MonoBehaviours of the sound settings (script classes; the APK's built-in files hold one of each) +SETTINGS_SCRIPTS = ("SoundVolumeSettings", "AppConfigDefaultData") +# the AppConfigDefaultData field of each live category's volume (AppConfig.UpdateLiveSoundVolumeConfig) +LIVE_VOLUMES = { + "LiveBgm": "LiveBgmSoundVolume", + "LiveSe": "LiveSeSoundVolume", + "LiveVoice": "LiveVoiceSoundVolume", + "LiveNotesSe": "LiveNotesSeSoundVolume", +} +SE_PATTERN = "NoteSePatternId" # the option of the note sound sets (liveoptions) +# intro / end timings of the live's sounds +TIMELINE = { + "startCheerDelaySec": 1.0, + "playVoiceSignalSec": 1.483333, + "startCheerStopFadeSec": 1.0, + "finishCheerStopFadeSec": 1.0, +} # App.Master.LiveNoteSeType / LiveSeType NOTE_SE_TYPES = { @@ -201,10 +232,9 @@ def cmd(table, index): return out -def acf_info(apk: Path) -> dict: - """Categories, buses and REACT entries of the game's ACF (`assets/Cri/Sound/Sirius.acf` in the APK).""" - with ApkSet(apk) as z: - top, T = _tables(z.read(ACF_IN_APK)) +def acf_info(acf: bytes) -> dict: + """Categories, buses and REACT entries of the game's ACF (ACF_IN_APK).""" + top, T = _tables(acf) cat_names = {r["Index"]: r["Name"] for r in T["CategoryNameTable"]} cats = {} for r in T["CategoryTable"]: @@ -234,40 +264,36 @@ def acf_info(apk: Path) -> dict: # --------------------------------------------------------------------------- game settings -def _player_mono(player: PlayerData, cls: str) -> dict: - hits = [] - for o in player.env.objects: - if o.type.name != "MonoBehaviour": - continue - try: - if player.script(o)[2] != cls: - continue - except Exception: # MonoBehaviours without a resolvable script are not ours - continue - hits.append(player.mono(o)) - if len(hits) != 1: - raise RuntimeError(f"{cls}: {len(hits)} objects in the player data") - return hits[0] - - -def category_volumes(player: PlayerData) -> dict: - """Fresh-profile CRI category volumes: `` = SoundVolumeSettings.DefaultVolume x 1.0 - (SoundManager.Boot ChangeVolume("All", 1)); `LiveConfig` = LocalCacheData live volume = AppConfigDefaultData +def category_volumes(svs: dict, acd: dict) -> dict: + """Fresh-profile CRI category volumes from the SoundVolumeSettings `svs` and AppConfigDefaultData `acd` + MonoBehaviours (SETTINGS_SCRIPTS): `` = SoundVolumeSettings.DefaultVolume x 1.0 (SoundManager.Boot + ChangeVolume("All", 1)); `LiveConfig` = LocalCacheData live volume = AppConfigDefaultData (AppConfig.UpdateLiveSoundVolumeConfig); Bgm/Se/VoiceConfig = system option 400-402 / 100 (preset 1: 1.0).""" - svs = {e["CategoryName"]: e["DefaultVolume"] for e in _player_mono(player, "SoundVolumeSettings")["_collection"]} - acd = _player_mono(player, "AppConfigDefaultData")["_defaultData"] - live = { - "LiveBgm": "LiveBgmSoundVolume", - "LiveSe": "LiveSeSoundVolume", - "LiveVoice": "LiveVoiceSoundVolume", - "LiveNotesSe": "LiveNotesSeSoundVolume", - } - out = dict(svs) - for cat, field in live.items(): - out[cat + "Config"] = acd[f"<{field}>k__BackingField"] + out = {e["CategoryName"]: e["DefaultVolume"] for e in svs["_collection"]} + defaults = acd["_defaultData"] + for cat, field in LIVE_VOLUMES.items(): + out[cat + "Config"] = defaults[f"<{field}>k__BackingField"] return out +def sound_settings(objects) -> tuple[dict, dict]: + """The SoundVolumeSettings and AppConfigDefaultData MonoBehaviours (SETTINGS_SCRIPTS) among the root objects of a + product's object table, by their script's class (exactly one of each).""" + found: dict[str, list] = {c: [] for c in SETTINGS_SCRIPTS} + for oid in objects.roots: + if oid not in objects or objects.info(oid).get("class") != "MonoBehaviour": + continue + doc = objects.json(oid) + cls = str(doc.get("$script") or "").rsplit("|", 1)[-1] + if cls in found: + found[cls].append(doc) + for cls, docs in found.items(): + if len(docs) != 1: + raise ValueError(f"{cls}: {len(docs)} objects among the product's roots") + svs, acd = (found[c][0] for c in SETTINGS_SCRIPTS) + return svs, acd + + def _option(rows: list[dict], preset: int, item: int) -> str: hit = [r["_valueString"] for r in rows if r["_presetId"] == preset and r["_optionItemType"] == item] if len(hit) != 1: @@ -275,10 +301,10 @@ def _option(rows: list[dict], preset: int, item: int) -> str: return hit[0] -def note_se_groups(master: Path) -> dict[int, dict[int, int]]: - """MasterLiveNoteSe: sound set (_groupID) -> LiveNoteSeType -> sound id.""" +def note_se_groups(rows: list[dict]) -> dict[int, dict[int, int]]: + """MasterLiveNoteSe rows: sound set (_groupID) -> LiveNoteSeType -> sound id.""" out: dict[int, dict[int, int]] = {} - for r in master_table(master, "MasterLiveNoteSe"): + for r in rows: out.setdefault(r["_groupID"], {})[r["_liveNoteSeType"]] = r["_seId"] return out @@ -291,13 +317,14 @@ def _volume(opt: list[dict], preset: int, item: int) -> float: return min(max(int(_option(opt, preset, item)) / 100.0, 0.0), 1.0) -def note_se_settings(master: Path, preset: int = 1, groups: tuple = ()) -> dict: - """LiveSettingCreator.CreateSESettings for a fresh profile (option preset `preset`): pattern mode takes the - MasterLiveNoteSe rows of the set NoteSePatternId (420); UseIndividualNoteSe (421) takes, per note SE type, the - row of the set its sound item names (BuildIndividualNoteSeDictionary; a type whose set has no row for it has no - entry). `groups`: sound set ids whose tables are written as `groups` as well (the sets a viewer may choose).""" - opt = master_table(master, "MasterOptionDefault") - table = note_se_groups(master) +def note_se_settings(tables: dict, preset: int = 1, groups: tuple = ()) -> dict: + """LiveSettingCreator.CreateSESettings for a fresh profile (option preset `preset`) from the master tables + `tables` {name: rows} (MasterOptionDefault, MasterLiveNoteSe): pattern mode takes the MasterLiveNoteSe rows of the + set NoteSePatternId (420); UseIndividualNoteSe (421) takes, per note SE type, the row of the set its sound item + names (BuildIndividualNoteSeDictionary; a type whose set has no row for it has no entry). `groups`: sound set ids + whose tables are written as `groups` as well (the sets a viewer may choose).""" + opt = tables["MasterOptionDefault"] + table = note_se_groups(tables["MasterLiveNoteSe"]) group = int(_option(opt, preset, 420)) individual = _option(opt, preset, 421).upper() == "TRUE" if individual: @@ -332,26 +359,77 @@ def note_se_settings(master: Path, preset: int = 1, groups: tuple = ()) -> dict: return out -# --------------------------------------------------------------------------- extraction -def _decode(cat: Catalog, sheet: str, out_dir: Path, fmt: str, flac_level: int) -> list[dict]: - """Decode a cue sheet into /audio// (reusing an existing decode) and return its stream list.""" - d = out_dir / "audio" / sheet - if not (d / "streams.json").exists(): - cri.decode(cat, sheet, d, fmt=fmt, flac_level=flac_level) - import json +# --------------------------------------------------------------------------- sounds +def sheet_key(sheet: str) -> str: + """The catalog key of a cue sheet.""" + return f"Cri/Sound/{sheet}" - return json.loads((d / "streams.json").read_text(encoding="utf-8")) +def music_sound(tables: dict, music_id: int) -> tuple[dict, dict]: + """(MasterSound row, MasterSoundCueSheet row) of a music's BGM (MasterLiveMusic._musicSoundID).""" + music = [r for r in tables["MasterLiveMusic"] if r["_id"] == music_id] + if len(music) != 1: + raise KeyError(f"MasterLiveMusic {music_id}") + sound = _row(tables, "MasterSound", music[0]["_musicSoundID"]) + return sound, _row(tables, "MasterSoundCueSheet", sound["_soundCueSheetID"]) + + +def _row(tables: dict, table: str, row_id: int) -> dict: + hit = [r for r in tables[table] if r["_id"] == row_id] + if len(hit) != 1: + raise KeyError(f"{table} {row_id}: {len(hit)} rows") + return hit[0] -def _sound_entry(row: dict, sheet: str, cue: dict, streams: list[dict], cue_name: str) -> dict: + +def live_sounds(tables: dict, groups: tuple = ()) -> dict: + """The live sounds of the default options with the note sound sets `groups` offered as well: {"noteSe": + note_se_settings, "liveSe": {LiveSeType: sound id}, "ids": the sound ids a live plays (the default note SE, the + live SE of LIVE_SE_SOUNDS, then the note SE of the other offered sets)}.""" + nse = note_se_settings(tables, groups=groups) + live_se = {r["_liveSeType"]: r["_seId"] for r in tables["MasterLiveSe"]} + ids = sorted(set(nse["types"].values())) + [live_se[t] for t in LIVE_SE_SOUNDS] + for se in sorted({v for g in nse.get("groups", {}).values() for v in g.values()}): + if se not in ids: + ids.append(se) + return {"noteSe": nse, "liveSe": live_se, "ids": ids} + + +def sheets_of(tables: dict, ids) -> dict[str, list[int]]: + """{cue sheet name: sound ids} of MasterSound ids, in the order of the ids.""" + sheets = {r["_id"]: r["_cueSheetName"] for r in tables["MasterSoundCueSheet"]} + out: dict[str, list[int]] = {} + for sid in ids: + out.setdefault(sheets[_row(tables, "MasterSound", sid)["_soundCueSheetID"]], []).append(sid) + return out + + +def sound_record(tables: dict, ids) -> dict: + """The MasterSound and MasterSoundCueSheet rows of sound ids (sorted by id): what sound_entry reads of them.""" + sounds = sorted((_row(tables, "MasterSound", i) for i in set(ids)), key=lambda r: r["_id"]) + sheets = {r["_soundCueSheetID"] for r in sounds} + return { + "MasterSound": sounds, + "MasterSoundCueSheet": sorted((_row(tables, "MasterSoundCueSheet", i) for i in sheets), key=lambda r: r["_id"]), + } + + +def sound_entry(row: dict, sheet: str, cues: dict, streams: list[dict], files=None) -> dict: + """The live-audio entry of a MasterSound row whose cue is in the sheet `sheet` (`cues`: acb_cues of its ACB, + `streams`: its decoded streams, streams.json): its categories, volume, bus sends and one layer per track with + the decoded stream (vgmstream streams follow the memory AWB order) at `audio//` (`files`: {stream + file: the layer's file name} where it differs).""" + if row["_isRandomPitch"]: + raise NotImplementedError(f"sound {row['_id']}: random pitch") + name = row["_cueName"] + cue = cues[name] layers = [] for L in cue["layers"]: - s = streams[L["awbId"]] # vgmstream streams follow the memory AWB order - if s["name"] != cue_name or abs(s["samples"] - L["samples"]) > 8: - raise RuntimeError(f"{sheet}/{cue_name}: AWB {L['awbId']} -> stream {s} does not match") + s = streams[L["awbId"]] + if s["name"] != name or abs(s["samples"] - L["samples"]) > 8: + raise RuntimeError(f"{sheet}/{name}: AWB {L['awbId']} -> stream {s} does not match") layers.append( { - "file": f"audio/{sheet}/{s['file']}", + "file": f"audio/{sheet}/{(files or {}).get(s['file'], s['file'])}", "stream": s["stream"], "awbId": L["awbId"], "sampleRate": s["sampleRate"], @@ -368,7 +446,7 @@ def _sound_entry(row: dict, sheet: str, cue: dict, streams: list[dict], cue_name return { "row": row, "sheet": sheet, - "cue": cue_name, + "cue": name, "category": row["_category"], "categories": cue["categories"], "volume": cue["volume"] * cue["acbVolume"], @@ -379,6 +457,125 @@ def _sound_entry(row: dict, sheet: str, cue: dict, streams: list[dict], cue_name } +def sound_entries(tables: dict, ids, sheet) -> tuple[dict, dict]: + """({sound id: sound_entry}, {cue sheet: (cues, streams)}) of sound ids; `sheet(name)` gives a cue sheet's + (ACB bytes, streams).""" + sounds = {r["_id"]: r for r in tables["MasterSound"]} + entries, read = {}, {} + for name, sids in sheets_of(tables, ids).items(): + acb, streams = sheet(name) + cues = acb_cues(acb) + read[name] = (cues, streams) + for sid in sids: + entries[str(sid)] = sound_entry(sounds[sid], name, cues, streams) + return entries, read + + +def music_doc(tables: dict, music_id: int, acb: bytes, streams: list[dict], ext: str) -> dict: + """The live.music document of a music (`tables`: music_record): its BGM sound (`music`: sound id, volume, start + time) and that sound's entry (`sounds`), each layer's file with the extension `ext` (".m4a": the web format).""" + row, sheet = music_sound(tables, music_id) + name = sheet["_cueSheetName"] + files = {s["file"]: s["file"].rsplit(".", 1)[0] + ext for s in streams} + return { + "format": MUSIC_FORMAT, + "musicId": music_id, + "music": {"soundId": row["_id"], "volume": 1.0, "startTimeSec": 0.0}, + "sounds": {str(row["_id"]): sound_entry(row, name, acb_cues(acb), streams, files)}, + } + + +def music_record(tables: dict, music_id: int) -> dict: + """What music_doc reads of the master tables for one music: its MasterLiveMusic row, its BGM's MasterSound and + MasterSoundCueSheet rows.""" + music = [r for r in tables["MasterLiveMusic"] if r["_id"] == music_id] + if len(music) != 1: + raise KeyError(f"MasterLiveMusic {music_id}") + row, sheet = music_sound(tables, music_id) + return {"MasterLiveMusic": music, "MasterSound": [row], "MasterSoundCueSheet": [sheet]} + + +def sounds_subject(groups: tuple = ()) -> str: + """The live.sounds subject of the note sound sets offered (LiveOptions.se_patterns: () for the default alone).""" + return f"{SE_PATTERN}={','.join(map(str, sorted(groups)))}" if groups else "default" + + +def sounds_groups(subject: str) -> tuple: + """The note sound sets of a live.sounds subject (sounds_subject).""" + if subject == "default": + return () + name, eq, values = subject.partition("=") + try: + groups = tuple(int(v) for v in values.split(",")) if name == SE_PATTERN and eq else () + except ValueError: + groups = () + if not groups or list(groups) != sorted(set(groups)): + raise ValueError(f"{subject}: not a live sounds subject (default, {SE_PATTERN}=)") + return groups + + +def sounds_record(tables: dict, subject: str) -> dict: + """What sounds_doc reads of the master tables for a live.sounds subject: the option defaults of preset 1, the + note SE and live SE tables, and the MasterSound / MasterSoundCueSheet rows of its sounds.""" + sounds = live_sounds(tables, sounds_groups(subject)) + return { + "MasterOptionDefault": [r for r in tables["MasterOptionDefault"] if r["_presetId"] == 1], + "MasterLiveNoteSe": tables["MasterLiveNoteSe"], + "MasterLiveSe": tables["MasterLiveSe"], + **sound_record(tables, sounds["ids"]), + } + + +def sounds_doc(tables: dict, subject: str, sheet, svs: dict, acd: dict, acf: bytes) -> dict: + """The live.sounds document (`tables`: sounds_record; `sheet(name)`: a cue sheet's (ACB bytes, streams); `svs`, + `acd`: the SETTINGS_SCRIPTS MonoBehaviours; `acf`: the ACF): every sound a live plays besides the BGM, the + category volumes, the ACF's buses and categories, REACT, the note SE and live SE tables and the intro / end + timings.""" + live = live_sounds(tables, sounds_groups(subject)) + entries, _ = sound_entries(tables, live["ids"], sheet) + info = acf_info(acf) + return { + "format": SOUNDS_FORMAT, + "sounds": entries, + "categories": category_volumes(svs, acd), + "acf": {"name": info["name"], "buses": info["buses"], "categories": info["categories"]}, + "react": info["react"], + "noteSe": live["noteSe"], + "liveSe": {str(k): v for k, v in sorted(live["liveSe"].items())}, + "liveSeNames": LIVE_SE_TYPES, + "timeline": TIMELINE, + } + + +# --------------------------------------------------------------------------- extraction +def _decode(cat: Catalog, sheet: str, out_dir: Path, fmt: str, flac_level: int) -> list[dict]: + """Decode a cue sheet into /audio// (reusing an existing decode) and return its stream list.""" + from ..formats import cri + + d = out_dir / "audio" / sheet + if not (d / "streams.json").exists(): + cri.decode(cat, sheet, d, fmt=fmt, flac_level=flac_level) + import json + + return json.loads((d / "streams.json").read_text(encoding="utf-8")) + + +def _player_mono(player: PlayerData, cls: str) -> dict: + hits = [] + for o in player.env.objects: + if o.type.name != "MonoBehaviour": + continue + try: + if player.script(o)[2] != cls: + continue + except Exception: # MonoBehaviours without a resolvable script are not ours + continue + hits.append(player.mono(o)) + if len(hits) != 1: + raise RuntimeError(f"{cls}: {len(hits)} objects in the player data") + return hits[0] + + def extract( cat: Catalog, master: Path, @@ -386,85 +583,59 @@ def extract( music_id: int, out_dir: Path, fmt: str = "flac", - voice_character: int | None = None, - options: LiveOptions = LiveOptions(), + options: LiveOptions | None = None, *, flac_level: int = FLAC_LEVEL, ) -> dict: - """Decode the live's sound cue sheets and write /audio/live-audio.json. Returns a summary. - voice_character: None = no character voice (the game picks the voice from the player's deck; there is no rule - without a deck). `options`: the note sound sets it offers are written as noteSe.groups, their sounds decoded - after the default ones. `flac_level`: ffmpeg's compression level of the FLAC files (cri.decode).""" + """Decode the live's sound cue sheets and write /audio/live-audio.json (module documentation). Returns a + summary. `options`: the note sound sets it offers are written as noteSe.groups, their sounds decoded after the + default ones. `flac_level`: ffmpeg's compression level of the FLAC files (cri.decode).""" + from ..formats import cri + from .score import master_table + out_dir = Path(out_dir) if cat.apk is None: raise RuntimeError("liveaudio needs the Catalog opened with apk= (ACF, HCA key)") - sounds = {str(r["_id"]): r for r in master_table(master, "MasterSound")} - sheets = {r["_id"]: r["_cueSheetName"] for r in master_table(master, "MasterSoundCueSheet")} - music = [r for r in master_table(master, "MasterLiveMusic") if r["_id"] == music_id] - if len(music) != 1: - raise KeyError(f"MasterLiveMusic {music_id}") - music_sound = music[0]["_musicSoundID"] - nse = note_se_settings(master, groups=options.se_patterns) - live_se = {r["_liveSeType"]: r["_seId"] for r in master_table(master, "MasterLiveSe")} - want = [music_sound] + sorted(set(nse["types"].values())) + [live_se[t] for t in LIVE_SE_SOUNDS] - voice = { - "decision": "caller", - "character": voice_character, - "rule": "LotteryStartVoice: random deck member, random MasterLiveStartCharacterVoice row of that " - "character; finish voice from LotteryFinishVoice; no deck -> undefined", - "startVoiceSoundIds": [], - } - if voice_character is not None: - rows = [ - r for r in master_table(master, "MasterLiveStartCharacterVoice") if r["_characterId"] == voice_character - ] - voice["startVoiceSoundIds"] = [r["_voiceSoundId"] for r in rows] - want += voice["startVoiceSoundIds"] - for se in sorted({v for g in nse.get("groups", {}).values() for v in g.values()}): - if se not in want: - want.append(se) - - by_sheet: dict[str, list[int]] = {} - for sid in want: - r = sounds[str(sid)] - by_sheet.setdefault(sheets[r["_soundCueSheetID"]], []).append(sid) - entries, decoded = {}, {} - for sheet, ids in by_sheet.items(): - acb, layout = cri.acb_data(cat, sheet) - cues = acb_cues(acb["acb"]) - streams = _decode(cat, sheet, out_dir, fmt, flac_level) - decoded[sheet] = {"layout": layout, "cues": len(cues), "streams": len(streams)} - for sid in ids: - r = sounds[str(sid)] - if r["_isRandomPitch"]: - raise NotImplementedError(f"sound {sid}: random pitch") - entries[str(sid)] = _sound_entry(r, sheet, cues[r["_cueName"]], streams, r["_cueName"]) - - acf = acf_info(cat.apk) + tables = {t: master_table(master, t) for t in dict.fromkeys((*MUSIC_TABLES, *SOUND_TABLES))} + music_row, _ = music_sound(tables, music_id) + live = live_sounds(tables, options.se_patterns if options is not None else ()) + layouts = {} + + def sheet(name: str): + acb, layouts[name] = cri.acb_data(cat, name) + return acb["acb"], _decode(cat, name, out_dir, fmt, flac_level) + + entries, read = sound_entries(tables, list(dict.fromkeys([music_row["_id"], *live["ids"]])), sheet) + with ApkSet(cat.apk) as z: + acf = acf_info(z.read(ACF_IN_APK)) + svs, acd = (_player_mono(player, s) for s in SETTINGS_SCRIPTS) doc = { "musicId": music_id, "spec": "nnnotes.content.liveaudio", "sounds": entries, - "categories": category_volumes(player), + "categories": category_volumes(svs, acd), "acf": {"name": acf["name"], "buses": acf["buses"], "categories": acf["categories"]}, "react": acf["react"], - "music": {"soundId": music_sound, "volume": 1.0, "startTimeSec": 0.0}, - "noteSe": nse, - "liveSe": {str(k): v for k, v in sorted(live_se.items())}, + "music": {"soundId": music_row["_id"], "volume": 1.0, "startTimeSec": 0.0}, + "noteSe": live["noteSe"], + "liveSe": {str(k): v for k, v in sorted(live["liveSe"].items())}, "liveSeNames": LIVE_SE_TYPES, - "timeline": { - "startCheerDelaySec": 1.0, - "playVoiceSignalSec": 1.483333, - "startCheerStopFadeSec": 1.0, - "finishCheerStopFadeSec": 1.0, + "timeline": TIMELINE, + "voice": { + "decision": "caller", + "character": None, + "rule": "LotteryStartVoice: random deck member, random MasterLiveStartCharacterVoice row of that " + "character; finish voice from LotteryFinishVoice; no deck -> undefined", + "startVoiceSoundIds": [], + }, + "decoded": { + n: {"layout": layouts[n], "cues": len(cues), "streams": len(streams)} for n, (cues, streams) in read.items() }, - "voice": voice, - "decoded": decoded, } write_json(out_dir / "audio" / "live-audio.json", doc) return { "index": "audio/live-audio.json", - "sheets": sorted(by_sheet), + "sheets": sorted(read), "sounds": len(entries), - "musicSoundId": music_sound, + "musicSoundId": music_row["_id"], } diff --git a/src/nnnotes/formats/encoders.py b/src/nnnotes/formats/encoders.py index b74ba81..a070d56 100644 --- a/src/nnnotes/formats/encoders.py +++ b/src/nnnotes/formats/encoders.py @@ -1,11 +1,28 @@ """The ffmpeg encoder settings the audio and movie writers share. Every encode is bit-exact: the same input and ffmpeg -build give the same bytes.""" +build give the same bytes. + +WEB_AUDIO lists the audio formats for the web (`format` of the live.music product, `site build --format`): per format +the file suffix and the ffmpeg codec options (None: the FLAC as decoded). A browser's decodeAudioData decodes each +sample-aligned with the FLAC (the encoder delay and pre-skip removed); AAC-LC 160 kbit/s in MP4, the default, decodes +in Safari and iOS as well. A decoder that does not apply the MP4 edit list outputs `mp4_priming(data)` priming samples +first. +""" from __future__ import annotations +import struct +import subprocess from pathlib import Path FLAC_LEVEL = 8 # ffmpeg's FLAC compression level by default (-compression_level of the flac output) +WEB_AUDIO = { + "aac": (".m4a", ["-c:a", "aac", "-b:a", "160k"]), + "opus": (".opus", ["-c:a", "libopus", "-b:a", "128k"]), + "vorbis": (".ogg", ["-c:a", "libvorbis", "-q:a", "5"]), + "mp3": (".mp3", ["-c:a", "libmp3lame", "-b:a", "192k"]), + "flac": (".flac", None), +} +DEFAULT_AUDIO_FORMAT = "aac" def encode_args(src: Path, dst: Path, args: list[str]) -> list[str]: @@ -27,3 +44,48 @@ def encode_args(src: Path, dst: Path, args: list[str]) -> list[str]: *args, str(dst), ] + + +def web_encode(ffmpeg: str, src: Path, dst: Path, fmt: str) -> bytes: + """The bytes of the audio file `src` in the web format `fmt` (WEB_AUDIO; a bit-exact encode written to `dst`; + "flac": the file as it is).""" + codec = WEB_AUDIO[fmt][1] + if codec is None: + return Path(src).read_bytes() + r = subprocess.run([ffmpeg, *encode_args(src, dst, codec)], capture_output=True, text=True) + if r.returncode != 0: + raise RuntimeError(f"ffmpeg {fmt} failed on {src}: {r.stderr[-2000:]}") + return Path(dst).read_bytes() + + +def mp4_priming(data: bytes) -> int: + """Encoder delay of an MP4 audio track: media_time of the first edit list entry (moov/trak/edts/elst).""" + + def boxes(b, off, end): + while off + 8 <= end: + size, kind = struct.unpack(">I4s", b[off : off + 8]) + hdr = 8 + if size == 1: + size, hdr = struct.unpack(">Q", b[off + 8 : off + 16])[0], 16 + elif size == 0: + size = end - off + yield kind, off + hdr, off + size + off += size + + def find(path, off, end): + for kind, a, z in boxes(data, off, end): + if kind == path[0]: + return (a, z) if len(path) == 1 else find(path[1:], a, z) + return None + + loc = find([b"moov", b"trak", b"edts", b"elst"], 0, len(data)) + if loc is None: + return 0 + a, _ = loc + version = data[a] + count = struct.unpack(">I", data[a + 4 : a + 8])[0] + if count < 1: + return 0 + if version == 1: + return max(0, struct.unpack(">q", data[a + 16 : a + 24])[0]) + return max(0, struct.unpack(">i", data[a + 12 : a + 16])[0]) diff --git a/src/nnnotes/pipeline/assemble.py b/src/nnnotes/pipeline/assemble.py index 9ba12f6..c49f3e3 100644 --- a/src/nnnotes/pipeline/assemble.py +++ b/src/nnnotes/pipeline/assemble.py @@ -1,7 +1,8 @@ """Product stages: products assembled from the unity.export results of the objects they reach, never from a bundle. A product (a Live2D model, a story, a live, ...) is a stage whose task per subject starts from root objects: the -objects of the catalog keys `keys(subject)` gives, of the Resources.Load paths `resources(subject)` gives and the +objects of the catalog keys `keys(subject)` gives, of the Resources.Load paths `resources(subject)` gives, of the +built-in selectors `builtins(subject)` gives (objects of the APK's built-in files by class or script class) and the objects `root_objects(subject)` names (all read from the planning facts alone, so a command can ask for the units its products need before anything runs: `request`). The objects it reaches are those of the units holding its roots and every unit an item of a reached unit references @@ -10,11 +11,19 @@ that no census holds is listed as missing. A task's inputs are the object table of what it reaches ("objects", nnnotes.objects/1: every object with its class, -name, item status and files, the contained ones with the object holding them) and what `more_inputs` adds (master -tables, parameters' data); it depends on the unity.export and sprite.crop tasks it read. `assemble` makes the -product's own files from the table and the bytes of its objects; the result holds them as "#", the -table of the objects they use as "#objects", and one item for the subject. A product layout -(layout.product_entries) places the files and the objects' files and writes the product's manifest. +name, item status and files, the contained ones with the object holding them), the APK files the stage names in `APK` +("apk:", from the fact "apk"), the cue sheets of the catalog keys `cue_sheets(subject)` gives and +what `more_inputs` adds (master tables, parameters' data); it depends on the unity.export and sprite.crop tasks it +read and the cri.audio tasks of its cue sheets. A cue sheet is the ACB the key loads: held in a bundle (the `acb` +file of an object of the key) or a raw file of the catalog (a dependency of the key that cri.audio decoded); its +inputs are the ACB ("acb:") and, in "sheets", the files cri.audio made of it ({key: {acb, status, reason?, +files}}). + +`assemble` makes the product's own files from the table, the bytes of its objects and its cue sheets (Sheet); a file +may be content the depot already holds (Stored: a decoded waveform), recorded without reading it again. The result +holds them as "#", the table of the objects they use as "#objects", and one item for the +subject. A product layout (layout.product_entries) places the files and the objects' files and writes the product's +manifest. """ from __future__ import annotations @@ -29,22 +38,29 @@ from .orchestrate import SEVERITY from .stages import Output, Stage -EXPORT, CROP = "unity.export", "sprite.crop" +EXPORT, CROP, AUDIO = "unity.export", "sprite.crop", "cri.audio" ADDRESSES = units.ADDRESSES TABLE_KIND = "product.objects" +SHEETS_INPUT = "sheets" def request(stages: dict, products: dict, env) -> dict: """The fact "roots" (pipeline.units) of a request for `products` {stage name: subjects}: the catalog keys, the - resource paths and the root objects of every subject, sorted.""" - keys, resources, objects = set(), set(), set() + resource paths, the built-in selectors and the root objects of every subject, sorted.""" + keys, resources, builtins, objects = set(), set(), set(), set() for name, subjects in sorted(products.items()): stage = stages[name] for s in subjects: keys.update(stage.keys(s, env)) resources.update(stage.resources(s, env)) + builtins.update(stage.builtins(s, env)) objects.update(stage.root_objects(s, env)) - return {"keys": sorted(keys), "resources": sorted(resources), "objects": sorted(objects)} + return { + "keys": sorted(keys), + "resources": sorted(resources), + "builtins": sorted(builtins), + "objects": sorted(objects), + } def _entry(item: dict, arts: dict) -> dict: @@ -166,14 +182,60 @@ def reach(env, roots, results: _Results | None = None) -> Reach: return Reach(doc, sorted(read), size) +@dataclass(frozen=True) +class Stored: + """A product file that is content the depot holds: its content id and size.""" + + sha256: str + size: int + + +class Sheet: + """A cue sheet of a product task: its ACB and the files cri.audio made of it (one per stream, `cues.json`, + `streams.json`).""" + + def __init__(self, key: str, entry: dict, acb: Input, depot): + self.key, self.entry, self._acb, self.depot = key, entry, acb, depot + + def _check(self) -> None: + if self.entry["status"] != "exported": + reason = self.entry.get("reason") or "unsupported.cri.sheet" + raise Unsupported(reason, f"cue sheet {self.key} was not decoded ({self.entry['status']})") + + def acb(self) -> bytes: + return self.depot.input_bytes(self._acb) + + def file(self, name: str) -> Stored: + self._check() + f = self.entry["files"].get(name) + if f is None: + raise KeyError(f"cue sheet {self.key} has no file {name!r}") + return Stored(f["sha256"], f["size"]) + + def bytes(self, name: str) -> bytes: + return self.depot.read(self.file(name).sha256) + + def streams(self) -> list[dict]: + """streams.json: every stream in decoder order ({stream, name, file, sampleRate, channels, samples, ...}).""" + return contract.loads(self.bytes("streams.json")) + + class Objects: - """The object table of a product task and the bytes of its objects' files (from the depot).""" + """The object table of a product task, the bytes of its objects' files (from the depot) and its cue sheets.""" - def __init__(self, doc: dict, depot): + def __init__(self, doc: dict, depot, sheets: dict | None = None): self.doc, self.depot = doc, depot self.roots: list[str] = doc["roots"] self._objects: dict = doc["objects"] self._json: dict = {} + self._sheets: dict[str, Sheet] = sheets or {} + + def sheet(self, key: str) -> Sheet: + """The cue sheet of the catalog key `key` (one of the stage's cue_sheets).""" + try: + return self._sheets[key] + except KeyError: + raise KeyError(f"cue sheet {key} is not an input of the product") from None def __contains__(self, oid: str) -> bool: return oid in self._objects @@ -204,10 +266,10 @@ def json(self, oid: str, role: str = "json"): @dataclass class Assembly: - """What `assemble` makes: the product's files {path: (bytes, extension, kind)} and what of the object table they - use: `objects` the object ids (None: every object of the table), `roles` {object id: the roles of its files the - product keeps} for an object of which it uses some files only (a shader's programs for one graphics API). The - product's object table, and so its manifest, holds only those.""" + """What `assemble` makes: the product's files {path: (bytes or Stored, extension, kind)} and what of the object + table they use: `objects` the object ids (None: every object of the table), `roles` {object id: the roles of its + files the product keeps} for an object of which it uses some files only (a shader's programs for one graphics + API). The product's object table, and so its manifest, holds only those.""" files: dict objects: set | None = None @@ -237,12 +299,14 @@ def _prune(tid: str, objects: dict, made: Assembly) -> dict: class ProductStage(Stage): """Base of the product stages (module documentation). Subclasses set `name`, `version` and `KIND`, and define - `keys`, `resources` and/or `root_objects` and `assemble`; `subjects` defaults to the subjects of the fact - "products". A stage that reads master tables names them in `MASTER` (its inputs come from the fact "master").""" + `keys`, `resources`, `builtins` and/or `root_objects` and `assemble`; `subjects` defaults to the subjects of the + fact "products". A stage that reads master tables names them in `MASTER` (its inputs come from the fact + "master"), one that reads files of the APK names them in `APK`.""" - after = (ADDRESSES, EXPORT, CROP) + after = (ADDRESSES, EXPORT, CROP, AUDIO) KIND = "product" MASTER: tuple[str, ...] = () + APK: tuple[str, ...] = () def __init__(self): self._memo: dict = {} @@ -259,8 +323,9 @@ def __setstate__(self, state): def subjects(self, env) -> list[str]: return sorted((env.facts.get("products") or {}).get(self.name, ())) - def select(self, keys: list[str]) -> list[str]: - """The subjects among the catalog keys `keys` (those a selection's bundles hold), sorted.""" + def select(self, keys: list[str], env) -> list[str]: + """The subjects among the catalog keys `keys` (those a selection's bundles hold), sorted (from the facts + alone).""" return [] def keys(self, subject: str, env) -> list[str]: @@ -271,11 +336,20 @@ def resources(self, subject: str, env) -> list[str]: """The Resources.Load paths whose objects are roots of `subject` (from the facts alone).""" return [] + def builtins(self, subject: str, env) -> list[str]: + """The built-in selectors ("", "MonoBehaviour: