feat(spot)!: assemble spots and rooms from the exported objects

docs/architecture.md

@@ -36,9 +36,10 @@ decrypted bundles in the depot's `downloads/` directory.
 
 ## Two ways to export
 
-- **One subject.** `export episode`, `export story`, `export live`, `export live2d`, `export spot`, `export room`,
-  `export shader`, `export audio`, `export ui`, `site build` and `music-data` read what one subject needs through the
-  Session and write their output directory directly ([Export stories and lives](export.md)).
+- **One subject.** `export episode`, `export story`, `export live`, `export shader`, `export audio`, `export ui`,
+  `site build` and `music-data` read what one subject needs through the Session and write their output directory
+  directly ([Export stories and lives](export.md)). `export live2d`, `export spot` and `export room` ask the pipeline
+  below for the products of one subject ([products](products.md)).
 - **Every asset.** `export assets` splits the work into stages ([stages](stages.md)). A task is one stage applied to
   one subject (a bundle, a content id, a view); its key covers exactly what its output depends on: the stage and
   converter versions, the input contents and the options ([contracts](contracts.md)). Results and their files are

docs/cli.md

@@ -243,21 +243,31 @@ a Live2D model -> its Cubism SDK files, or the player's model
 
 ## nnnotes export spot
 
-a spot -> spot.json + Spine + room.glb + shaders
+a spot -> spot.json + Spine files + room.glb + room.json
 
 | Argument | Description |
 | --- | --- |
 | `spot_id` | MasterHomeSpot id |
 | `-o`, `--out` `OUT` | output directory (required) |
+| `--stored-catalog` `LABEL\|SHA` | a catalog version of the depot (`catalog versions`; default: the current catalog) |
+| `--workers` `N` | worker processes (default: the usable CPUs) |
+| `--link` `LINK` | as export assets --link (one of `auto`, `clone`, `hard`, `copy`) |
+| `--explain` | print every setting of the run first |
+| `--dry-run` | print the plan, run nothing |
 
 ## nnnotes export room
 
-a background prefab -> glb
+a spot background -> room.glb + room.json
 
 | Argument | Description |
 | --- | --- |
-| `key` | addressable key of the prefab |
-| `-o`, `--out` `OUT` | output .glb file (required) |
+| `key` | background key Spot/<place>/Background/<place> |
+| `-o`, `--out` `OUT` | output directory (required) |
+| `--stored-catalog` `LABEL\|SHA` | a catalog version of the depot (`catalog versions`; default: the current catalog) |
+| `--workers` `N` | worker processes (default: the usable CPUs) |
+| `--link` `LINK` | as export assets --link (one of `auto`, `clone`, `hard`, `copy`) |
+| `--explain` | print every setting of the run first |
+| `--dry-run` | print the plan, run nothing |
 
 ## nnnotes export shader
 

docs/commands.md

@@ -320,42 +320,26 @@ nnnotes export spot SPOT_ID -o OUT
 ```
 OUT/spot.json         master row, names, tap-talk episodes, situation settings, tap targets, Spine characters
 OUT/spine/            Spine skeletons (.json or .skel), atlases, atlas page PNGs
-OUT/room.glb          the spot's background as binary glTF (as `export room`), with room.json
-OUT/shaders/          shaders of the spot's bundles (as `export shader`)
+OUT/room.glb          the spot's background as binary glTF (as `export room`)
+OUT/room.json         its summary
 ```
 
-Needs `[paths] apk` and master data (`MasterHomeSpot`, `MasterText`, `MasterStoryHomeSpotTapTalkEpisode`). `room.glb` and
-`room.json` are the `export room` export of the spot's background.
-
-A reference the situation prefab leaves empty is written as null: a Spine character with no `_animation` has null
-`skeletonData`, `animation` and `world`, and a tap target with no `_focus` has a null `focusWorld`.
+The spot whose `MasterHomeSpot` row has the id `SPOT_ID`: the products `spot.files` of its situation
+(`_situationAssetPath`) and `room.gltf` of its background (`_backgroundAssetPath`), their files at their own paths
+([products](products.md#spots)). The command runs the depot's pipeline for the two products alone, as `export live2d`
+does, writes the same reports in `OUT/_reports/` and takes the same options. Needs `[paths] apk` (the Spot and Spine
+component classes are in the APK) and master data (`MasterHomeSpot`, `MasterText`,
+`MasterStoryHomeSpotTapTalkEpisode`).
 
 ## export room
 
 ```
-nnnotes export room KEY -o OUT.glb
+nnnotes export room KEY -o OUT
 ```
 
-A spot background prefab as binary glTF: every mesh baked into prefab space, converted to glTF's right-handed space,
-textures embedded, materials translated from the shader's render state. Objects inactive in the prefab are kept with
-`extras.unityActive = false`. A summary (`meshCount`, materials, textures, samplers) is written next to it as
-`OUT.json`.
-
-Materials and textures are told apart by serialized file and path id (the prefab's bundle and its dependencies can
-reuse a path id). Each texture is one glTF texture with its own sampler, and textures whose PNG bytes are equal share
-one glTF image. The summary's `textures` has one entry per glTF texture, in glTF order (the index a material's
-`baseColorTexture` names): the Unity texture's name.
-
-The render state is that of the shader's first pass for the material: blend factors (colour and alpha), blend
-operations, colour mask, culling, depth write, depth test and alpha to mask. A state the shader takes from a
-property (`Blend [_SrcBlend] [_DstBlend]`, `Cull [_Cull]`, `ZWrite [_ZWrite]`, ...) is the material's value of that
-property, else the shader's default for it. Each glTF material keeps the resolved state in
-`extras.unityRenderState` (and `OUT.json` in `materials[].renderState`); a shader or a state glTF cannot express
-stops the command. A submesh whose material slot is empty is not written (there is no material to draw it with): the
-summary lists each as `nullMaterialSubmeshes` (`mesh`, object `path` in the prefab, `submesh` index), and a mesh
-left with no submesh is not written. A MeshFilter with no mesh, or a mesh with no triangles (no index data), draws
-nothing and is not written; the summary lists the object path in `nullMeshFilters`, or the mesh and object path in
-`meshesWithoutTriangles`.
+A spot background as binary glTF, `OUT/room.glb`, with its summary `OUT/room.json`: the product `room.gltf` of the
+background key `KEY` (`Spot/<place>/Background/<place>`; [products](products.md#spots)). Runs and reports as
+`export spot`.
 
 ## export shader
 

docs/export.md

@@ -59,11 +59,11 @@ Details: [export live2d](commands.md#export-live2d).
 
 ```sh
 nnnotes export spot SPOT_ID -o out/spots/SPOT_ID
-nnnotes export room KEY -o out/rooms/room.glb             # a background prefab key
+nnnotes export room KEY -o out/rooms/PLACE                 # a background key
 ```
 
-`export spot` writes the spot's master row with its names and tap-talk episodes, its Spine characters, its
-background as binary glTF and the shaders of its bundles. `export room` writes one background prefab as binary glTF
+`export spot` writes the spot's master row with its names and tap-talk episodes, its tap targets, its Spine
+characters and their files, and its background as binary glTF. `export room` writes one background as binary glTF
 with a summary next to it. Details: [export spot](commands.md#export-spot), [export room](commands.md#export-room).
 
 ## Shaders, audio and the UI library

docs/products.md

@@ -67,9 +67,10 @@ A product layout places products for a player (`export assets --layout product`,
   `<stage>/<subject>.json` with the subject escaped as one path segment).
 
 A command asks for products in one of two ways: `export assets --layout product` makes, with each product stage, the
-products of the catalog keys the selected bundles hold (the stage's `select`); `export live2d` asks for one model.
-`export live2d` without `--player` places the files of its one product at their own paths instead (the `files`
-placement: the Cubism SDK files as a directory).
+products of the catalog keys the selected bundles hold (the stage's `select`; a stage that reads master data has none
+without it); `export live2d`, `export spot` and `export room` ask for the products of one subject. `export live2d`
+without `--player`, `export spot` and `export room` place the files of their products at their own paths instead (the
+`files` placement: the Cubism SDK files as a directory, a spot's files).
 
 ## Product manifest: nnnotes.product/1
 
@@ -127,3 +128,65 @@ story renderer adds to a character draw), and the mask materials' when a drawabl
 | `<expression>.exp3.json` | each CubismExpressionData of the table |
 | `motions/<motion>.motion3.json` | each CubismFadeMotionData, with the sample rate of the clip of its name |
 | `<id>.model3.json` | the files, the model's fade times of each motion, and the EyeBlink and LipSync groups (the GameObjects holding a CubismEyeBlinkParameter or a CubismMouthParameter) |
+
+## Spots
+
+A spot of the home is a background prefab (a room) and a situation prefab placed on it. Two product stages
+(`nnnotes.pipeline.spot`) assemble them; the roots of each are the objects of its subject's catalog key, and neither
+keeps objects: its files are all it holds.
+
+`room.gltf` is a background as binary glTF. Its subject is a background key, `Spot/<place>/Background/<place>`, and
+its manifest `rooms/<place>.json`. Its files:
+
+| File | |
+|---|---|
+| `room.glb` | every drawn mesh of the prefab (a MeshFilter with a mesh beside an enabled MeshRenderer), in hierarchy order |
+| `room.json` | the summary: `backgroundKey`, `glb`, `meshCount`, `inactiveMeshes`, `materials`, `textures`, `samplers`, and what draws nothing (below) |
+
+Each drawn mesh is one glTF node without a transform: its vertices are baked into the prefab's space through the
+world matrix of its node (the root's own transform included) and written in glTF's right-handed space (z negated,
+v = 1 - v). Each non-empty submesh is one primitive with the renderer's material of the same index. A node inactive in
+the prefab is kept with `extras.unityActive = false`. The situation's `backgroundPosition`, `backgroundRotation` and
+`backgroundScale` are not baked in: they go on top, as the game applies them to the prefab root.
+
+Materials are translated from the shader's own render state, the first pass's: blend factors (colour and alpha), blend
+operations, colour mask, culling, depth write, depth test and alpha to mask. A state the shader takes from a property
+(`Blend [_SrcBlend] [_DstBlend]`, `Cull [_Cull]`, `ZWrite [_ZWrite]`, ...) is the material's value of that property,
+else the shader's default for it. Each glTF material keeps the resolved state in `extras.unityRenderState` (and
+`room.json` in `materials[].renderState`); a shader or a state glTF cannot express is an error of the task. Each
+material and each texture is one object of the table, so materials and textures are told apart by object id. Each
+texture a material's main slot names is one glTF texture with its own sampler (from the texture's filter, wrap and
+mip count), and textures whose PNG bytes are equal share one glTF image. The `textures` of `room.json` has one entry
+per glTF texture, in glTF order: the Unity texture's name.
+
+What draws nothing is not written and `room.json` lists it: a submesh whose material slot is empty
+(`nullMaterialSubmeshes`: `mesh`, object `path` in the prefab, `submesh` index), a MeshFilter without a mesh
+(`nullMeshFilters`: the object path) and a mesh without vertices or triangles (`emptyMeshes`: `mesh` and `path`).
+
+`spot.files` is a situation: its subject is a situation key, `Spot/<place>/<situation>/<situation>`, and its manifest
+`spots/<situation>.json`. Its input `master` is the spot's record of the master tables `MasterHomeSpot`,
+`MasterText` and `MasterStoryHomeSpotTapTalkEpisode` (the row whose `_situationAssetPath` is the key, its name, its
+tap talks), made at planning from the master data. Its files are `spot.json` and the Spine files under `spine/`:
+
+| Field of spot.json | |
+|---|---|
+| `spotId`, `master` | the spot's id and its `MasterHomeSpot` row |
+| `name` | its name per language (`japanese`, `english`, `traditionalChinese`, `simplifiedChinese`, `korean`) |
+| `tapTalk` | `{character id: [ADV ids]}` of its tap talks |
+| `situationSettings` | the fields of the prefab's SpotSituationSettings (camera and interaction parameters) |
+| `controller` | the fields of its SpotCharacterController but the character lists |
+| `characters` | each SpotCharacter (a tap target) by character id: its object `path`, its fields, its `world` matrix and the world matrix of its `_focus` (`focusWorld`) |
+| `spineCharacters` | each SpotSpineCharacter in hierarchy order: `path`, the name of the skeleton its SkeletonAnimation draws (`skeletonData`), that animation's settings (`animation`) and its `world` matrix |
+| `skeletons` | each SkeletonDataAsset of the table, by name: `name`, `skeleton` and `atlases` (its files), `scale`, `defaultMix`, `mixes` |
+| `matrixConvention` | `Unity world space (left-handed), 4x4 column-major` |
+
+World matrices are in the space of the situation prefab, its root's own transform included. Fields that name objects
+are written as the prefab document writes them. A reference the prefab leaves empty is written as null: a Spine
+character with no `_animation` has null `skeletonData`, `animation` and `world`, and a tap target with no `_focus` a
+null `focusWorld`.
+
+| Spine file | From |
+|---|---|
+| `spine/<skeleton>.json` or `.skel` | the bytes of the SkeletonDataAsset's `skeletonJSON` (JSON when they start with `{`) |
+| `spine/<atlas>.atlas` | the bytes of each SpineAtlasAsset's `atlasFile`, as stored (an atlas two skeletons share is written once) |
+| `spine/<page>` | for each page of the atlas, the PNG of the `_MainTex` of the atlas asset's material of the same index |

docs/stages.md

@@ -27,7 +27,8 @@ codec and channel of a USM) and `catalog.index` (version 2: gzip input and Japan
 | `cri.audio` | ACB content | `acb` (a raw file of the catalog, or the `acb` artifact of a cue sheet held in a bundle), `awb` (an AWB the catalog pairs with it), `boot` (the game's boot data: the key's source) | FLAC per stream, `cues.json`, `streams.json` |
 | `cri.movie` | USM content | `usm` (a raw file of the catalog, or the `usm` artifact of a movie held in a bundle), `boot` | every stream as stored (video, alpha, audio, subtitles), `movie.mkv` (or `movie.webm`) |
 | `live2d.model`, `live2d.cubism` | a Live2D model key (`Character/Live2D/<group>/<name>/model/<name>`) a command asks for | `objects`: the object table of what the model reaches ([products](products.md#live2d-models)); depends on `link.addresses` and the `unity.export` tasks it reads | `model.json` (the player's model) and the object table; the Cubism SDK files |
-| `spine.skeleton` | a SkeletonDataAsset of a selected bundle, by its stable address `<bundle>:<pathId>` (in a bundle of several serialized files the file's tag follows the bundle name, as in `link.addresses`) | `bundle:<stable name>`: the bundles holding the serialized files the asset's file reaches through its externals; context `object` (the asset's object id); atom `spot.write_skeleton` (the Spine writer of `nnnotes export spot`) | `spine.skeleton:<subject>#<file>`: the skeleton (`.json` or `.skel`), its atlases (`.atlas`) and their page images, the bytes `nnnotes export spot` writes into `spine/`; one item for the asset |
+| `room.gltf` | a spot background key (`Spot/<place>/Background/<place>`) | `objects`: the object table of what the background reaches ([products](products.md#spots)) | `room.glb` and `room.json` |
+| `spot.files` | a spot situation key (`Spot/<place>/<situation>/<situation>`) | `objects`; `master`: the spot's master record | `spot.json` and the Spine files |
 | `link.addresses` | global | catalog index, censuses (the built-in files' too) | key -> objects, Resources.Load path -> objects, serialized file -> bundle or built-in file, bundle -> keys |
 | `view.<name>` | a catalog (`main`) | `master:<Table>` per master table it reads, `rules` (the view's rule and the resolvers it uses), `addresses` (the part of the address table it read: the keys looked up and the prefixes listed); no parameters, atoms or context | `view.<name>:<subject>#view` (facts: rows, entries, gaps, unreferenced) |
 | `view.voices` | a catalog (`main`) | `master:<Table>` per master table its rules read, `rules`, `addresses`, `sheets` (each cue sheet the `cri.audio` results are known by that a voice names or the rules' prefixes list: its task and ACB content id), `acb:<sha256>` (the ACB of each decoded one), `episodes` (per story episode and part, the `unity.export` artifact holding the object its key names), `adv:<sha256>` (those contents); context `audio` (the keys of those `cri.audio` results); runs after `unity.export` and `cri.audio` | `view.voices:<subject>#view` (facts: rows, entries, gaps, unreferenced streams) |

src/nnnotes/cli/assets.py

@@ -115,9 +115,10 @@ def matches(self, stage: str) -> bool:
     StageSource("sprite.crop", "nnnotes.pipeline.objexport:SpriteCropStage", output=True),
     StageSource("cri.audio", "nnnotes.pipeline.cristages:AudioStage", output=True),
     StageSource("cri.movie", "nnnotes.pipeline.cristages:MovieStage", output=True),
-    StageSource("spine.skeleton", "nnnotes.pipeline.modelstages:SpineStage", output=True),
     StageSource("live2d.model", "nnnotes.pipeline.live2d:ModelStage", output=True, product=True),
     StageSource("live2d.cubism", "nnnotes.pipeline.live2d:CubismStage", output=True, product=True),
+    StageSource("room.gltf", "nnnotes.pipeline.spot:RoomStage", output=True, product=True),
+    StageSource("spot.files", "nnnotes.pipeline.spot:SpotStage", output=True, product=True),
     StageSource("view.", "nnnotes.pipeline.views:stages", output=True),
     StageSource("view.voices", "nnnotes.pipeline.voices:VoiceStage", output=True),
 )
@@ -722,25 +723,31 @@ def facts(self, pipeline: list[str], fetch: bool, background: bool = False) -> d
         inputs = self._inputs("bundle", [bundles[s] for s in sel.census if s not in builtin], fetch)
         inputs = inputs.plus(builtin)
         sel.selected = [s for s in sel.selected if s in inputs]  # files that cannot be read: problems()
+        products = [n for n in pipeline if self.installed.sources[n].product]
+        readers = [n for n in products if self.installed.stages[n].MASTER]  # products that read master tables
         facts = {
             "catalogs": self.catalogs_fact(),
             "index": index,
             "selected": sel.selected,
-            "master": self.master() if any(n.startswith("view.") for n in pipeline) else None,
+            "master": self.master() if readers or any(n.startswith("view.") for n in pipeline) else None,
             "views": sorted(n[5:] for n in pipeline if n.startswith("view.")),
             "bundles": inputs,
             "classes": self.classes(),
             "raw": Inputs({}),
         }
-        products = [n for n in pipeline if self.installed.sources[n].product]
         if products:
             from ..pipeline import assemble
 
             if self.products is not None:
                 chosen = {n: self.products.get(n, []) for n in products}
-            else:
+                if facts["master"] is None and any(chosen[n] for n in readers):
+                    raise self.cfg.missing(*self.cfg.master(self.cfg.get("catalog", "region")))
+            else:  # without master data, the products that read it have no subjects (as the views)
                 held = held_keys(index, sel.selected)
-                chosen = {n: self.installed.stages[n].select(held) for n in products}
+                chosen = {
+                    n: [] if n in readers and facts["master"] is None else self.installed.stages[n].select(held)
+                    for n in products
+                }
             facts["products"] = chosen
             facts["roots"] = assemble.request(self.installed.stages, chosen, Env(self.depot, facts))
             if self.products is not None or not set(self.layouts) & set(OBJECT_LAYOUTS):