diff --git a/docs/architecture.md b/docs/architecture.md index 21d43e3..42acf20 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -36,9 +36,10 @@ decrypted bundles in the depot's `downloads/` directory. ## Two ways to export -- **One subject.** `export episode`, `export story`, `export live`, `export live2d`, `export spot`, `export room`, - `export shader`, `export audio`, `export ui`, `site build` and `music-data` read what one subject needs through the - Session and write their output directory directly ([Export stories and lives](export.md)). +- **One subject.** `export episode`, `export story`, `export live`, `export shader`, `export audio`, `export ui`, + `site build` and `music-data` read what one subject needs through the Session and write their output directory + directly ([Export stories and lives](export.md)). `export live2d`, `export spot` and `export room` ask the pipeline + below for the products of one subject ([products](products.md)). - **Every asset.** `export assets` splits the work into stages ([stages](stages.md)). A task is one stage applied to one subject (a bundle, a content id, a view); its key covers exactly what its output depends on: the stage and converter versions, the input contents and the options ([contracts](contracts.md)). Results and their files are diff --git a/docs/cli.md b/docs/cli.md index 158e10c..3baa75e 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -243,21 +243,31 @@ a Live2D model -> its Cubism SDK files, or the player's model ## nnnotes export spot -a spot -> spot.json + Spine + room.glb + shaders +a spot -> spot.json + Spine files + room.glb + room.json | Argument | Description | | --- | --- | | `spot_id` | MasterHomeSpot id | | `-o`, `--out` `OUT` | output directory (required) | +| `--stored-catalog` `LABEL\|SHA` | a catalog version of the depot (`catalog versions`; default: the current catalog) | +| `--workers` `N` | worker processes (default: the usable CPUs) | +| `--link` `LINK` | as export assets --link (one of `auto`, `clone`, `hard`, `copy`) | +| `--explain` | print every setting of the run first | +| `--dry-run` | print the plan, run nothing | ## nnnotes export room -a background prefab -> glb +a spot background -> room.glb + room.json | Argument | Description | | --- | --- | -| `key` | addressable key of the prefab | -| `-o`, `--out` `OUT` | output .glb file (required) | +| `key` | background key Spot//Background/ | +| `-o`, `--out` `OUT` | output directory (required) | +| `--stored-catalog` `LABEL\|SHA` | a catalog version of the depot (`catalog versions`; default: the current catalog) | +| `--workers` `N` | worker processes (default: the usable CPUs) | +| `--link` `LINK` | as export assets --link (one of `auto`, `clone`, `hard`, `copy`) | +| `--explain` | print every setting of the run first | +| `--dry-run` | print the plan, run nothing | ## nnnotes export shader diff --git a/docs/commands.md b/docs/commands.md index 67c2697..87191f5 100644 --- a/docs/commands.md +++ b/docs/commands.md @@ -320,42 +320,26 @@ nnnotes export spot SPOT_ID -o OUT ``` OUT/spot.json master row, names, tap-talk episodes, situation settings, tap targets, Spine characters OUT/spine/ Spine skeletons (.json or .skel), atlases, atlas page PNGs -OUT/room.glb the spot's background as binary glTF (as `export room`), with room.json -OUT/shaders/ shaders of the spot's bundles (as `export shader`) +OUT/room.glb the spot's background as binary glTF (as `export room`) +OUT/room.json its summary ``` -Needs `[paths] apk` and master data (`MasterHomeSpot`, `MasterText`, `MasterStoryHomeSpotTapTalkEpisode`). `room.glb` and -`room.json` are the `export room` export of the spot's background. - -A reference the situation prefab leaves empty is written as null: a Spine character with no `_animation` has null -`skeletonData`, `animation` and `world`, and a tap target with no `_focus` has a null `focusWorld`. +The spot whose `MasterHomeSpot` row has the id `SPOT_ID`: the products `spot.files` of its situation +(`_situationAssetPath`) and `room.gltf` of its background (`_backgroundAssetPath`), their files at their own paths +([products](products.md#spots)). The command runs the depot's pipeline for the two products alone, as `export live2d` +does, writes the same reports in `OUT/_reports/` and takes the same options. Needs `[paths] apk` (the Spot and Spine +component classes are in the APK) and master data (`MasterHomeSpot`, `MasterText`, +`MasterStoryHomeSpotTapTalkEpisode`). ## export room ``` -nnnotes export room KEY -o OUT.glb +nnnotes export room KEY -o OUT ``` -A spot background prefab as binary glTF: every mesh baked into prefab space, converted to glTF's right-handed space, -textures embedded, materials translated from the shader's render state. Objects inactive in the prefab are kept with -`extras.unityActive = false`. A summary (`meshCount`, materials, textures, samplers) is written next to it as -`OUT.json`. - -Materials and textures are told apart by serialized file and path id (the prefab's bundle and its dependencies can -reuse a path id). Each texture is one glTF texture with its own sampler, and textures whose PNG bytes are equal share -one glTF image. The summary's `textures` has one entry per glTF texture, in glTF order (the index a material's -`baseColorTexture` names): the Unity texture's name. - -The render state is that of the shader's first pass for the material: blend factors (colour and alpha), blend -operations, colour mask, culling, depth write, depth test and alpha to mask. A state the shader takes from a -property (`Blend [_SrcBlend] [_DstBlend]`, `Cull [_Cull]`, `ZWrite [_ZWrite]`, ...) is the material's value of that -property, else the shader's default for it. Each glTF material keeps the resolved state in -`extras.unityRenderState` (and `OUT.json` in `materials[].renderState`); a shader or a state glTF cannot express -stops the command. A submesh whose material slot is empty is not written (there is no material to draw it with): the -summary lists each as `nullMaterialSubmeshes` (`mesh`, object `path` in the prefab, `submesh` index), and a mesh -left with no submesh is not written. A MeshFilter with no mesh, or a mesh with no triangles (no index data), draws -nothing and is not written; the summary lists the object path in `nullMeshFilters`, or the mesh and object path in -`meshesWithoutTriangles`. +A spot background as binary glTF, `OUT/room.glb`, with its summary `OUT/room.json`: the product `room.gltf` of the +background key `KEY` (`Spot//Background/`; [products](products.md#spots)). Runs and reports as +`export spot`. ## export shader diff --git a/docs/export.md b/docs/export.md index dbf7d9c..5ddeded 100644 --- a/docs/export.md +++ b/docs/export.md @@ -59,11 +59,11 @@ Details: [export live2d](commands.md#export-live2d). ```sh nnnotes export spot SPOT_ID -o out/spots/SPOT_ID -nnnotes export room KEY -o out/rooms/room.glb # a background prefab key +nnnotes export room KEY -o out/rooms/PLACE # a background key ``` -`export spot` writes the spot's master row with its names and tap-talk episodes, its Spine characters, its -background as binary glTF and the shaders of its bundles. `export room` writes one background prefab as binary glTF +`export spot` writes the spot's master row with its names and tap-talk episodes, its tap targets, its Spine +characters and their files, and its background as binary glTF. `export room` writes one background as binary glTF with a summary next to it. Details: [export spot](commands.md#export-spot), [export room](commands.md#export-room). ## Shaders, audio and the UI library diff --git a/docs/products.md b/docs/products.md index c3be8a6..366e81d 100644 --- a/docs/products.md +++ b/docs/products.md @@ -67,9 +67,10 @@ A product layout places products for a player (`export assets --layout product`, `/.json` with the subject escaped as one path segment). A command asks for products in one of two ways: `export assets --layout product` makes, with each product stage, the -products of the catalog keys the selected bundles hold (the stage's `select`); `export live2d` asks for one model. -`export live2d` without `--player` places the files of its one product at their own paths instead (the `files` -placement: the Cubism SDK files as a directory). +products of the catalog keys the selected bundles hold (the stage's `select`; a stage that reads master data has none +without it); `export live2d`, `export spot` and `export room` ask for the products of one subject. `export live2d` +without `--player`, `export spot` and `export room` place the files of their products at their own paths instead (the +`files` placement: the Cubism SDK files as a directory, a spot's files). ## Product manifest: nnnotes.product/1 @@ -127,3 +128,65 @@ story renderer adds to a character draw), and the mask materials' when a drawabl | `.exp3.json` | each CubismExpressionData of the table | | `motions/.motion3.json` | each CubismFadeMotionData, with the sample rate of the clip of its name | | `.model3.json` | the files, the model's fade times of each motion, and the EyeBlink and LipSync groups (the GameObjects holding a CubismEyeBlinkParameter or a CubismMouthParameter) | + +## Spots + +A spot of the home is a background prefab (a room) and a situation prefab placed on it. Two product stages +(`nnnotes.pipeline.spot`) assemble them; the roots of each are the objects of its subject's catalog key, and neither +keeps objects: its files are all it holds. + +`room.gltf` is a background as binary glTF. Its subject is a background key, `Spot//Background/`, and +its manifest `rooms/.json`. Its files: + +| File | | +|---|---| +| `room.glb` | every drawn mesh of the prefab (a MeshFilter with a mesh beside an enabled MeshRenderer), in hierarchy order | +| `room.json` | the summary: `backgroundKey`, `glb`, `meshCount`, `inactiveMeshes`, `materials`, `textures`, `samplers`, and what draws nothing (below) | + +Each drawn mesh is one glTF node without a transform: its vertices are baked into the prefab's space through the +world matrix of its node (the root's own transform included) and written in glTF's right-handed space (z negated, +v = 1 - v). Each non-empty submesh is one primitive with the renderer's material of the same index. A node inactive in +the prefab is kept with `extras.unityActive = false`. The situation's `backgroundPosition`, `backgroundRotation` and +`backgroundScale` are not baked in: they go on top, as the game applies them to the prefab root. + +Materials are translated from the shader's own render state, the first pass's: blend factors (colour and alpha), blend +operations, colour mask, culling, depth write, depth test and alpha to mask. A state the shader takes from a property +(`Blend [_SrcBlend] [_DstBlend]`, `Cull [_Cull]`, `ZWrite [_ZWrite]`, ...) is the material's value of that property, +else the shader's default for it. Each glTF material keeps the resolved state in `extras.unityRenderState` (and +`room.json` in `materials[].renderState`); a shader or a state glTF cannot express is an error of the task. Each +material and each texture is one object of the table, so materials and textures are told apart by object id. Each +texture a material's main slot names is one glTF texture with its own sampler (from the texture's filter, wrap and +mip count), and textures whose PNG bytes are equal share one glTF image. The `textures` of `room.json` has one entry +per glTF texture, in glTF order: the Unity texture's name. + +What draws nothing is not written and `room.json` lists it: a submesh whose material slot is empty +(`nullMaterialSubmeshes`: `mesh`, object `path` in the prefab, `submesh` index), a MeshFilter without a mesh +(`nullMeshFilters`: the object path) and a mesh without vertices or triangles (`emptyMeshes`: `mesh` and `path`). + +`spot.files` is a situation: its subject is a situation key, `Spot///`, and its manifest +`spots/.json`. Its input `master` is the spot's record of the master tables `MasterHomeSpot`, +`MasterText` and `MasterStoryHomeSpotTapTalkEpisode` (the row whose `_situationAssetPath` is the key, its name, its +tap talks), made at planning from the master data. Its files are `spot.json` and the Spine files under `spine/`: + +| Field of spot.json | | +|---|---| +| `spotId`, `master` | the spot's id and its `MasterHomeSpot` row | +| `name` | its name per language (`japanese`, `english`, `traditionalChinese`, `simplifiedChinese`, `korean`) | +| `tapTalk` | `{character id: [ADV ids]}` of its tap talks | +| `situationSettings` | the fields of the prefab's SpotSituationSettings (camera and interaction parameters) | +| `controller` | the fields of its SpotCharacterController but the character lists | +| `characters` | each SpotCharacter (a tap target) by character id: its object `path`, its fields, its `world` matrix and the world matrix of its `_focus` (`focusWorld`) | +| `spineCharacters` | each SpotSpineCharacter in hierarchy order: `path`, the name of the skeleton its SkeletonAnimation draws (`skeletonData`), that animation's settings (`animation`) and its `world` matrix | +| `skeletons` | each SkeletonDataAsset of the table, by name: `name`, `skeleton` and `atlases` (its files), `scale`, `defaultMix`, `mixes` | +| `matrixConvention` | `Unity world space (left-handed), 4x4 column-major` | + +World matrices are in the space of the situation prefab, its root's own transform included. Fields that name objects +are written as the prefab document writes them. A reference the prefab leaves empty is written as null: a Spine +character with no `_animation` has null `skeletonData`, `animation` and `world`, and a tap target with no `_focus` a +null `focusWorld`. + +| Spine file | From | +|---|---| +| `spine/.json` or `.skel` | the bytes of the SkeletonDataAsset's `skeletonJSON` (JSON when they start with `{`) | +| `spine/.atlas` | the bytes of each SpineAtlasAsset's `atlasFile`, as stored (an atlas two skeletons share is written once) | +| `spine/` | for each page of the atlas, the PNG of the `_MainTex` of the atlas asset's material of the same index | diff --git a/docs/stages.md b/docs/stages.md index 693b619..6e26c6f 100644 --- a/docs/stages.md +++ b/docs/stages.md @@ -27,7 +27,8 @@ codec and channel of a USM) and `catalog.index` (version 2: gzip input and Japan | `cri.audio` | ACB content | `acb` (a raw file of the catalog, or the `acb` artifact of a cue sheet held in a bundle), `awb` (an AWB the catalog pairs with it), `boot` (the game's boot data: the key's source) | FLAC per stream, `cues.json`, `streams.json` | | `cri.movie` | USM content | `usm` (a raw file of the catalog, or the `usm` artifact of a movie held in a bundle), `boot` | every stream as stored (video, alpha, audio, subtitles), `movie.mkv` (or `movie.webm`) | | `live2d.model`, `live2d.cubism` | a Live2D model key (`Character/Live2D///model/`) a command asks for | `objects`: the object table of what the model reaches ([products](products.md#live2d-models)); depends on `link.addresses` and the `unity.export` tasks it reads | `model.json` (the player's model) and the object table; the Cubism SDK files | -| `spine.skeleton` | a SkeletonDataAsset of a selected bundle, by its stable address `:` (in a bundle of several serialized files the file's tag follows the bundle name, as in `link.addresses`) | `bundle:`: the bundles holding the serialized files the asset's file reaches through its externals; context `object` (the asset's object id); atom `spot.write_skeleton` (the Spine writer of `nnnotes export spot`) | `spine.skeleton:#`: the skeleton (`.json` or `.skel`), its atlases (`.atlas`) and their page images, the bytes `nnnotes export spot` writes into `spine/`; one item for the asset | +| `room.gltf` | a spot background key (`Spot//Background/`) | `objects`: the object table of what the background reaches ([products](products.md#spots)) | `room.glb` and `room.json` | +| `spot.files` | a spot situation key (`Spot///`) | `objects`; `master`: the spot's master record | `spot.json` and the Spine files | | `link.addresses` | global | catalog index, censuses (the built-in files' too) | key -> objects, Resources.Load path -> objects, serialized file -> bundle or built-in file, bundle -> keys | | `view.` | a catalog (`main`) | `master:` per master table it reads, `rules` (the view's rule and the resolvers it uses), `addresses` (the part of the address table it read: the keys looked up and the prefixes listed); no parameters, atoms or context | `view.:#view` (facts: rows, entries, gaps, unreferenced) | | `view.voices` | a catalog (`main`) | `master:
` per master table its rules read, `rules`, `addresses`, `sheets` (each cue sheet the `cri.audio` results are known by that a voice names or the rules' prefixes list: its task and ACB content id), `acb:` (the ACB of each decoded one), `episodes` (per story episode and part, the `unity.export` artifact holding the object its key names), `adv:` (those contents); context `audio` (the keys of those `cri.audio` results); runs after `unity.export` and `cri.audio` | `view.voices:#view` (facts: rows, entries, gaps, unreferenced streams) | diff --git a/src/nnnotes/cli/assets.py b/src/nnnotes/cli/assets.py index c570b66..ae71088 100644 --- a/src/nnnotes/cli/assets.py +++ b/src/nnnotes/cli/assets.py @@ -115,9 +115,10 @@ def matches(self, stage: str) -> bool: StageSource("sprite.crop", "nnnotes.pipeline.objexport:SpriteCropStage", output=True), StageSource("cri.audio", "nnnotes.pipeline.cristages:AudioStage", output=True), StageSource("cri.movie", "nnnotes.pipeline.cristages:MovieStage", output=True), - StageSource("spine.skeleton", "nnnotes.pipeline.modelstages:SpineStage", output=True), StageSource("live2d.model", "nnnotes.pipeline.live2d:ModelStage", output=True, product=True), StageSource("live2d.cubism", "nnnotes.pipeline.live2d:CubismStage", output=True, product=True), + StageSource("room.gltf", "nnnotes.pipeline.spot:RoomStage", output=True, product=True), + StageSource("spot.files", "nnnotes.pipeline.spot:SpotStage", output=True, product=True), StageSource("view.", "nnnotes.pipeline.views:stages", output=True), StageSource("view.voices", "nnnotes.pipeline.voices:VoiceStage", output=True), ) @@ -722,25 +723,31 @@ def facts(self, pipeline: list[str], fetch: bool, background: bool = False) -> d inputs = self._inputs("bundle", [bundles[s] for s in sel.census if s not in builtin], fetch) inputs = inputs.plus(builtin) sel.selected = [s for s in sel.selected if s in inputs] # files that cannot be read: problems() + products = [n for n in pipeline if self.installed.sources[n].product] + readers = [n for n in products if self.installed.stages[n].MASTER] # products that read master tables facts = { "catalogs": self.catalogs_fact(), "index": index, "selected": sel.selected, - "master": self.master() if any(n.startswith("view.") for n in pipeline) else None, + "master": self.master() if readers or any(n.startswith("view.") for n in pipeline) else None, "views": sorted(n[5:] for n in pipeline if n.startswith("view.")), "bundles": inputs, "classes": self.classes(), "raw": Inputs({}), } - products = [n for n in pipeline if self.installed.sources[n].product] if products: from ..pipeline import assemble if self.products is not None: chosen = {n: self.products.get(n, []) for n in products} - else: + if facts["master"] is None and any(chosen[n] for n in readers): + raise self.cfg.missing(*self.cfg.master(self.cfg.get("catalog", "region"))) + else: # without master data, the products that read it have no subjects (as the views) held = held_keys(index, sel.selected) - chosen = {n: self.installed.stages[n].select(held) for n in products} + chosen = { + n: [] if n in readers and facts["master"] is None else self.installed.stages[n].select(held) + for n in products + } facts["products"] = chosen facts["roots"] = assemble.request(self.installed.stages, chosen, Env(self.depot, facts)) if self.products is not None or not set(self.layouts) & set(OBJECT_LAYOUTS): diff --git a/src/nnnotes/cli/export.py b/src/nnnotes/cli/export.py index bc35a77..d66b5e1 100644 --- a/src/nnnotes/cli/export.py +++ b/src/nnnotes/cli/export.py @@ -141,43 +141,35 @@ def cmd_live2d(args, cfg): def cmd_spot(args, cfg): - from ..content import room, spot - from ..formats import shader + from ..pipeline.spot import ROOM, SPOT + from ..source.master import table cfg.require_path("paths", "apk") # the Spot / Spine component classes are read from the APK + use(cfg) # the tools the atoms name are part of the keys md = Session(cfg).master_dir() known_row(args, md, "MasterHomeSpot", args.spot_id, "spot") - cat = Session(cfg).catalog() - out = Path(args.out) - doc = spot.extract(cat, md, args.spot_id, out) - bg = doc["master"]["_backgroundAssetPath"] - rm = room.extract_room(cat, bg, out / "room.glb") - bundles = cat.fetch_key(bg) + cat.fetch_key(doc["master"]["_situationAssetPath"]) - s = shader.dump(bundles, out / "shaders") - r = { - "out": str(out), - "spot": {"tapTargets": len(doc["characters"]), "skeletons": len(doc["skeletons"])}, - "room": {"meshes": rm["meshCount"], "inactiveMeshes": rm["inactiveMeshes"]}, - "shaders": {"shaders": s["count"], "variants": s["variants"]}, - } - report( - args, - r, - lambda r: [ - f"{r['out']}/spot.json ({r['spot']['tapTargets']} tap targets, {r['spot']['skeletons']} skeletons)", - f"{r['out']}/room.glb ({r['room']['meshes']} meshes, {r['room']['inactiveMeshes']} inactive)", - f"{r['out']}/shaders/ ({r['shaders']['shaders']} shaders, {r['shaders']['variants']} variants)", - ], - ) + (row,) = [r for r in table(md, "MasterHomeSpot") if r.get("_id") == args.spot_id] + situation, background = row["_situationAssetPath"], row["_backgroundAssetPath"] + ws = assets.Workspace(args, cfg, layouts=["files"]) + ws.request({SPOT: [situation], ROOM: [background]}, [f"key:{situation}", f"key:{background}"]) + assets.export(args, cfg, ws) def cmd_room(args, cfg): from ..content import room + from ..pipeline.spot import ROOM - cat = Session(cfg).catalog() - known_keys(args, cat, [args.key]) - r = room.extract_room(cat, args.key, Path(args.out)) - report(args, r, lambda r: [f"{r['glb']} ({r['meshCount']} meshes, {r['inactiveMeshes']} inactive)"]) + try: + room.room_name(args.key) + except ValueError as e: + usage(args, str(e)) + use(cfg) + ws = assets.Workspace(args, cfg, layouts=["files"]) + ws.run_index() + if not any(loc["primaryKey"] == args.key for loc in ws.index()["locations"]): + usage(args, f"{args.key}: not a key of the catalog") + ws.request({ROOM: [args.key]}, [f"key:{args.key}"]) + assets.export(args, cfg, ws) def cmd_shader(args, cfg): @@ -240,6 +232,20 @@ def cmd_ui(args, cfg): sys.exit(1) +def product_args(c) -> None: + """The options of a command that exports products through the depot (as export assets).""" + c.add_argument( + "--stored-catalog", + metavar="LABEL|SHA", + help="a catalog version of the depot (`catalog versions`; default: the current catalog)", + ) + c.add_argument("--workers", type=nonnegative, metavar="N", help="worker processes (default: the usable CPUs)") + c.add_argument("--link", choices=("auto", "clone", "hard", "copy"), help="as export assets --link") + c.add_argument("--explain", action="store_true", help="print every setting of the run first") + c.add_argument("--dry-run", action="store_true", help="print the plan, run nothing") + c.set_defaults(views="none", strict=False) + + def register(sub) -> None: es = group(sub, "export", "game data -> common formats: the depot's asset export and single subjects") @@ -307,24 +313,17 @@ def register(sub) -> None: action="store_true", help="write the player's model (models/.json and assets/) instead of the Cubism SDK files", ) - c.add_argument( - "--stored-catalog", - metavar="LABEL|SHA", - help="a catalog version of the depot (`catalog versions`; default: the current catalog)", - ) - c.add_argument("--workers", type=nonnegative, metavar="N", help="worker processes (default: the usable CPUs)") - c.add_argument("--link", choices=("auto", "clone", "hard", "copy"), help="as export assets --link") - c.add_argument("--explain", action="store_true", help="print every setting of the run first") - c.add_argument("--dry-run", action="store_true", help="print the plan, run nothing") - c.set_defaults(views="none", strict=False) + product_args(c) - c = command(es, "spot", cmd_spot, "a spot -> spot.json + Spine + room.glb + shaders") + c = command(es, "spot", cmd_spot, "a spot -> spot.json + Spine files + room.glb + room.json") c.add_argument("spot_id", type=int, help="MasterHomeSpot id") out_arg(c, "output directory") + product_args(c) - c = command(es, "room", cmd_room, "a background prefab -> glb") - c.add_argument("key", help="addressable key of the prefab") - out_arg(c, "output .glb file") + c = command(es, "room", cmd_room, "a spot background -> room.glb + room.json") + c.add_argument("key", help="background key Spot//Background/") + out_arg(c, "output directory") + product_args(c) c = command(es, "shader", cmd_shader, "the shaders of a key's bundle closure or of APK bundles") g = c.add_mutually_exclusive_group(required=True) diff --git a/src/nnnotes/content/hostspot.py b/src/nnnotes/content/hostspot.py new file mode 100644 index 0000000..0f74f88 --- /dev/null +++ b/src/nnnotes/content/hostspot.py @@ -0,0 +1,372 @@ +"""The spot files a story's home host writes from the bundles of a spot (content.storyhost): the background as +binary glTF and the situation's spot.json with its Spine files, read with UnityPy from the bundle closures of the +spot's catalog keys. + +`extract_room` writes .glb and .json as content.room makes room.glb and room.json from a product's objects +(the same material translation, sampler and glTF writer): every MeshFilter of the background's bundle closure, in the +bundles' object order; the empty meshes listed apart (`skippedEmptyMeshes`: no vertex data, +`meshesWithoutTriangles`: vertices only). `extract_spot` writes spot.json and spine/ as content.spot makes them, +the skeletons in the bundles' object order, and `resources`: the bundle names of the background and situation keys. +""" + +from __future__ import annotations + +import io +import json +from pathlib import Path + +import numpy as np +import UnityPy + +from ..core.jsonio import write_json +from ..formats.shader import parsed_summary +from ..source.catalog import Catalog +from ..source.config import apk_missing +from ..unity.convert.material import material_doc +from ..unity.reader import SceneGraph, deref, mesh_arrays, referent, script_class, strip_pptrs, texture_image +from . import room +from .prefabs import mat4 +from .spot import LANGS, atlas_pages + +__all__ = [ + "drawn_meshes", + "extract_room", + "extract_spot", + "main_textures", + "mesh_components", + "object_key", + "submesh_primitives", + "text_bytes", +] + + +def object_key(obj) -> tuple[str, int]: + """(serialized file name, path id) of an object: a path id is unique only within its file.""" + return obj.assets_file.name, obj.path_id + + +def main_textures(owner, mat_tt: dict, shader_name: str) -> list: + """[(texture object, texture env)] of a material's main texture slot (room.main_tex_slot) when it holds a texture; + `owner` is the material's object, whose file the env's PPtr is read from.""" + slot = room.main_tex_slot(shader_name) + return [ + (deref(owner, env["m_Texture"]), env) + for key, env in mat_tt["m_SavedProperties"]["m_TexEnvs"] + if key == slot and env["m_Texture"]["m_PathID"] + ] + + +# ---- extraction --------------------------------------------------------- +def submesh_primitives(tris: list, mats: list, material_for, on_null) -> list: + """[(glTF triangle indices, material index)] of a mesh's submeshes: `tris` (per submesh, Unity winding), + `mats` the renderer's material PPtrs of the same index. An empty submesh gives nothing; a submesh whose material + slot is empty (path id 0) gives nothing and is reported to `on_null(submesh index)`.""" + prims = [] + for i, t in enumerate(tris): + if not len(t): + continue + if not mats[i].path_id: # empty slot: no material to draw the submesh with + on_null(i) + continue + prims.append((t[:, [0, 2, 1]].reshape(-1).astype(np.uint32), material_for(mats[i]))) + return prims + + +def mesh_components(env) -> tuple[dict, dict]: + """({GameObject key: MeshFilter}, {GameObject key: MeshRenderer}) of an environment. The key is (file name, + GameObject path id), as object_key: a component is in its GameObject's file.""" + mf_by_go, mr_by_go = {}, {} + for o in env.objects: + if o.type.name == "MeshFilter": + mf_by_go[(o.assets_file.name, o.read_typetree()["m_GameObject"]["m_PathID"])] = o + elif o.type.name == "MeshRenderer": + mr_by_go[(o.assets_file.name, o.read_typetree()["m_GameObject"]["m_PathID"])] = o + return mf_by_go, mr_by_go + + +def drawn_meshes(mf_by_go: dict, mr_by_go: dict, graph, on_null_mesh): + """(transform path id, Mesh, material PPtrs) of each GameObject with a MeshFilter and an enabled MeshRenderer + (mesh_components maps). A MeshFilter whose mesh reference is empty (path id 0) draws nothing: its object path + goes to `on_null_mesh(path)`.""" + for key, mf in mf_by_go.items(): + tf_pid = graph.tf_of_go.get(key[1]) + mr = mr_by_go.get(key) + if tf_pid is None or mr is None: + continue + if not mr.read_typetree().get("m_Enabled", 1): + continue + mesh_ref = mf.read().m_Mesh + if not mesh_ref.path_id: + on_null_mesh(graph.path(tf_pid)) + continue + yield tf_pid, mesh_ref.read(), mr.read().m_Materials + + +def extract_room(cat: Catalog, bg_key: str, out_path: Path) -> dict: + env = UnityPy.load(*[str(p) for p in cat.fetch_key(bg_key)]) + graph = SceneGraph(env) + mf_by_go, mr_by_go = mesh_components(env) + + glb = room.Glb() + tex_cache: dict[tuple[str, int], int] = {} # texture object_key -> glTF texture + mat_cache: dict[tuple[str, int], int] = {} # material object_key -> glTF material + + def material_for(pptr) -> int: + mat = pptr.deref() + mkey = object_key(mat) + if mkey in mat_cache: + return mat_cache[mkey] + mat_tt = mat.read_typetree() + shader_tt = referent(mat, mat_tt["m_Shader"]).read_typetree() + gm = room.translate_material(material_doc(mat_tt, lambda p: p), parsed_summary(shader_tt)) + for tex_obj, env_ in main_textures(mat, mat_tt, shader_tt["m_ParsedForm"]["m_Name"]): + tid = object_key(tex_obj) + if tid not in tex_cache: + tex_tt = tex_obj.read_typetree() + buf = io.BytesIO() + texture_image(tex_obj.read()).save(buf, format="PNG") + tex_cache[tid] = glb.texture(buf.getvalue(), tex_tt["m_Name"], room.sampler(tex_tt)) + info, transform = room.texture_info(tex_cache[tid], env_["m_Scale"], env_["m_Offset"]) + if transform: + glb.ext_used.add("KHR_texture_transform") + gm["pbrMetallicRoughness"]["baseColorTexture"] = info + mat_cache[mkey] = glb.material(gm) + return mat_cache[mkey] + + n_mesh = 0 + n_inactive = 0 + skipped = [] + null_slots = [] # submeshes whose material slot is empty + null_meshes = [] # objects whose MeshFilter has no mesh + no_triangles = [] # meshes with vertices but no triangles + for tf_pid, mesh, mats in drawn_meshes(mf_by_go, mr_by_go, graph, null_meshes.append): + arrays = mesh_arrays(mesh) + if arrays is None: + skipped.append(mesh.m_Name) # empty mesh (no vertex data) + continue + v, vn, uv0, tris = arrays + if not any(len(t) for t in tris): # vertices only (e.g. a placeholder mesh without index data) + no_triangles.append({"mesh": mesh.m_Name, "path": graph.path(tf_pid)}) + continue + if len(mats) < len(tris): + raise NotImplementedError(f"{mesh.m_Name}: {len(tris)} submeshes, {len(mats)} materials") + m = graph.world(tf_pid) + p = (np.hstack([v, np.ones((len(v), 1))]) @ m.T)[:, :3] + p[:, 2] *= -1.0 # Unity LH -> glTF RH + if len(vn): + nw = vn @ np.linalg.inv(m[:3, :3]).T + nw /= np.maximum(np.linalg.norm(nw, axis=1, keepdims=True), 1e-12) + nw[:, 2] *= -1.0 + else: + nw = np.empty((0, 3)) + uv = uv0.copy() + uv[:, 1] = 1.0 - uv[:, 1] # Unity UV origin bottom-left -> glTF top-left + prims = submesh_primitives( + tris, + mats, + material_for, + lambda i: null_slots.append({"mesh": mesh.m_Name, "path": graph.path(tf_pid), "submesh": i}), + ) + if not prims: + continue + is_active = graph.active(tf_pid) + glb.add_mesh(mesh.m_Name, p, nw, uv, prims, active=is_active) + n_mesh += 1 + n_inactive += 0 if is_active else 1 + + out_path = Path(out_path) + out_path.parent.mkdir(parents=True, exist_ok=True) + out_path.write_bytes(glb.bytes({"backgroundKey": bg_key})) + doc = { + "backgroundKey": bg_key, + "glb": out_path.name, + "meshCount": n_mesh, + "inactiveMeshes": n_inactive, + "skippedEmptyMeshes": skipped, + "nullMaterialSubmeshes": null_slots, + "nullMeshFilters": null_meshes, + "meshesWithoutTriangles": no_triangles, + **glb.summary(), + } + write_json(out_path.with_suffix(".json"), doc) + return doc + + +def _master(md: Path, name: str) -> list[dict]: + return json.loads((md / f"{name}.json").read_text(encoding="utf-8"))["_allData"] + + +def text_bytes(ta) -> bytes: + s = ta.m_Script + return s.encode("utf-8", "surrogateescape") if isinstance(s, str) else bytes(s) + + +def _target(ref): + """The object a PPtr refers to, None for an empty reference (path id 0).""" + return ref.read() if ref.path_id else None + + +def write_skeleton(o, write, written_atlases: dict[int, str] | None = None) -> dict: + """The Spine files of the SkeletonDataAsset object `o` (module docstring), each given to `write(file name, + bytes)` in this order: the skeleton, then for each of its atlases not in `written_atlases` ({SpineAtlasAsset + path id: atlas file name}, updated here) the atlas text as stored and one PNG per page. Returns the skeleton's + record of spot.json: name, skeleton file, atlas files, scale, default mix, mixes.""" + written = {} if written_atlases is None else written_atlases + sda = o.read_typetree() + sk = o.read().skeletonJSON.read() + raw = text_bytes(sk) + is_json = raw.lstrip()[:1] == b"{" + ext = ".json" if is_json else ".skel" + sk_file = sk.m_Name if sk.m_Name.endswith(ext) else f"{sk.m_Name}{ext}" # as the atlas below + write(sk_file, raw) + atlases = [] + for ap in o.read().atlasAssets: + atlas_asset = ap.read() + if ap.path_id not in written: + af = atlas_asset.atlasFile.read() + araw = text_bytes(af) + atext = araw.decode("utf-8") + a_file = af.m_Name if af.m_Name.endswith(".atlas") else f"{af.m_Name}.atlas" + write(a_file, araw) # the atlas text exactly as the game stores it + pages = atlas_pages(atext) + mats = list(atlas_asset.materials) + if len(mats) != len(pages): + raise RuntimeError(f"{a_file}: {len(pages)} pages vs {len(mats)} materials") + for page, mp in zip(pages, mats): + mat = mp.read() + tex = next(e.m_Texture.read() for k, e in mat.m_SavedProperties.m_TexEnvs if k == "_MainTex") + buf = io.BytesIO() + texture_image(tex).save(buf, format="PNG") + write(page, buf.getvalue()) + written[ap.path_id] = a_file + atlases.append(written[ap.path_id]) + return { + "name": sda["m_Name"], + "skeleton": sk_file, + "atlases": atlases, + "scale": sda["scale"], + "defaultMix": sda["defaultMix"], + "mixes": [ + {"from": f, "to": t, "duration": d} + for f, t, d in zip(sda.get("fromAnimation", []), sda.get("toAnimation", []), sda.get("duration", [])) + ], + } + + +def spine_characters(objs: list, graph, skeletons: dict, world_of) -> list[dict]: + """SpotSpineCharacter components: object path, the SkeletonAnimation they drive (its skeleton data name, + initial animation settings and world matrix). `skeletons`: SkeletonDataAsset path id -> skeleton record. One + whose `_animation` is empty has those three null.""" + out = [] + for o in objs: + ssc = o.read_typetree() + path = graph.path(graph.tf_of_go[ssc["m_GameObject"]["m_PathID"]]) + anim_o = _target(o.read()._animation) + if anim_o is None: + out.append({"path": path, "skeletonData": None, "animation": None, "world": None}) + continue + at = anim_o.object_reader.read_typetree() + out.append( + { + "path": path, + "skeletonData": skeletons[anim_o.skeletonDataAsset.path_id]["name"], + "animation": { + k: at[k] + for k in ( + "_animationName", + "loop", + "timeScale", + "initialSkinName", + "initialFlipX", + "initialFlipY", + "pmaVertexColors", + "tintBlack", + "zSpacing", + ) + }, + "world": world_of(at["m_GameObject"]), + } + ) + return out + + +def tap_targets(objs: list, graph, world_of) -> list[dict]: + """SpotCharacter components (tap targets) by character id: object path, their fields, world matrix and the world + matrix of their `_focus` transform (null when the reference is empty).""" + out = [] + for o in objs: + sc = o.read_typetree() + focus_o = _target(o.read()._focus) + out.append( + { + "path": graph.path(graph.tf_of_go[sc["m_GameObject"]["m_PathID"]]), + **strip_pptrs({k: v for k, v in sc.items() if k != "_focus"}), + "world": world_of(sc["m_GameObject"]), + "focusWorld": None + if focus_o is None + else world_of(focus_o.object_reader.read_typetree()["m_GameObject"]), + } + ) + out.sort(key=lambda c: c["_characterId"]) + return out + + +def extract_spot(cat: Catalog, md: Path, spot_id: int, out_dir: Path) -> dict: + if cat.apk is None: + raise apk_missing("resolving the Spot / Spine component classes") + spots = {r["_id"]: r for r in _master(md, "MasterHomeSpot")} + sp = spots[spot_id] + txt = {r["_id"]: r for r in _master(md, "MasterText")} + taps: dict[int, list[int]] = {} + for r in _master(md, "MasterStoryHomeSpotTapTalkEpisode"): + if r.get("_spotId") == spot_id: + taps.setdefault(r["_characterId"], []).append(r["_advId"]) + + out_dir = Path(out_dir) + spine_dir = out_dir / "spine" + spine_dir.mkdir(parents=True, exist_ok=True) + + env = UnityPy.load(*[str(p) for p in cat.fetch_key(sp["_situationAssetPath"])]) + graph = SceneGraph(env) + by_cls: dict[str | None, list] = {} + for o in env.objects: + if o.type.name == "MonoBehaviour": + by_cls.setdefault(script_class(o), []).append(o) + + def world_of(go_pptr) -> list[float]: + return mat4(graph.world_of_go(go_pptr["m_PathID"])) + + settings = strip_pptrs(by_cls["SpotSituationSettings"][0].read_typetree()) + controller_o = by_cls["SpotCharacterController"][0] + controller = strip_pptrs(controller_o.read_typetree()) + + # Spine skeleton data -> files + def write(name: str, data: bytes) -> None: + (spine_dir / name).write_bytes(data) + + skeleton_files: dict[int, dict] = {} + written_atlases: dict[int, str] = {} + for o in by_cls.get("SkeletonDataAsset", []): + skeleton_files[o.path_id] = write_skeleton(o, write, written_atlases) + + spine_chars = spine_characters(by_cls.get("SpotSpineCharacter", []), graph, skeleton_files, world_of) + characters = tap_targets(by_cls.get("SpotCharacter", []), graph, world_of) + + doc = { + "spotId": spot_id, + "master": sp, + "name": {lang[1:]: txt.get(sp.get("_nameTextId"), {}).get(lang, "") for lang in LANGS}, + "tapTalk": {str(k): v for k, v in sorted(taps.items())}, + "situationSettings": settings, + "controller": {k: v for k, v in controller.items() if k not in ("_spotSpineCharacters", "_characters")}, + "characters": characters, + "spineCharacters": spine_chars, + "skeletons": list(skeleton_files.values()), + "resources": { + "background": [b.name for b in cat.resolve(sp["_backgroundAssetPath"])], + "situation": [b.name for b in cat.resolve(sp["_situationAssetPath"])], + }, + "matrixConvention": "Unity world space (left-handed), 4x4 column-major", + } + write_json(out_dir / "spot.json", doc) + return doc diff --git a/src/nnnotes/content/live2d.py b/src/nnnotes/content/live2d.py index 5897b3e..471b03c 100644 --- a/src/nnnotes/content/live2d.py +++ b/src/nnnotes/content/live2d.py @@ -37,14 +37,13 @@ import re from collections.abc import Iterator -from typing import Any, Protocol from ..core.jsonio import dumps from ..formats import motion from ..formats.cubism import expression_json, moc3_canvas, physics_json from ..formats.shader import player_program from ..unity import bindings -from ..unity.ids import object_id +from .prefabs import Objects, component_class, ref_id, root_game_object, script_class __all__ = [ "MASK_KEYWORD", @@ -83,22 +82,6 @@ _UNSAFE = re.compile(r"[^A-Za-z0-9_.-]+") -class Objects(Protocol): - """A product's object table and its objects' files (pipeline.assemble.Objects).""" - - roots: list[str] - - def __contains__(self, oid: str) -> bool: ... - - def ids(self, cls: str | None = None) -> list[str]: ... - - def info(self, oid: str) -> dict: ... - - def bytes(self, oid: str, role: str) -> bytes: ... - - def json(self, oid: str, role: str = "json") -> Any: ... - - def model_match(key: str) -> re.Match[str]: """The parts of a model key: group, name (ValueError for a key that is not a model's).""" m = MODEL_KEY.fullmatch(key) @@ -148,28 +131,12 @@ def select_models(specs, models: dict[str, str]) -> dict[str, str]: return dict(sorted(out.items())) -def ref_id(v) -> str | None: - """The object id of a reference value ({"$ref": {file, pathId}}); None for anything else.""" - r = v.get("$ref") if isinstance(v, dict) else None - return object_id(r["file"], r["pathId"]) if isinstance(r, dict) and "file" in r else None - - -def _class(c: dict) -> str: - """A component's class: a MonoBehaviour's is its script's.""" - if c["type"] == "MonoBehaviour" and isinstance(c.get("script"), str): - return c["script"].rsplit("|", 1)[-1] - return c["type"] - - class _Model: """A model's GameObject, prefab document and components.""" def __init__(self, objects: Objects, key: str): self.objects, self.key, self.name = objects, key, model_id(key) - gos = [o for o in objects.roots if o in objects and objects.info(o).get("class") == "GameObject"] - if len(gos) != 1: - raise ValueError(f"{key}: {len(gos)} GameObjects among the roots, expected the model's") - self.id = gos[0] + self.id = root_game_object(objects, key) self.prefab = objects.json(self.id) self.root = self.prefab["nodes"][0] @@ -177,7 +144,7 @@ def components(self, cls: str, root_only: bool = False) -> Iterator[tuple[dict, """(node, component) of every component of class `cls` (of the root node only).""" for n in [self.root] if root_only else self.prefab["nodes"]: for c in n["components"]: - if _class(c) == cls: + if component_class(c) == cls: yield n, c def one(self, cls: str) -> dict: @@ -213,14 +180,7 @@ def drawable_materials(self) -> list[str]: def scripted(self, cls: str) -> list[str]: """The MonoBehaviours of the table whose script class is `cls` (outside the prefab: data assets).""" - return [o for o in self.objects.ids("MonoBehaviour") if _script_class(self.objects, o) == cls] - - -def _script_class(objects: Objects, oid: str) -> str | None: - if "json" not in (objects.info(oid).get("files") or {}): - return None - entry = objects.json(oid).get("$script") - return entry.rsplit("|", 1)[-1] if isinstance(entry, str) else None + return [o for o in self.objects.ids("MonoBehaviour") if script_class(self.objects, o) == cls] def _resources(m: _Model) -> dict[str, str]: @@ -274,7 +234,7 @@ def model(objects: Objects, key: str) -> tuple[dict, set[str], dict[str, set[str except ValueError as e: raise ValueError(f"{key}: shader {index['name']}: {e}") from None roles[sid] = keep - classes = {_class(c) for c in m.root["components"]} + classes = {component_class(c) for c in m.root["components"]} def material(ref) -> dict | None: oid = ref_id(ref) diff --git a/src/nnnotes/content/prefabs.py b/src/nnnotes/content/prefabs.py new file mode 100644 index 0000000..3bd73da --- /dev/null +++ b/src/nnnotes/content/prefabs.py @@ -0,0 +1,148 @@ +"""Prefab documents of a product's object table read as the game's hierarchy. + +A root GameObject's `json` is its prefab document (prefab.json): `{name, nodes}`, the nodes in hierarchy order (depth +first), each with its path, local transform, `active` (GameObject.m_IsActive) and components `{type, pathId, script, +fields}`. A component's fields name objects of the table by reference (`{"$ref": {file, pathId}}`) and objects of the +same hierarchy by path: `{"gameObject": path}`, `{"transform": path}`, or `{"component": type, "gameObject": path, +"class": script class}` for a component. `Prefab` gives each node's parent, its world matrix (Unity's left-handed +space, the root's own transform included) and whether it is active in the hierarchy, and resolves those references. +""" + +from __future__ import annotations + +from collections.abc import Iterator +from typing import TYPE_CHECKING, Any, Protocol + +from ..unity.ids import object_id + +if TYPE_CHECKING: + import numpy as np + +__all__ = ["Objects", "Prefab", "component_class", "mat4", "ref_id", "root_game_object", "script_class"] + + +class Objects(Protocol): + """A product's object table and its objects' files (pipeline.assemble.Objects).""" + + roots: list[str] + + def __contains__(self, oid: str) -> bool: ... + + def ids(self, cls: str | None = None) -> list[str]: ... + + def info(self, oid: str) -> dict: ... + + def bytes(self, oid: str, role: str) -> bytes: ... + + def json(self, oid: str, role: str = "json") -> Any: ... + + +def ref_id(v) -> str | None: + """The object id of a reference value ({"$ref": {file, pathId}}); None for anything else.""" + r = v.get("$ref") if isinstance(v, dict) else None + return object_id(r["file"], r["pathId"]) if isinstance(r, dict) and "file" in r else None + + +def component_class(c: dict) -> str: + """A component's class: a MonoBehaviour's is its script's (the last part of its script entry).""" + if c["type"] == "MonoBehaviour" and isinstance(c.get("script"), str): + return c["script"].rsplit("|", 1)[-1] + return c["type"] + + +def script_class(objects: Objects, oid: str) -> str | None: + """The script class of a MonoBehaviour outside a hierarchy (its `json`'s `$script`), None without one.""" + if "json" not in (objects.info(oid).get("files") or {}): + return None + entry = objects.json(oid).get("$script") + return entry.rsplit("|", 1)[-1] if isinstance(entry, str) else None + + +def root_game_object(objects: Objects, what: str) -> str: + """The one GameObject among the table's roots (ValueError naming `what` otherwise).""" + gos = [o for o in objects.roots if o in objects and objects.info(o).get("class") == "GameObject"] + if len(gos) != 1: + raise ValueError(f"{what}: {len(gos)} GameObjects among the roots, expected one") + return gos[0] + + +def mat4(m: np.ndarray) -> list[float]: + """A 4x4 matrix as 16 floats, column-major.""" + return [float(x) for x in m.T.reshape(-1)] + + +class Prefab: + """The nodes of a prefab document with their parents, world matrices and activity (module documentation).""" + + def __init__(self, doc: dict): + self.name: str = doc["name"] + self.nodes: list[dict] = doc["nodes"] + self.parent: list[int | None] = [] + stack: list[int] = [] + for n in self.nodes: + while stack and not n["path"].startswith(self.nodes[stack[-1]]["path"] + "/"): + stack.pop() + self.parent.append(stack[-1] if stack else None) + stack.append(len(self.parent) - 1) + self._by_path: dict[str, list[int]] = {} + for i, n in enumerate(self.nodes): + self._by_path.setdefault(n["path"], []).append(i) + self._world: dict[int, np.ndarray] = {} # numpy is imported when a world matrix is asked for + + def at(self, path: str) -> int: + """The node at `path` (ValueError when no node, or several, has it).""" + hits = self._by_path.get(path, []) + if len(hits) != 1: + raise ValueError(f"{self.name}: {len(hits)} nodes at {path!r}") + return hits[0] + + def world(self, i: int) -> np.ndarray: + """The world matrix of node `i`: its local transform under its parents'.""" + from ..unity.transform import trs + + if i not in self._world: + n = self.nodes[i] + m = trs(n["localPosition"], n["localRotation"], n["localScale"]) + p = self.parent[i] + self._world[i] = m if p is None else self.world(p) @ m + return self._world[i] + + def active(self, i: int | None) -> bool: + """Node `i` and every parent of it are active.""" + while i is not None: + if not self.nodes[i].get("active", True): + return False + i = self.parent[i] + return True + + def components(self, cls: str) -> Iterator[tuple[int, dict]]: + """(node, component) of every component of class `cls`, in hierarchy order.""" + for i, n in enumerate(self.nodes): + for c in n["components"]: + if component_class(c) == cls: + yield i, c + + def node_of(self, v) -> int | None: + """The node a reference to an object of the hierarchy names (its GameObject, Transform or a component of + it); None for null.""" + if v is None: + return None + if not isinstance(v, dict) or not ({"gameObject", "transform"} & v.keys()): + raise ValueError(f"{self.name}: not a reference to an object of the hierarchy: {v!r}") + return self.at(v["gameObject"] if "gameObject" in v else v["transform"]) + + def component_of(self, v) -> tuple[int, dict] | None: + """(node, component) a reference to a component of the hierarchy names; None for null.""" + i = self.node_of(v) + if i is None: + return None + if "component" not in v: + raise ValueError(f"{self.name}: not a reference to a component: {v!r}") + hits = [ + c + for c in self.nodes[i]["components"] + if c["type"] == v["component"] and ("class" not in v or component_class(c) == v["class"]) + ] + if len(hits) != 1: + raise ValueError(f"{self.name}: {len(hits)} components {v!r}") + return i, hits[0] diff --git a/src/nnnotes/content/room.py b/src/nnnotes/content/room.py index e8ba1ac..f3ce16e 100644 --- a/src/nnnotes/content/room.py +++ b/src/nnnotes/content/room.py @@ -1,49 +1,45 @@ -"""Spot background prefab -> binary glTF (.glb). - -A spot background is a few hundred textured cards (MeshFilter + MeshRenderer -under a Transform hierarchy). Each mesh is baked into prefab space by walking -the Transform TRS chain, converted from Unity's left-handed space to glTF's -right-handed space (negate Z, reverse winding), and written with its texture(s) -embedded. Geometry is read from the mesh's vertex/index buffers; each submesh -becomes one primitive with the renderer's material of the same index. - -Objects are told apart by (serialized file name, path id) (object_key): a path -id is unique only within its file, and a bundle's closure loads several files. -Each Unity texture is one glTF texture with its own sampler; textures whose PNG -bytes are equal share one glTF image. - -Materials are translated from the Unity material's shader and that shader's own -pass render state (culling, blend factors and operations, depth write and test, -colour mask, alpha to mask, queue tags) -- not from names. A render state value -that names a property (URP/Lit's `Blend [_SrcBlend] [_DstBlend]`, `Cull [_Cull]`, -`ZWrite [_ZWrite]`) takes the material's value of that property, else the -shader's default for it, else the value the shader stores (render_state). Each -glTF material records the resolved state in `extras.unityRenderState`. -Unsupported shaders and states raise instead of being approximated. A submesh -whose material slot is empty (null) is not written: nothing says what it would -be drawn with; room.json lists it (`nullMaterialSubmeshes`). A MeshFilter with -no mesh, or a mesh with no triangles, draws nothing; room.json lists the object -(`nullMeshFilters`, `meshesWithoutTriangles`). - -The situation's backgroundPosition/Rotation/Scale (spot.json) is not baked in: it -goes on top of this, as the game applies it to the prefab root. Objects that -are inactive in the prefab are kept, flagged `extras.unityActive = false`. +"""A spot background (a room) assembled from the objects its prefab reaches: binary glTF (.glb) and its summary. + +A spot background is a few hundred textured cards: MeshFilter and MeshRenderer components under a Transform +hierarchy. `room` reads them from the prefab document of the background's GameObject (content.prefabs) and the +objects its components name: each Mesh's `mesh` (binary glTF in the mesh's local space, unity.atoms.gltf), each +Material's `json` and its Shader's parsed form, each Texture2D's `json` and `image`. It never reads a bundle. + +Each drawn mesh (a MeshFilter with a mesh beside an enabled MeshRenderer) becomes one node without a transform: its +vertices are baked into the prefab's space through the world matrix of its node (the root's own transform included), +in glTF's right-handed space (z negated), with Unity's texture coordinates flipped (v = 1 - v), as the mesh's glTF +holds them. Each non-empty submesh becomes one primitive with the renderer's material of the same index. A node +inactive in the prefab is kept, flagged `extras.unityActive = false`. + +Materials are translated from the Unity material's shader and that shader's own pass render state (culling, blend +factors and operations, depth write and test, colour mask, alpha to mask, queue tags), not from names. A render +state value that names a property (URP/Lit's `Blend [_SrcBlend] [_DstBlend]`, `Cull [_Cull]`, `ZWrite [_ZWrite]`) +takes the material's value of that property, else the shader's default for it, else the value the shader stores +(render_state). Each glTF material records the resolved state in `extras.unityRenderState`. Unsupported shaders and +states raise instead of being approximated. Each Texture2D a material's main slot names is one glTF texture with its +own sampler; textures whose PNG bytes are equal share one glTF image. + +What draws nothing is not written and room.json lists it: a submesh whose material slot is empty +(`nullMaterialSubmeshes`), a MeshFilter without a mesh (`nullMeshFilters`), a mesh without vertices or triangles +(`emptyMeshes`). + +The situation's backgroundPosition, backgroundRotation and backgroundScale (spot.json) are not baked in: they go on +top of this, as the game applies them to the prefab root. """ from __future__ import annotations import hashlib -import io +import re import struct -from pathlib import Path -import numpy as np -import UnityPy +from ..core.jsonio import dumps +from .prefabs import Objects, Prefab, component_class, ref_id, root_game_object -from ..core.jsonio import dumps, write_json -from ..source.catalog import Catalog -from ..unity.reader import SceneGraph, deref, mesh_arrays, referent, texture_image +__all__ = ["BACKGROUND_KEY", "room", "room_name", "render_state", "translate_material"] +# a spot background's catalog key: Spot//Background/ +BACKGROUND_KEY = r"Spot/(?P[^/]+)/Background/(?P=place)" # UnityEngine.Rendering.BlendMode BLEND_ONE, BLEND_ZERO, BLEND_SRC_ALPHA, BLEND_ONE_MINUS_SRC_ALPHA = 1, 0, 5, 10 BLEND_OP_ADD = 0 # UnityEngine.Rendering.BlendOp.Add @@ -55,7 +51,7 @@ # UnityEngine.TextureWrapMode WRAP = {0: REPEAT, 1: CLAMP, 2: MIRROR, 3: MIRROR} UNLIT_SHADERS = ("Unlit/", "Universal Render Pipeline/Unlit") -# render state of a pass (SerializedShaderState) read by room: record key -> path in m_State (render target 0) +# render state of a pass (SerializedShaderState) read by room: record key -> path in the pass state (render target 0) STATE_FIELDS = { "srcBlend": ("rtBlend0", "srcBlend"), "dstBlend": ("rtBlend0", "destBlend"), @@ -70,19 +66,28 @@ "alphaToMask": ("alphaToMask",), } NO_PROPERTY = "" # SerializedShaderFloatValue.name of a fixed value +TRIANGLES = 4 # glTF primitive mode + + +def room_name(key: str) -> str: + """The of a background key (ValueError for another key).""" + m = re.fullmatch(BACKGROUND_KEY, key) + if m is None: + raise ValueError(f"{key}: not a spot background key (Spot//Background/)") + return m["place"] # ---- material translation ------------------------------------------------- -def shader_defaults(shader_tt: dict) -> dict: - """{property name: default value} of a shader's properties (m_ParsedForm.m_PropInfo, first component).""" - props = shader_tt["m_ParsedForm"].get("m_PropInfo", {}).get("m_Props", []) - return {p["m_Name"]: p["m_DefValue[0]"] for p in props if "m_DefValue[0]" in p} +# A material is its `json` (material.json: material, shader, keywords, floats, colors, ints, textures), a shader its +# parsed form (name, properties, subShaders[].tags, subShaders[].passes[].tags and state). +def shader_defaults(shader: dict) -> dict: + """{property name: default value} of a shader's properties (the first component).""" + return {p["m_Name"]: p["m_DefValue[0]"] for p in shader.get("properties") or () if "m_DefValue[0]" in p} -def material_values(mat_tt: dict) -> dict: +def material_values(material: dict) -> dict: """{property name: value} of a material's int and float properties.""" - sp = mat_tt["m_SavedProperties"] - return {**dict(sp.get("m_Ints", [])), **dict(sp.get("m_Floats", []))} + return {**(material.get("ints") or {}), **(material.get("floats") or {})} def state_value(x, values: dict, defaults: dict): @@ -98,10 +103,19 @@ def state_value(x, values: dict, defaults: dict): return int(v) if isinstance(v, float) and v.is_integer() else v -def render_state(mat_tt: dict, shader_tt: dict) -> dict: +def _pass_state(shader: dict) -> tuple[dict, dict]: + """(render state of the first pass, merged subshader and pass tags).""" + ss = shader["subShaders"][0] + ps = ss["passes"][0] + tags = dict((ss.get("tags") or {}).get("tags", [])) + tags.update(dict((ps.get("tags") or {}).get("tags", []))) + return ps["state"], tags + + +def render_state(material: dict, shader: dict) -> dict: """The render state (STATE_FIELDS) of the shader's first pass for this material.""" - state, _ = _pass_state(shader_tt) - values, defaults = material_values(mat_tt), shader_defaults(shader_tt) + state, _ = _pass_state(shader) + values, defaults = material_values(material), shader_defaults(shader) out = {} for key, path in STATE_FIELDS.items(): x = state @@ -111,29 +125,20 @@ def render_state(mat_tt: dict, shader_tt: dict) -> dict: return out -def _pass_state(shader_tt: dict) -> tuple[dict, dict]: - """(render state of the first pass, merged subshader+pass tags).""" - ss = shader_tt["m_ParsedForm"]["m_SubShaders"][0] - ps = ss["m_Passes"][0] - tags = dict(ss.get("m_Tags", {}).get("tags", [])) - tags.update(dict(ps.get("m_Tags", {}).get("tags", []))) - return ps["m_State"], tags - - -def translate_material(mat_tt: dict, shader_tt: dict) -> dict: - """Unity material + shader -> glTF material fields (texture filled by caller).""" - shader_name = shader_tt["m_ParsedForm"]["m_Name"] - _, tags = _pass_state(shader_tt) - rs = render_state(mat_tt, shader_tt) +def translate_material(material: dict, shader: dict) -> dict: + """A Unity material and its shader -> glTF material fields (the texture is the caller's).""" + shader_name = shader["name"] + _, tags = _pass_state(shader) + rs = render_state(material, shader) src, dst, cull = rs["srcBlend"], rs["dstBlend"], rs["cull"] - floats = dict(mat_tt["m_SavedProperties"]["m_Floats"]) - colors = dict(mat_tt["m_SavedProperties"]["m_Colors"]) + floats = material.get("floats") or {} + colors = material.get("colors") or {} unlit = shader_name.startswith(UNLIT_SHADERS) if not unlit and shader_name != "Universal Render Pipeline/Lit": raise NotImplementedError(f"shader not translated: {shader_name}") - out: dict = {"name": mat_tt["m_Name"], "extras": {"unityShader": shader_name, "unityRenderState": rs}} + out: dict = {"name": material["material"], "extras": {"unityShader": shader_name, "unityRenderState": rs}} if (src, dst) == (BLEND_ONE, BLEND_ZERO): cutout = tags.get("RenderType") == "TransparentCutout" or tags.get("QUEUE") == "AlphaTest" if cutout: @@ -164,29 +169,27 @@ def translate_material(mat_tt: dict, shader_tt: dict) -> dict: return out -def _main_tex_slot(shader_name: str) -> str: +def main_tex_slot(shader_name: str) -> str: + """The texture property a shader samples as its main texture.""" return "_BaseMap" if shader_name.startswith("Universal Render Pipeline/") else "_MainTex" -def object_key(obj) -> tuple[str, int]: - """(serialized file name, path id) of an object: a path id is unique only within its file.""" - return obj.assets_file.name, obj.path_id - +def texture_info(index: int, scale: dict, offset: dict) -> tuple[dict, bool]: + """(glTF texture info of texture `index`, whether it uses KHR_texture_transform) for a texture slot's scale and + offset.""" + info: dict = {"index": index} + sx, sy, ox, oy = scale["x"], scale["y"], offset["x"], offset["y"] + if (sx, sy, ox, oy) == (1.0, 1.0, 0.0, 0.0): + return info, False + # Unity samples (u*sx+ox, v*sy+oy) with a bottom-left origin; in glTF's top-left texture space that is this + info["extensions"] = {"KHR_texture_transform": {"scale": [sx, sy], "offset": [ox, 1.0 - sy - oy]}} + return info, True -def main_textures(owner, mat_tt: dict, shader_name: str) -> list: - """[(texture object, texture env)] of a material's main texture slot (_main_tex_slot) when it holds a texture; - `owner` is the material's object, whose file the env's PPtr is read from.""" - slot = _main_tex_slot(shader_name) - return [ - (deref(owner, env["m_Texture"]), env) - for key, env in mat_tt["m_SavedProperties"]["m_TexEnvs"] - if key == slot and env["m_Texture"]["m_PathID"] - ] - -def _sampler(tex_tt: dict) -> dict: - s = tex_tt.get("m_TextureSettings", {}) - mips = tex_tt.get("m_MipCount", 1) > 1 +def sampler(texture: dict) -> dict: + """The glTF sampler of a Texture2D's settings and mip count (its typetree fields).""" + s = texture.get("m_TextureSettings", {}) + mips = texture.get("m_MipCount", 1) > 1 fm = s.get("m_FilterMode", 1) mag = NEAREST if fm == 0 else LINEAR if fm == 0: @@ -202,7 +205,9 @@ def _sampler(tex_tt: dict) -> dict: # ---- glTF builder ------------------------------------------------------- -class _Glb: +class Glb: + """A binary glTF of baked meshes: one node per mesh, one primitive per submesh.""" + def __init__(self): self.bin = bytearray() self.bufferViews, self.accessors, self.meshes, self.nodes = [], [], [], [] @@ -220,7 +225,9 @@ def _view(self, data: bytes, target=None) -> int: self.bufferViews.append(bv) return len(self.bufferViews) - 1 - def accessor_f32(self, arr: np.ndarray, kind: str, minmax=False) -> int: + def accessor_f32(self, arr, kind: str, minmax=False) -> int: + import numpy as np + a = np.ascontiguousarray(arr, dtype=" int: self.accessors.append(acc) return len(self.accessors) - 1 - def accessor_idx(self, idx: np.ndarray) -> int: + def accessor_idx(self, idx) -> int: + import numpy as np + a = np.ascontiguousarray(idx, dtype=" None: attrs["NORMAL"] = self.accessor_f32(nrm, "VEC3") primitives = [] for idx, material in prims: - p = {"attributes": attrs, "indices": self.accessor_idx(idx), "mode": 4} + p = {"attributes": attrs, "indices": self.accessor_idx(idx), "mode": TRIANGLES} if material is not None: p["material"] = material primitives.append(p) self.meshes.append({"name": name, "primitives": primitives}) self.nodes.append({"name": name, "mesh": len(self.meshes) - 1, "extras": {"unityActive": active}}) - def write(self, path: Path, extras: dict): + def bytes(self, extras: dict) -> bytes: """glTF requires the arrays it has to be non-empty and a buffer to hold bytes: an empty array is left out, and a room without geometry is a scene without nodes and a file without the BIN chunk.""" scene = {"nodes": list(range(len(self.nodes)))} if self.nodes else {} @@ -300,166 +309,174 @@ def write(self, path: Path, extras: dict): gltf["extensionsUsed"] = sorted(self.ext_used) js = dumps(gltf, separators=(",", ":")).encode("utf-8") js += b" " * (-len(js) % 4) - self.bin.extend(b"\0" * (-len(self.bin) % 4)) - size = 12 + 8 + len(js) + (8 + len(self.bin) if self.bin else 0) - with open(path, "wb") as f: - f.write(struct.pack(" list: - """[(glTF triangle indices, material index)] of a mesh's submeshes: `tris` (per submesh, Unity winding), - `mats` the renderer's material PPtrs of the same index. An empty submesh gives nothing; a submesh whose material - slot is empty (path id 0) gives nothing and is reported to `on_null(submesh index)`.""" - prims = [] - for i, t in enumerate(tris): - if not len(t): - continue - if not mats[i].path_id: # empty slot: no material to draw the submesh with - on_null(i) - continue - prims.append((t[:, [0, 2, 1]].reshape(-1).astype(np.uint32), material_for(mats[i]))) - return prims - - -def mesh_components(env) -> tuple[dict, dict]: - """({GameObject key: MeshFilter}, {GameObject key: MeshRenderer}) of an environment. The key is (file name, - GameObject path id), as object_key: a component is in its GameObject's file.""" - mf_by_go, mr_by_go = {}, {} - for o in env.objects: - if o.type.name == "MeshFilter": - mf_by_go[(o.assets_file.name, o.read_typetree()["m_GameObject"]["m_PathID"])] = o - elif o.type.name == "MeshRenderer": - mr_by_go[(o.assets_file.name, o.read_typetree()["m_GameObject"]["m_PathID"])] = o - return mf_by_go, mr_by_go - - -def drawn_meshes(mf_by_go: dict, mr_by_go: dict, graph, on_null_mesh): - """(transform path id, Mesh, material PPtrs) of each GameObject with a MeshFilter and an enabled MeshRenderer - (mesh_components maps). A MeshFilter whose mesh reference is empty (path id 0) draws nothing: its object path - goes to `on_null_mesh(path)`.""" - for key, mf in mf_by_go.items(): - tf_pid = graph.tf_of_go.get(key[1]) - mr = mr_by_go.get(key) - if tf_pid is None or mr is None: + body = bytes(self.bin) + b"\0" * (-len(self.bin) % 4) + size = 12 + 8 + len(js) + (8 + len(body) if body else 0) + out = struct.pack(" dict: + """The materials, textures and samplers as room.json lists them.""" + return { + "materials": [ + { + "name": m["name"], + "shader": m["extras"]["unityShader"], + "alphaMode": m.get("alphaMode", "OPAQUE"), + "alphaCutoff": m.get("alphaCutoff"), + "doubleSided": m.get("doubleSided", False), + "unlit": "KHR_materials_unlit" in m.get("extensions", {}), + "renderState": m["extras"]["unityRenderState"], + } + for m in self.materials + ], + "textures": [t["name"] for t in self.textures], + "samplers": self.samplers, + } + + +# ---- assembly ------------------------------------------------------------- +def _floats(doc: dict, body: bytes, index: int): + """The values of a glTF accessor as float32, a normalized integer accessor dequantized.""" + import numpy as np + + from ..unity.atoms.gltf import accessor_array + + a = accessor_array(doc, body, index) + if doc["accessors"][index].get("normalized"): + return (a.astype(np.float64) / np.iinfo(a.dtype).max).astype(np.float32) + return a.astype(np.float32) + + +class _Mesh: + """A Mesh object's local-space glTF: positions and normals back in Unity's space (z as the game stores it), + texture coordinates as the glTF holds them (v = 1 - v), and per submesh its triangle indices (glTF winding) or + None for an empty submesh; no submeshes for a mesh without an index buffer (one points primitive).""" + + def __init__(self, objects: Objects, oid: str): + from ..unity.atoms.gltf import accessor_array, read + + doc, body = read(objects.bytes(oid, "mesh")) + (mesh,) = doc["meshes"] + prims = mesh["primitives"] + attrs = prims[0]["attributes"] + self.name = mesh.get("name") or objects.info(oid).get("name") or oid + self.pos = _floats(doc, body, attrs["POSITION"])[:, :3].copy() + self.pos[:, 2] *= -1.0 + self.nrm = _floats(doc, body, attrs["NORMAL"])[:, :3].copy() if "NORMAL" in attrs else None + if self.nrm is not None: + self.nrm[:, 2] *= -1.0 + self.uv = _floats(doc, body, attrs["TEXCOORD_0"])[:, :2] if "TEXCOORD_0" in attrs else None + unity = (mesh.get("extras") or {}).get("unity") or {} + empty = set(unity.get("emptySubmeshes") or ()) + count = len(unity.get("submeshes") or ()) + self.submeshes: list = [] + if any("indices" not in p for p in prims): + return + it = iter(prims) + for i in range(count): + if i in empty: + self.submeshes.append(None) + continue + p = next(it) + if p.get("mode", TRIANGLES) != TRIANGLES or "indices" not in p: + raise NotImplementedError(f"mesh {self.name!r}: submesh {i} is not a triangle list") + self.submeshes.append(accessor_array(doc, body, p["indices"]).reshape(-1)) + if next(it, None) is not None: + raise ValueError(f"mesh {self.name!r}: more primitives than non-empty submeshes") + + +def room(objects: Objects, key: str) -> tuple[bytes, dict]: + """(room.glb, room.json) of the background `key` (module documentation).""" + import numpy as np + + prefab = Prefab(objects.json(root_game_object(objects, key))) + glb = Glb() + textures: dict[str, int] = {} # Texture2D object id -> glTF texture + materials: dict[str, int] = {} # Material object id -> glTF material + + def material_for(oid: str) -> int: + if oid in materials: + return materials[oid] + mat = objects.json(oid) + sid = ref_id(mat["shader"]) + if sid is None or sid not in objects: + raise ValueError(f"{key}: material {mat['material']!r} has no shader in the object table") + shader = objects.json(sid) + gm = translate_material(mat, shader) + slot = (mat.get("textures") or {}).get(main_tex_slot(shader["name"])) + tid = ref_id(slot["texture"]) if slot else None + if slot and tid is not None: + if tid not in textures: + tex = objects.json(tid) + textures[tid] = glb.texture(objects.bytes(tid, "image"), tex["m_Name"], sampler(tex)) + info, transform = texture_info(textures[tid], slot["scale"], slot["offset"]) + if transform: + glb.ext_used.add("KHR_texture_transform") + gm["pbrMetallicRoughness"]["baseColorTexture"] = info + materials[oid] = glb.material(gm) + return materials[oid] + + counts = {"meshes": 0, "inactive": 0} + null_slots, null_meshes, empty = [], [], [] + for i, node in enumerate(prefab.nodes): + comps = {component_class(c): c["fields"] for c in node["components"]} + mf, mr = comps.get("MeshFilter"), comps.get("MeshRenderer") + if mf is None or mr is None or not mr.get("m_Enabled", 1): continue - if not mr.read_typetree().get("m_Enabled", 1): + path = node["path"] + mesh_id = ref_id(mf.get("m_Mesh")) + if mesh_id is None: + null_meshes.append(path) continue - mesh_ref = mf.read().m_Mesh - if not mesh_ref.path_id: - on_null_mesh(graph.path(tf_pid)) + info = objects.info(mesh_id) + if "mesh" not in (info.get("files") or {}): + if info.get("reason") != "empty.mesh": + raise ValueError(f"{key}: {path}: mesh {info.get('name')!r} was not converted ({info.get('reason')})") + empty.append({"mesh": info.get("name"), "path": path}) continue - yield tf_pid, mesh_ref.read(), mr.read().m_Materials - - -def extract_room(cat: Catalog, bg_key: str, out_path: Path) -> dict: - env = UnityPy.load(*[str(p) for p in cat.fetch_key(bg_key)]) - graph = SceneGraph(env) - mf_by_go, mr_by_go = mesh_components(env) - - glb = _Glb() - tex_cache: dict[tuple[str, int], int] = {} # texture object_key -> glTF texture - mat_cache: dict[tuple[str, int], int] = {} # material object_key -> glTF material - - def material_for(pptr) -> int: - mat = pptr.deref() - mkey = object_key(mat) - if mkey in mat_cache: - return mat_cache[mkey] - mat_tt = mat.read_typetree() - shader_tt = referent(mat, mat_tt["m_Shader"]).read_typetree() - gm = translate_material(mat_tt, shader_tt) - for tex_obj, env_ in main_textures(mat, mat_tt, shader_tt["m_ParsedForm"]["m_Name"]): - tid = object_key(tex_obj) - if tid not in tex_cache: - tex_tt = tex_obj.read_typetree() - buf = io.BytesIO() - texture_image(tex_obj.read()).save(buf, format="PNG") - tex_cache[tid] = glb.texture(buf.getvalue(), tex_tt["m_Name"], _sampler(tex_tt)) - info: dict = {"index": tex_cache[tid]} - sx, sy = env_["m_Scale"]["x"], env_["m_Scale"]["y"] - ox, oy = env_["m_Offset"]["x"], env_["m_Offset"]["y"] - if (sx, sy, ox, oy) != (1.0, 1.0, 0.0, 0.0): - # Unity samples (u*sx+ox, v*sy+oy) with a bottom-left origin; - # in glTF's top-left texture space that is this transform. - info["extensions"] = {"KHR_texture_transform": {"scale": [sx, sy], "offset": [ox, 1.0 - sy - oy]}} - glb.ext_used.add("KHR_texture_transform") - gm["pbrMetallicRoughness"]["baseColorTexture"] = info - mat_cache[mkey] = glb.material(gm) - return mat_cache[mkey] - - n_mesh = 0 - n_inactive = 0 - skipped = [] - null_slots = [] # submeshes whose material slot is empty - null_meshes = [] # objects whose MeshFilter has no mesh - no_triangles = [] # meshes with vertices but no triangles - for tf_pid, mesh, mats in drawn_meshes(mf_by_go, mr_by_go, graph, null_meshes.append): - arrays = mesh_arrays(mesh) - if arrays is None: - skipped.append(mesh.m_Name) # empty mesh (no vertex data) + mesh = _Mesh(objects, mesh_id) + if not any(tri is not None and len(tri) for tri in mesh.submeshes): + empty.append({"mesh": mesh.name, "path": path}) continue - v, vn, uv0, tris = arrays - if not any(len(t) for t in tris): # vertices only (e.g. a placeholder mesh without index data) - no_triangles.append({"mesh": mesh.m_Name, "path": graph.path(tf_pid)}) + mats = mr.get("m_Materials") or [] + if len(mats) < len(mesh.submeshes): + raise NotImplementedError(f"{mesh.name}: {len(mesh.submeshes)} submeshes, {len(mats)} materials") + prims = [] + for s, tri in enumerate(mesh.submeshes): + if tri is None or not len(tri): + continue + mid = ref_id(mats[s]) + if mid is None: # empty slot: no material to draw the submesh with + null_slots.append({"mesh": mesh.name, "path": path, "submesh": s}) + continue + prims.append((tri.astype(np.uint32), material_for(mid))) + if not prims: continue - if len(mats) < len(tris): - raise NotImplementedError(f"{mesh.m_Name}: {len(tris)} submeshes, {len(mats)} materials") - m = graph.world(tf_pid) - p = (np.hstack([v, np.ones((len(v), 1))]) @ m.T)[:, :3] + m = prefab.world(i) + p = (np.hstack([mesh.pos, np.ones((len(mesh.pos), 1))]) @ m.T)[:, :3] p[:, 2] *= -1.0 # Unity LH -> glTF RH - if len(vn): - nw = vn @ np.linalg.inv(m[:3, :3]).T + if mesh.nrm is not None: + nw = mesh.nrm @ np.linalg.inv(m[:3, :3]).T nw /= np.maximum(np.linalg.norm(nw, axis=1, keepdims=True), 1e-12) nw[:, 2] *= -1.0 else: nw = np.empty((0, 3)) - uv = uv0.copy() - uv[:, 1] = 1.0 - uv[:, 1] # Unity UV origin bottom-left -> glTF top-left - prims = submesh_primitives( - tris, - mats, - material_for, - lambda i: null_slots.append({"mesh": mesh.m_Name, "path": graph.path(tf_pid), "submesh": i}), - ) - if not prims: - continue - is_active = graph.active(tf_pid) - glb.add_mesh(mesh.m_Name, p, nw, uv, prims, active=is_active) - n_mesh += 1 - n_inactive += 0 if is_active else 1 - - out_path = Path(out_path) - out_path.parent.mkdir(parents=True, exist_ok=True) - glb.write(out_path, {"backgroundKey": bg_key}) + uv = mesh.uv if mesh.uv is not None else np.zeros((len(mesh.pos), 2), np.float32) + active = prefab.active(i) + glb.add_mesh(mesh.name, p, nw, uv, prims, active=active) + counts["meshes"] += 1 + counts["inactive"] += 0 if active else 1 + doc = { - "backgroundKey": bg_key, - "glb": out_path.name, - "meshCount": n_mesh, - "inactiveMeshes": n_inactive, - "skippedEmptyMeshes": skipped, + "backgroundKey": key, + "glb": "room.glb", + "meshCount": counts["meshes"], + "inactiveMeshes": counts["inactive"], "nullMaterialSubmeshes": null_slots, "nullMeshFilters": null_meshes, - "meshesWithoutTriangles": no_triangles, - "materials": [ - { - "name": m["name"], - "shader": m["extras"]["unityShader"], - "alphaMode": m.get("alphaMode", "OPAQUE"), - "alphaCutoff": m.get("alphaCutoff"), - "doubleSided": m.get("doubleSided", False), - "unlit": "KHR_materials_unlit" in m.get("extensions", {}), - "renderState": m["extras"]["unityRenderState"], - } - for m in glb.materials - ], - "textures": [t["name"] for t in glb.textures], - "samplers": glb.samplers, + "emptyMeshes": empty, + **glb.summary(), } - write_json(out_path.with_suffix(".json"), doc) - return doc + return glb.bytes({"backgroundKey": key}), doc diff --git a/src/nnnotes/content/spot.py b/src/nnnotes/content/spot.py index 6c05f16..4a34a6a 100644 --- a/src/nnnotes/content/spot.py +++ b/src/nnnotes/content/spot.py @@ -1,46 +1,79 @@ -"""Spot (explorable 2.5D scene) extractor -> spot.json + Spine assets. - -A spot = background prefab (cardboard room, see room.py) + situation prefab: -SpotSituationSettings (camera/interaction parameters), SpotCharacterController, -SpotCharacter x N (tap targets with focus transforms) and SpotSpineCharacter x M, -each driving a Spine.Unity SkeletonAnimation. Components are identified by -their MonoScript class (Catalog must be opened with `apk=`). - -A reference the prefab leaves empty is written as null, not filled in: a -SpotSpineCharacter with no `_animation` has null `skeletonData`, `animation` -and `world`; a SpotCharacter with no `_focus` has a null `focusWorld`. - -Spine files are written from the SkeletonDataAsset reference chain (write_skeleton): - SkeletonDataAsset.skeletonJSON -> skeleton (.json or binary .skel) - SkeletonDataAsset.atlasAssets[i].atlasFile -> .atlas - SpineAtlasAsset.materials[j]._MainTex -> PNG named after atlas page j +"""A spot (an explorable 2.5D scene of the home) assembled from the objects its situation prefab reaches: spot.json and +the Spine files of its characters. + +A spot is a background prefab (a room, content.room) and a situation prefab: SpotSituationSettings (camera and +interaction parameters), SpotCharacterController, SpotCharacter x N (tap targets with focus transforms) and +SpotSpineCharacter x M, each driving a Spine.Unity SkeletonAnimation. `spot` reads the situation prefab's document +(content.prefabs) and the objects its components reach, never a bundle; a MonoBehaviour's class is its script's. The +spot's row of MasterHomeSpot, its name and its tap talks come from the master data (`master_record`). + +A reference the prefab leaves empty is written as null: a SpotSpineCharacter with no `_animation` has null +`skeletonData`, `animation` and `world`; a SpotCharacter with no `_focus` has a null `focusWorld`. Fields that name +objects are written as the prefab document writes them (references). World matrices are 4x4 column-major in Unity's +world space (left-handed) of the situation prefab, its root's own transform included. + +Spine files (spine/) of each SkeletonDataAsset of the table (spine_files): + SkeletonDataAsset.skeletonJSON -> the TextAsset's bytes: the skeleton (.json or binary .skel) + SkeletonDataAsset.atlasAssets[i].atlasFile -> .atlas, as stored + SpineAtlasAsset.materials[j]._MainTex -> the Texture2D's PNG, named after atlas page j """ from __future__ import annotations -import io -import json -from pathlib import Path +import re -import UnityPy +from .prefabs import Objects, Prefab, mat4, ref_id, root_game_object, script_class -from ..core.jsonio import write_json -from ..source.catalog import Catalog -from ..source.config import apk_missing -from ..unity.reader import SceneGraph, script_class, strip_pptrs, texture_image +__all__ = ["LANGS", "SITUATION_KEY", "atlas_pages", "master_record", "situation_name", "spine_files", "spot"] +# a spot situation's catalog key: Spot/// +SITUATION_KEY = r"Spot/(?P[^/]+)/(?P[^/]+)/(?P=situation)" LANGS = ("_japanese", "_english", "_traditionalChinese", "_simplifiedChinese", "_korean") +MASTER_TABLES = ("MasterHomeSpot", "MasterText", "MasterStoryHomeSpotTapTalkEpisode") +HEADER = ("m_GameObject", "m_Script", "m_Enabled", "m_Name") # a component's Unity header fields +ANIMATION_FIELDS = ( + "_animationName", + "loop", + "timeScale", + "initialSkinName", + "initialFlipX", + "initialFlipY", + "pmaVertexColors", + "tintBlack", + "zSpacing", +) +MATRIX_CONVENTION = "Unity world space (left-handed), 4x4 column-major" + + +def situation_name(key: str) -> str: + """The of a situation key (ValueError for another key, a background's included).""" + m = re.fullmatch(SITUATION_KEY, key) + if m is None or m["situation"] == "Background": + raise ValueError(f"{key}: not a spot situation key (Spot///)") + return m["situation"] + + +def master_record(tables: dict[str, list[dict]], key: str) -> dict: + """{row, name, tapTalk} of the spot whose situation is `key`, from the master tables MASTER_TABLES ({table: + rows}): its MasterHomeSpot row, its name per language and {character id: adv ids} of its tap talks (ValueError + when no row, or several, has the key).""" + rows = [r for r in tables["MasterHomeSpot"] if r.get("_situationAssetPath") == key] + if len(rows) != 1: + raise ValueError(f"{key}: {len(rows)} MasterHomeSpot rows have it as their situation") + row = rows[0] + text = next((t for t in tables["MasterText"] if t.get("_id") == row.get("_nameTextId")), {}) + taps: dict[int, list[int]] = {} + for r in tables["MasterStoryHomeSpotTapTalkEpisode"]: + if r.get("_spotId") == row["_id"]: + taps.setdefault(r["_characterId"], []).append(r["_advId"]) + return { + "row": row, + "name": {lang[1:]: text.get(lang, "") for lang in LANGS}, + "tapTalk": {str(k): v for k, v in sorted(taps.items())}, + } -def _master(md: Path, name: str) -> list[dict]: - return json.loads((md / f"{name}.json").read_text(encoding="utf-8"))["_allData"] - - -def _mat4(m) -> list[float]: - return [float(x) for x in m.T.reshape(-1)] # column-major - - -def _atlas_pages(text: str) -> list[str]: +def atlas_pages(text: str) -> list[str]: """Page file names of a Spine atlas (a page line is followed by 'size:').""" lines = [ln.strip() for ln in text.splitlines()] return [ @@ -50,177 +83,129 @@ def _atlas_pages(text: str) -> list[str]: ] -def _text_bytes(ta) -> bytes: - s = ta.m_Script - return s.encode("utf-8", "surrogateescape") if isinstance(s, str) else bytes(s) +def _object(objects: Objects, v, what: str) -> str: + oid = ref_id(v) + if oid is None or oid not in objects: + raise ValueError(f"{what} is not in the object table") + return oid -def _target(ref): - """The object a PPtr refers to, None for an empty reference (path id 0).""" - return ref.read() if ref.path_id else None +def _text_asset(objects: Objects, v, what: str) -> tuple[str, bytes]: + """(name, bytes) of the TextAsset a reference names.""" + oid = _object(objects, v, what) + return objects.json(oid)["m_Name"], objects.bytes(oid, "data") -def write_skeleton(o, write, written_atlases: dict[int, str] | None = None) -> dict: - """The Spine files of the SkeletonDataAsset object `o` (module docstring), each given to `write(file name, - bytes)` in this order: the skeleton, then for each of its atlases not in `written_atlases` ({SpineAtlasAsset - path id: atlas file name}, updated here) the atlas text as stored and one PNG per page. Returns the skeleton's - record of spot.json: name, skeleton file, atlas files, scale, default mix, mixes.""" - written = {} if written_atlases is None else written_atlases - sda = o.read_typetree() - sk = o.read().skeletonJSON.read() - raw = _text_bytes(sk) - is_json = raw.lstrip()[:1] == b"{" - ext = ".json" if is_json else ".skel" - sk_file = sk.m_Name if sk.m_Name.endswith(ext) else f"{sk.m_Name}{ext}" # as the atlas below - write(sk_file, raw) +def spine_files(objects: Objects, sda: str, written: dict[str, str]) -> tuple[dict, dict[str, bytes]]: + """(the skeleton's record of spot.json: name, skeleton, atlases, scale, defaultMix, mixes; {file name: bytes}) + of the SkeletonDataAsset `sda`: the skeleton, then for each of its atlases not in `written` ({SpineAtlasAsset + object id: atlas file name}, updated here) the atlas text as stored and one PNG per page.""" + d = objects.json(sda) + name = d["m_Name"] + files: dict[str, bytes] = {} + sk_name, raw = _text_asset(objects, d.get("skeletonJSON"), f"{name}: the skeleton") + ext = ".json" if raw.lstrip()[:1] == b"{" else ".skel" + sk_file = sk_name if sk_name.endswith(ext) else f"{sk_name}{ext}" + files[sk_file] = raw atlases = [] - for ap in o.read().atlasAssets: - atlas_asset = ap.read() - if ap.path_id not in written: - af = atlas_asset.atlasFile.read() - araw = _text_bytes(af) - atext = araw.decode("utf-8") - a_file = af.m_Name if af.m_Name.endswith(".atlas") else f"{af.m_Name}.atlas" - write(a_file, araw) # the atlas text exactly as the game stores it - pages = _atlas_pages(atext) - mats = list(atlas_asset.materials) + for ap in d.get("atlasAssets") or (): + aid = _object(objects, ap, f"{name}: an atlas asset") + if aid not in written: + asset = objects.json(aid) + a_name, a_raw = _text_asset(objects, asset.get("atlasFile"), f"{name}: an atlas file") + a_file = a_name if a_name.endswith(".atlas") else f"{a_name}.atlas" + files[a_file] = a_raw # the atlas text exactly as the game stores it + pages = atlas_pages(a_raw.decode("utf-8")) + mats = asset.get("materials") or [] if len(mats) != len(pages): - raise RuntimeError(f"{a_file}: {len(pages)} pages vs {len(mats)} materials") + raise ValueError(f"{a_file}: {len(pages)} pages vs {len(mats)} materials") for page, mp in zip(pages, mats): - mat = mp.read() - tex = next(e.m_Texture.read() for k, e in mat.m_SavedProperties.m_TexEnvs if k == "_MainTex") - buf = io.BytesIO() - texture_image(tex).save(buf, format="PNG") - write(page, buf.getvalue()) - written[ap.path_id] = a_file - atlases.append(written[ap.path_id]) - return { - "name": sda["m_Name"], + mat = objects.json(_object(objects, mp, f"{a_file}: the material of page {page}")) + slot = (mat.get("textures") or {}).get("_MainTex") + files[page] = objects.bytes(_object(objects, slot and slot["texture"], f"{page}: the texture"), "image") + written[aid] = a_file + atlases.append(written[aid]) + record = { + "name": name, "skeleton": sk_file, "atlases": atlases, - "scale": sda["scale"], - "defaultMix": sda["defaultMix"], + "scale": d["scale"], + "defaultMix": d["defaultMix"], "mixes": [ - {"from": f, "to": t, "duration": d} - for f, t, d in zip(sda.get("fromAnimation", []), sda.get("toAnimation", []), sda.get("duration", [])) + {"from": f, "to": t, "duration": x} + for f, t, x in zip(d.get("fromAnimation", []), d.get("toAnimation", []), d.get("duration", [])) ], } - - -def spine_characters(objs: list, graph, skeletons: dict, world_of) -> list[dict]: - """SpotSpineCharacter components: object path, the SkeletonAnimation they drive (its skeleton data name, - initial animation settings and world matrix). `skeletons`: SkeletonDataAsset path id -> skeleton record. One - whose `_animation` is empty has those three null.""" - out = [] - for o in objs: - ssc = o.read_typetree() - path = graph.path(graph.tf_of_go[ssc["m_GameObject"]["m_PathID"]]) - anim_o = _target(o.read()._animation) - if anim_o is None: - out.append({"path": path, "skeletonData": None, "animation": None, "world": None}) + return record, files + + +def _fields(f: dict, *drop: str) -> dict: + return {k: v for k, v in f.items() if k not in HEADER and k not in drop} + + +def _first(prefab: Prefab, cls: str, key: str) -> dict: + found = [c for _, c in prefab.components(cls)] + if not found: + raise ValueError(f"{key}: no {cls} in the situation prefab") + return found[0]["fields"] + + +def spot(objects: Objects, key: str, master: dict) -> tuple[dict, dict[str, bytes]]: + """(spot.json, {spine/: bytes}) of the spot whose situation is `key` and whose master_record is `master` + (module documentation).""" + prefab = Prefab(objects.json(root_game_object(objects, key))) + files: dict[str, bytes] = {} + skeletons: dict[str, dict] = {} # SkeletonDataAsset object id -> its record + written: dict[str, str] = {} + sdas = [o for o in objects.ids("MonoBehaviour") if script_class(objects, o) == "SkeletonDataAsset"] + for sda in sorted(sdas, key=lambda o: (objects.json(o)["m_Name"], o)): + record, made = spine_files(objects, sda, written) + skeletons[sda] = record + for f, data in made.items(): + if f"spine/{f}" in files: + raise ValueError(f"{key}: two Spine files are named {f!r}") + files[f"spine/{f}"] = data + + spine_characters = [] + for i, c in prefab.components("SpotSpineCharacter"): + path = prefab.nodes[i]["path"] + anim = prefab.component_of(c["fields"].get("_animation")) + if anim is None: + spine_characters.append({"path": path, "skeletonData": None, "animation": None, "world": None}) continue - at = anim_o.object_reader.read_typetree() - out.append( + j, a = anim + sda = _object(objects, a["fields"].get("skeletonDataAsset"), f"{path}: the skeleton data") + spine_characters.append( { "path": path, - "skeletonData": skeletons[anim_o.skeletonDataAsset.path_id]["name"], - "animation": { - k: at[k] - for k in ( - "_animationName", - "loop", - "timeScale", - "initialSkinName", - "initialFlipX", - "initialFlipY", - "pmaVertexColors", - "tintBlack", - "zSpacing", - ) - }, - "world": world_of(at["m_GameObject"]), + "skeletonData": skeletons[sda]["name"], + "animation": {k: a["fields"][k] for k in ANIMATION_FIELDS}, + "world": mat4(prefab.world(j)), } ) - return out - - -def tap_targets(objs: list, graph, world_of) -> list[dict]: - """SpotCharacter components (tap targets) by character id: object path, their fields, world matrix and the world - matrix of their `_focus` transform (null when the reference is empty).""" - out = [] - for o in objs: - sc = o.read_typetree() - focus_o = _target(o.read()._focus) - out.append( + characters = [] + for i, c in prefab.components("SpotCharacter"): + focus = prefab.node_of(c["fields"].get("_focus")) + characters.append( { - "path": graph.path(graph.tf_of_go[sc["m_GameObject"]["m_PathID"]]), - **strip_pptrs({k: v for k, v in sc.items() if k != "_focus"}), - "world": world_of(sc["m_GameObject"]), - "focusWorld": None - if focus_o is None - else world_of(focus_o.object_reader.read_typetree()["m_GameObject"]), + "path": prefab.nodes[i]["path"], + **_fields(c["fields"], "_focus"), + "world": mat4(prefab.world(i)), + "focusWorld": None if focus is None else mat4(prefab.world(focus)), } ) - out.sort(key=lambda c: c["_characterId"]) - return out - - -def extract(cat: Catalog, md: Path, spot_id: int, out_dir: Path) -> dict: - if cat.apk is None: - raise apk_missing("resolving the Spot / Spine component classes") - spots = {r["_id"]: r for r in _master(md, "MasterHomeSpot")} - sp = spots[spot_id] - txt = {r["_id"]: r for r in _master(md, "MasterText")} - taps: dict[int, list[int]] = {} - for r in _master(md, "MasterStoryHomeSpotTapTalkEpisode"): - if r.get("_spotId") == spot_id: - taps.setdefault(r["_characterId"], []).append(r["_advId"]) - - out_dir = Path(out_dir) - spine_dir = out_dir / "spine" - spine_dir.mkdir(parents=True, exist_ok=True) - - env = UnityPy.load(*[str(p) for p in cat.fetch_key(sp["_situationAssetPath"])]) - graph = SceneGraph(env) - by_cls: dict[str | None, list] = {} - for o in env.objects: - if o.type.name == "MonoBehaviour": - by_cls.setdefault(script_class(o), []).append(o) - - def world_of(go_pptr) -> list[float]: - return _mat4(graph.world_of_go(go_pptr["m_PathID"])) - - settings = strip_pptrs(by_cls["SpotSituationSettings"][0].read_typetree()) - controller_o = by_cls["SpotCharacterController"][0] - controller = strip_pptrs(controller_o.read_typetree()) - - # Spine skeleton data -> files - def write(name: str, data: bytes) -> None: - (spine_dir / name).write_bytes(data) - - skeleton_files: dict[int, dict] = {} - written_atlases: dict[int, str] = {} - for o in by_cls.get("SkeletonDataAsset", []): - skeleton_files[o.path_id] = write_skeleton(o, write, written_atlases) - - spine_chars = spine_characters(by_cls.get("SpotSpineCharacter", []), graph, skeleton_files, world_of) - characters = tap_targets(by_cls.get("SpotCharacter", []), graph, world_of) - + characters.sort(key=lambda x: x["_characterId"]) + row = master["row"] doc = { - "spotId": spot_id, - "master": sp, - "name": {lang[1:]: txt.get(sp.get("_nameTextId"), {}).get(lang, "") for lang in LANGS}, - "tapTalk": {str(k): v for k, v in sorted(taps.items())}, - "situationSettings": settings, - "controller": {k: v for k, v in controller.items() if k not in ("_spotSpineCharacters", "_characters")}, + "spotId": row["_id"], + "master": row, + "name": master["name"], + "tapTalk": master["tapTalk"], + "situationSettings": _fields(_first(prefab, "SpotSituationSettings", key)), + "controller": _fields(_first(prefab, "SpotCharacterController", key), "_spotSpineCharacters", "_characters"), "characters": characters, - "spineCharacters": spine_chars, - "skeletons": list(skeleton_files.values()), - "resources": { - "background": [b.name for b in cat.resolve(sp["_backgroundAssetPath"])], - "situation": [b.name for b in cat.resolve(sp["_situationAssetPath"])], - }, - "matrixConvention": "Unity world space (left-handed), 4x4 column-major", + "spineCharacters": spine_characters, + "skeletons": list(skeletons.values()), + "matrixConvention": MATRIX_CONVENTION, } - write_json(out_dir / "spot.json", doc) - return doc + return doc, files diff --git a/src/nnnotes/content/storyhost.py b/src/nnnotes/content/storyhost.py index 462af37..8183627 100644 --- a/src/nnnotes/content/storyhost.py +++ b/src/nnnotes/content/storyhost.py @@ -9,8 +9,8 @@ host/host.json kind, the capture camera of a slot (CameraTargetRenderer prefab), the SimpleAdv root and its SimpleAdvLayoutRoot fields, the sequences the host screen has played when the talk starts, and per kind: - home spot.json + Spine files (spot.extract) and room.glb + room.json - (room.extract_room) under host/spot/; per placed glb node its object path, its + home spot.json + Spine files (hostspot.extract_spot) and room.glb + room.json + (hostspot.extract_room) under host/spot/; per placed glb node its object path, its state after SpotBackground.Prepare and the SpotSceneRoot.SetObject "floor" render queue; the room materials as saved; the "Spot" scene's object root, lights, volumes and RenderSettings; the lights of the placed prefabs; the background @@ -43,7 +43,8 @@ from ..unity.convert.animation import streamed_frames from ..unity.export import Exporter, TexelPacker from ..unity.reader import SceneGraph, deref, mesh_arrays, referent, script_class -from . import advui, room, spot, storyfonts +from . import advui, hostspot, storyfonts +from .spot import atlas_pages FORMAT = "ournotes.story-host/1" OVERLAY = 1 # MasterAdv._playbackMode of the episodes SimpleAdvPlayer plays @@ -600,14 +601,14 @@ def _go_ref(owner, pptr: dict) -> int | None: def background(cat, key: str, situation_name: str, ex: Exporter) -> dict: """The placed background prefab `key` as SpotSceneRoot.SetObject leaves it, over the nodes and materials of - room.extract_room (the same bundle load, glb_order): SpotBackground.Prepare(situation_name) on the prefab + hostspot.extract_room (the same bundle load, glb_order): SpotBackground.Prepare(situation_name) on the prefab root, the "floor" render queue (FindFirstChildWithName(background, "floor", "shadow"), its GetComponentInChildren().material.renderQueue = 2000, before the root is activated), then SetActiveFast(background, true). Objects outside the placed prefab (other prefabs of the bundle closure) are not placed: inactive.""" env = UnityPy.load(*[str(p) for p in cat.fetch_key(key)]) graph = SceneGraph(env) - mf_by_go, mr_by_go = room.mesh_components(env) + mf_by_go, mr_by_go = hostspot.mesh_components(env) rendered = {go for _, go in mr_by_go} # SceneGraph: a GameObject path id is in one file only root_go = container_go(env, cat.entry(key)["internal_id"]).path_id root = graph.tf_of_go[root_go] @@ -648,7 +649,7 @@ def background(cat, key: str, situation_name: str, ex: Exporter) -> dict: ) ) shown = {**overrides, root_go: True} - nodes, mats = glb_order(room.drawn_meshes(mf_by_go, mr_by_go, graph, lambda p: None)) + nodes, mats = glb_order(hostspot.drawn_meshes(mf_by_go, mr_by_go, graph, lambda p: None)) rows = [] for p in mats: mat = p.deref() @@ -712,7 +713,7 @@ def background(cat, key: str, situation_name: str, ex: Exporter) -> dict: def spine_records(env, ex: Exporter) -> tuple[dict, list, dict]: - """The Spine data of a situation prefab's bundle closure (loaded as spot.extract loads it): the material of + """The Spine data of a situation prefab's bundle closure (loaded as hostspot.extract_spot loads it): the material of every atlas page (Exporter.material records by name), per SpotSpineCharacter in spot.json's order its SkeletonAnimation's MeshRenderer sorting order / layer and {atlas page: material name} (SpineAtlasAsset materials in page order), and the shaders the materials name. -> (materials, characters, {name: shader}).""" @@ -734,7 +735,7 @@ def pages(sda) -> dict: if a is None: continue att = a.read_typetree() - names = spot._atlas_pages(spot._text_bytes(referent(a, att["atlasFile"]).read()).decode("utf-8")) + names = atlas_pages(hostspot.text_bytes(referent(a, att["atlasFile"]).read()).decode("utf-8")) mats = [deref(a, m) for m in att["materials"]] if len(names) != len(mats): raise RuntimeError(f"atlas {att.get('m_Name')}: {len(names)} pages vs {len(mats)} materials") @@ -860,17 +861,17 @@ def scene_root(ex: Exporter, key: str | None) -> dict: def home_host(cat, master_dir, player, host: dict, hdir: Path, keys: dict) -> dict: - """host.json `home`: spot.extract and room.extract_room into host/spot/, the placed background (background), - the Spine records, the Spot scene root, the blur, the ambient sound and the shaders (host/shaders/, GLES3 - programs only) with each material's GLES3 keyword set.""" + """host.json `home`: hostspot.extract_spot and hostspot.extract_room into host/spot/, the placed background + (background), the Spine records, the Spot scene root, the blur, the ambient sound and the shaders (host/shaders/, + GLES3 programs only) with each material's GLES3 keyword set.""" sdir = hdir / "spot" - doc = spot.extract(cat, Path(master_dir), host["spotId"], sdir) + doc = hostspot.extract_spot(cat, Path(master_dir), host["spotId"], sdir) row = doc["master"] # A tap talk's character need not be among the spot's SpotCharacters (spot.json `characters`): the game then has # no tap target that starts the talk (HomeSpotAfterTalkRequestFactory.TrySelectAdv matches the tapped # SpotCharacter's _characterId), and the player plays it without the camera focus bg_key, sit_key = row["_backgroundAssetPath"], row["_situationAssetPath"] - room_doc = room.extract_room(cat, bg_key, sdir / "room.glb") + room_doc = hostspot.extract_room(cat, bg_key, sdir / "room.glb") ex = Exporter(cat, hdir, player=player, textures="deferred", inline_meshes=False) sit_env = UnityPy.load(*[str(p) for p in cat.fetch_key(sit_key)]) sit_go = container_go(sit_env, cat.entry(sit_key)["internal_id"]) @@ -1082,15 +1083,15 @@ def component_in_children(graph, tf: int, has, active) -> int | None: def glb_order(drawn, arrays_of=mesh_arrays) -> tuple[list, list]: - """The nodes and materials room.extract_room writes, in its order, from its room.drawn_meshes items + """The nodes and materials hostspot.extract_room writes, in its order, from its hostspot.drawn_meshes items [(transform id, Mesh, material PPtrs)]: a mesh without vertex data or triangles, or whose submeshes all lack - a material, is no node; a material is numbered at its first use (room.submesh_primitives order, by - room.object_key). -> ([(transform id, mesh name)], [material PPtr]).""" + a material, is no node; a material is numbered at its first use (hostspot.submesh_primitives order, by + hostspot.object_key). -> ([(transform id, mesh name)], [material PPtr]).""" index: dict[tuple[str, int], int] = {} mats, nodes = [], [] def material_for(p) -> int: - key = room.object_key(p.deref()) + key = hostspot.object_key(p.deref()) if key not in index: index[key] = len(mats) mats.append(p) @@ -1105,29 +1106,29 @@ def material_for(p) -> int: continue if len(pptrs) < len(tris): raise NotImplementedError(f"{mesh.m_Name}: {len(tris)} submeshes, {len(pptrs)} materials") - if room.submesh_primitives(tris, pptrs, material_for, lambda i: None): + if hostspot.submesh_primitives(tris, pptrs, material_for, lambda i: None): nodes.append((tf, mesh.m_Name)) return nodes, mats def glb_textures(materials: list) -> dict[tuple[str, int], int]: - """{texture room.object_key: glb texture index} of room.extract_room for its materials [(material object, - typetree, shader name)] in glb order: the main texture slot of each (room.main_textures), numbered at first + """{texture hostspot.object_key: glb texture index} of hostspot.extract_room for its materials [(material object, + typetree, shader name)] in glb order: the main texture slot of each (hostspot.main_textures), numbered at first use.""" index: dict[tuple[str, int], int] = {} for owner, tt, shader_name in materials: - for tex, _ in room.main_textures(owner, tt, shader_name): - index.setdefault(room.object_key(tex), len(index)) + for tex, _ in hostspot.main_textures(owner, tt, shader_name): + index.setdefault(hostspot.object_key(tex), len(index)) return index def _texture_key(owner, ref: dict) -> tuple[str, int] | None: - """room.object_key of the texture a material's PPtr `ref` names (read from `owner`'s typetree); None for an + """hostspot.object_key of the texture a material's PPtr `ref` names (read from `owner`'s typetree); None for an empty slot or a texture outside the loaded files (so not in the glb).""" if not ref["m_PathID"]: return None try: - return room.object_key(deref(owner, ref)) + return hostspot.object_key(deref(owner, ref)) except FileNotFoundError: return None @@ -1169,7 +1170,7 @@ def glb_json(path) -> dict: def check_room(key: str, glb: dict, room_doc: dict, bg: dict) -> None: - """room.extract_room's room.glb (its JSON chunk `glb`) and room.json (`room_doc`) have the nodes, materials and + """hostspot.extract_room's room.glb (its JSON chunk `glb`) and room.json (`room_doc`) have the nodes, materials and glTF textures of the background record `bg` (background), in its order.""" if ( [n["name"] for n in glb["nodes"]] != bg["check"]["nodes"] diff --git a/src/nnnotes/formats/shader.py b/src/nnnotes/formats/shader.py index a957473..08a5a72 100644 --- a/src/nnnotes/formats/shader.py +++ b/src/nnnotes/formats/shader.py @@ -132,7 +132,7 @@ def variants(tt: dict, blobs: dict[int, list[bytes]]): n += 1 -def _parsed_summary(tt: dict) -> dict: +def parsed_summary(tt: dict) -> dict: pf = tt["m_ParsedForm"] return { "name": pf.get("m_Name"), @@ -160,7 +160,7 @@ def _parsed_summary(tt: dict) -> dict: def _render(o, tt: dict, name: str) -> tuple[str, list[dict], list[bytes]]: """(parsed summary JSON text, variant records, variant program bytes) of one Shader object.""" safe = _safe(name) - text = dumps(_parsed_summary(tt), indent=1, ensure_ascii=False, default=str) + text = dumps(parsed_summary(tt), indent=1, ensure_ascii=False, default=str) recs, codes = [], [] for plat, si, pi, stage, n, t, kws, code in variants(tt, platform_blobs(o.read())): tag = PLATFORM.get(plat, str(plat)) diff --git a/src/nnnotes/pipeline/assemble.py b/src/nnnotes/pipeline/assemble.py index f2fa27e..9ba12f6 100644 --- a/src/nnnotes/pipeline/assemble.py +++ b/src/nnnotes/pipeline/assemble.py @@ -238,10 +238,11 @@ def _prune(tid: str, objects: dict, made: Assembly) -> dict: class ProductStage(Stage): """Base of the product stages (module documentation). Subclasses set `name`, `version` and `KIND`, and define `keys`, `resources` and/or `root_objects` and `assemble`; `subjects` defaults to the subjects of the fact - "products".""" + "products". A stage that reads master tables names them in `MASTER` (its inputs come from the fact "master").""" after = (ADDRESSES, EXPORT, CROP) KIND = "product" + MASTER: tuple[str, ...] = () def __init__(self): self._memo: dict = {} diff --git a/src/nnnotes/pipeline/modelstages.py b/src/nnnotes/pipeline/modelstages.py deleted file mode 100644 index ca4c19a..0000000 --- a/src/nnnotes/pipeline/modelstages.py +++ /dev/null @@ -1,227 +0,0 @@ -"""The Spine skeleton stage, written by the extractor of `nnnotes export spot` (unchanged) from the bundles of its -tasks. - -spine.skeleton, one task per SkeletonDataAsset (a MonoBehaviour whose script the script table names -SkeletonDataAsset) of a selected bundle; its subject is the asset's stable address by path id, -":" (link.file_tags): - inputs "bundle:": the bundles holding the serialized files the asset's file reaches through - externals, transitively (every object a reference chain from the asset can reach); externals no - census holds (the built-in files) are left out - context {"object": the asset's object id} - artifacts "spine.skeleton:#": the skeleton (.json or .skel), each of its atlases (.atlas) and - the atlas pages (PNG), as spot.write_skeleton writes them for `nnnotes export spot` - item the asset: exported (its artifacts), or failed with the writer's error - -The task names its extractor as its one atom ("spot.write_skeleton"), whose id carries the versions of the libraries -the files depend on; this module imports UnityPy only when a task runs. -""" - -from __future__ import annotations - -import re - -from ..unity.atoms import impl_id -from . import contract, link -from .contract import Cost, IncompatibleTask, Input -from .stages import Output, Pending, Stage - -SPINE = "spine.skeleton" -CENSUS = "unity.census" -SCRIPTS = "link.scripts" -SCRIPTS_TASK = contract.task_id(SCRIPTS, "all") -SKELETON_CLASS = "SkeletonDataAsset" -# the libraries the extractor's output depends on (export._LIBS, the salt of export's PNG cache) -LIBRARIES = ("UnityPy", "Pillow", "texture2ddecoder", "astc-encoder-py", "etcpak", "numpy") -_ADDRESS = re.compile(r"0x[0-9a-fA-F]{6,}") -_EXT = re.compile(r"[0-9a-z]+") - - -def _message(e: BaseException) -> str: - return _ADDRESS.sub("0x?", f"{type(e).__name__}: {e}")[:500] - - -def _ext(name: str, default: str = "bin") -> str: - ext = name.rsplit("/", 1)[-1].rpartition(".")[2].lower() if "." in name.rsplit("/", 1)[-1] else "" - return ext if _EXT.fullmatch(ext) else default - - -def _check_atoms(task, atoms: dict) -> None: - if task.atoms != atoms: - raise IncompatibleTask(f"task {task.id}: this installation runs {atoms}, the task names {task.atoms}") - - -def _fact_input(bundles, stable: str, what: str) -> Input: - """The Input of a bundle from the fact "bundles" (Pending while it is not fetched).""" - if stable not in bundles: - raise ValueError(f"{what}: bundle {stable} is not readable") - inp = bundles[stable] - if inp is None: - raise Pending(f"fetch:{stable}") - if isinstance(inp, Exception): - raise inp - if inp.role != "bundle": - raise ValueError(f"{what}: input role {inp.role!r}, expected 'bundle'") - return inp - - -# ---------------------------------------------------------------- spine.skeleton -def skeleton_index(env, selected) -> dict: - """What spine.skeleton plans from, read from the census results and the script table: {"subjects": {subject: - (stable bundle name, file, pathId)} of the SkeletonDataAssets of the `selected` bundles, "files": {serialized - file (case-folded): stable bundle name} (a file two bundles hold: the first by name), "externals": {serialized - file (case-folded): [its external files]}}. Raises Pending while a census task has not run or the script table - is missing; failed censuses are left out.""" - pending = env.pending(CENSUS) - if pending: - raise Pending(pending[0]) - table = contract.loads(env.depot.read(env.input_of(SCRIPTS_TASK, "scripts").sha256))["scripts"] - scripts = {oid for oid, entry in table.items() if entry.rsplit("|", 1)[-1] == SKELETON_CLASS} - files: dict[str, str] = {} - externals: dict[str, list[str]] = {} - subjects: dict[str, tuple] = {} - for tid in env.tasks(CENSUS): - stable = contract.parse_task_id(tid, CENSUS)[1] - doc = contract.loads(env.depot.read(env.artifact(tid, "census")["content"]["sha256"])) - for f in doc["files"]: - files.setdefault(f["name"].casefold(), stable) - externals.setdefault(f["name"].casefold(), [x["file"] for x in f["externals"]]) - if not scripts or stable not in selected or "MonoBehaviour" not in doc["classes"]: - continue - tags = link.file_tags(f["name"] for f in doc["files"]) - for f in doc["files"]: - for o in f["objects"]: - s = o.get("script") - if ( - o["class"] == "MonoBehaviour" - and s - and s.get("file") is not None - and contract.object_id(s["file"], s["pathId"]) in scripts - ): - subject = contract.stable_address(stable + tags[f["name"]], path_id=o["pathId"]) - subjects[subject] = (stable, f["name"], int(o["pathId"])) - return {"subjects": subjects, "files": files, "externals": externals} - - -def reach(index: dict, file: str) -> list[str]: - """The stable names of the bundles holding `file` and every serialized file it reaches through externals, - sorted (skeleton_index document; files no census holds are left out).""" - start = file.casefold() - seen, stack, out = {start}, [start], set() - while stack: - f = stack.pop() - b = index["files"].get(f) - if b is None: - continue - out.add(b) - for x in index["externals"].get(f, ()): - xf = x.casefold() - if xf not in seen: - seen.add(xf) - stack.append(xf) - return sorted(out) - - -class SpineStage(Stage): - """spine.skeleton: a SkeletonDataAsset -> its Spine files as `nnnotes export spot` writes them (module - documentation). Subjects from the census results, the script table (link.scripts:all) and the fact "selected"; - inputs from the fact "bundles". Waits for census tasks that have not run.""" - - name = SPINE - version = 1 - after = (CENSUS, SCRIPTS) - ATOMS = {"spot.write_skeleton": impl_id(LIBRARIES, 2)} - - def __init__(self): - self._memo = None - - def __getstate__(self): - return {} - - def __setstate__(self, state): - self.__init__() - - def index(self, env) -> dict: - """skeleton_index, kept while the census results, the script table and the selection stay the same.""" - pending = env.pending(CENSUS) - if pending: - raise Pending(pending[0]) - selected = tuple(sorted(env.fact("selected"))) - sig = (selected, tuple(env.key(t) for t in env.tasks(CENSUS)), env.key(SCRIPTS_TASK)) - if self._memo is None or self._memo[0] != sig: - self._memo = (sig, skeleton_index(env, set(selected))) - return self._memo[1] - - def _subject(self, subject: str, env) -> tuple[dict, tuple]: - idx = self.index(env) - if subject not in idx["subjects"]: - raise ValueError(f"{self.name}: no SkeletonDataAsset {subject} in the selected bundles") - return idx, idx["subjects"][subject] - - def subjects(self, env) -> list[str]: - return sorted(self.index(env)["subjects"]) - - def closure(self, subject: str, env) -> list[str]: - idx, (_stable, file, _pid) = self._subject(subject, env) - return reach(idx, file) - - def inputs(self, subject: str, env) -> list[Input]: - bundles, what = env.fact("bundles"), f"{self.name} {subject}" - out = [] - for stable in self.closure(subject, env): - inp = _fact_input(bundles, stable, what) - out.append(Input(f"bundle:{stable}", inp.sha256, inp.size, inp.name, inp.locators)) - return out - - def context(self, subject: str, env) -> dict: - _idx, (_stable, file, pid) = self._subject(subject, env) - return {"object": contract.object_id(file, pid)} - - def depends(self, subject: str, env) -> list[str]: - return sorted({*(contract.task_id(CENSUS, s) for s in self.closure(subject, env)), SCRIPTS_TASK}) - - def estimate(self, subject: str, env, inputs: list[Input]) -> Cost: - """From the closure's bytes (least squares over the skeletons of a catalog).""" - size = sum(i.size for i in inputs) - return Cost(0.27 + size * 1.9e-7, (440 << 20) + 20 * size) - - def run(self, task, depot) -> Output: - _check_atoms(task, self.ATOMS) - from ..content import spot - from ..unity import reader as unity - - oid = task.context["object"] - paths = [depot.open_input(i) for i in task.inputs if i.role.startswith("bundle:")] - env = unity.load_closure(paths) - file, pid = contract.parse_object_id(oid) - written: dict[str, bytes] = {} - try: - sf = env.get_cab(file) - o = getattr(sf, "objects", {}).get(pid) - if o is None: - raise KeyError(f"{oid} is not in the task's bundles") - record = spot.write_skeleton(o, written.__setitem__) - except (MemoryError, OSError): - raise - except Exception as e: - return Output( - [], - [contract.item(oid, "failed", why=contract.reason("failed.export", _message(e)), cls="MonoBehaviour")], - ) - arts = [] - for name in sorted(written): - data = written[name] - if name == record["skeleton"]: - ext, kind, facts = _ext(name), "spine.skeleton", record - elif name in record["atlases"]: - ext, kind, facts = _ext(name, "atlas"), "spine.atlas", {} - else: - ext, kind, facts = "png", "spine.page", {} - arts.append( - contract.artifact( - contract.artifact_id(task.id, name), - depot.add(data, ext), - contract.provenance(task), - {"kind": kind, "format": ext, "facts": facts}, - ) - ) - return Output(arts, [contract.item(oid, "exported", artifacts=[a["id"] for a in arts], cls="MonoBehaviour")]) diff --git a/src/nnnotes/pipeline/spot.py b/src/nnnotes/pipeline/spot.py new file mode 100644 index 0000000..28e1bc1 --- /dev/null +++ b/src/nnnotes/pipeline/spot.py @@ -0,0 +1,106 @@ +"""The spot products: a spot background as binary glTF (room.gltf) and a spot situation's spot.json with its Spine +files (spot.files). + +Both are product stages (pipeline.assemble) whose roots are the objects of their subject's catalog key; they +assemble with content.room and content.spot and keep no objects (their files are all they hold): + +room.gltf subject a background key (Spot//Background/): "#room.glb" (kind spot.room) and + "#room.json" (kind spot.room-summary); its manifest is rooms/.json +spot.files subject a situation key (Spot///): "#spot.json" (kind spot.situation) + and "#spine/" for each Spine file (kind spine.skeleton, spine.atlas or spine.texture); its + manifest is spots/.json. Its input "master" is the spot's master record + (content.spot.master_record) from the master tables of the fact "master", made at planning +""" + +from __future__ import annotations + +import re + +from ..content import room, spot +from . import contract +from .assemble import Assembly, ProductStage +from .contract import Input +from .views import master_rows + +__all__ = ["ROOM", "SPOT", "RoomStage", "SpotStage", "spine_files"] + +ROOM, SPOT = "room.gltf", "spot.files" +MASTER_INPUT = "master" + + +def spine_files(doc: dict, spine: dict[str, bytes]) -> dict: + """{path: (bytes, extension, kind)} of the Spine files of spot.json `doc`: the skeletons (spine.skeleton), the + atlases (spine.atlas) and the PNG of each atlas page (spine.texture).""" + skeletons = {f"spine/{s['skeleton']}" for s in doc["skeletons"]} + atlases = {f"spine/{a}" for s in doc["skeletons"] for a in s["atlases"]} + out = {} + for path, data in spine.items(): + if path in skeletons: + out[path] = (data, path.rsplit(".", 1)[-1], "spine.skeleton") + elif path in atlases: + out[path] = (data, "atlas", "spine.atlas") + else: + out[path] = (data, "png", "spine.texture") + return out + + +class RoomStage(ProductStage): + """room.gltf: a background key -> room.glb and room.json (module documentation).""" + + name = ROOM + version = 1 + KIND = "SpotRoom" + + def select(self, keys: list[str]) -> list[str]: + return sorted(k for k in keys if re.fullmatch(room.BACKGROUND_KEY, k)) + + def keys(self, subject: str, env) -> list[str]: + room.room_name(subject) + return [subject] + + def manifest(self, subject: str) -> str: + return f"rooms/{room.room_name(subject)}.json" + + def assemble(self, task, objects) -> Assembly: + glb, doc = room.room(objects, task.subject) + files = { + "room.glb": (glb, "glb", "spot.room"), + "room.json": (contract.encode(doc), "json", "spot.room-summary"), + } + return Assembly(files, set()) + + +class SpotStage(ProductStage): + """spot.files: a situation key -> spot.json and its Spine files (module documentation).""" + + name = SPOT + version = 1 + KIND = "SpotSituation" + MASTER = spot.MASTER_TABLES + + def select(self, keys: list[str]) -> list[str]: + out = [] + for k in keys: + try: + spot.situation_name(k) + except ValueError: + continue + out.append(k) + return sorted(out) + + def keys(self, subject: str, env) -> list[str]: + spot.situation_name(subject) + return [subject] + + def more_inputs(self, subject: str, env) -> list[Input]: + data = contract.encode(spot.master_record(master_rows(env, self.MASTER), subject)) + return [Input(MASTER_INPUT, env.depot.put(data), len(data), None, ({"kind": "depot"},))] + + def manifest(self, subject: str) -> str: + return f"spots/{spot.situation_name(subject)}.json" + + def assemble(self, task, objects) -> Assembly: + master = contract.loads(objects.depot.input_bytes(task.input(MASTER_INPUT))) + doc, spine = spot.spot(objects, task.subject, master) + files = {"spot.json": (contract.encode(doc), "json", "spot.situation"), **spine_files(doc, spine)} + return Assembly(files, set()) diff --git a/src/nnnotes/pipeline/views.py b/src/nnnotes/pipeline/views.py index a8c3683..9ba60a7 100644 --- a/src/nnnotes/pipeline/views.py +++ b/src/nnnotes/pipeline/views.py @@ -874,6 +874,20 @@ def _cached_rows(sha: str, read) -> list[dict]: return rows +def master_rows(env, tables: Iterable[str]) -> dict[str, list[dict]]: + """{table: rows} of the master tables `tables` in the directory of the fact "master" (ViewError for a table the + master data lacks).""" + mdir = Path(env.fact("master")) + out = {} + for t in tables: + p = mdir / f"{t}.json" + if not p.is_file(): + raise ViewError(f"master table {t} is not in the master data") + sha, _ = env.depot.identify(p) + out[t] = _cached_rows(sha, p.read_bytes) + return out + + def _address_index(env, tid: str) -> AddressIndex: """The address index of the result of the link.addresses task `tid` (Pending until it has run).""" sha = env.artifact(tid, "addresses")["content"]["sha256"] diff --git a/src/nnnotes/unity/reader.py b/src/nnnotes/unity/reader.py index fd600e4..eef4285 100644 --- a/src/nnnotes/unity/reader.py +++ b/src/nnnotes/unity/reader.py @@ -19,6 +19,7 @@ from UnityPy.helpers.TypeTreeNode import TypeTreeNode from ..core import cache +from .transform import trs # ---- typetree blobs --------------------------------------------------------- # A serialized file lists the typetree of each of its classes as a blob, and UnityPy parses every blob in Python @@ -249,25 +250,6 @@ def texture_image(tex): # ---- scene graph ---------------------------------------------------------- -def quat_matrix(q: dict) -> np.ndarray: - x, y, z, w = q["x"], q["y"], q["z"], q["w"] - return np.array( - [ - [1 - 2 * (y * y + z * z), 2 * (x * y - z * w), 2 * (x * z + y * w), 0], - [2 * (x * y + z * w), 1 - 2 * (x * x + z * z), 2 * (y * z - x * w), 0], - [2 * (x * z - y * w), 2 * (y * z + x * w), 1 - 2 * (x * x + y * y), 0], - [0, 0, 0, 1], - ], - dtype=np.float64, - ) - - -def trs(pos: dict, rot: dict, scale: dict) -> np.ndarray: - t = np.eye(4) - t[:3, 3] = [pos["x"], pos["y"], pos["z"]] - return t @ quat_matrix(rot) @ np.diag([scale["x"], scale["y"], scale["z"], 1.0]) - - class SceneGraph: """GameObject/Transform hierarchy of an environment, with world matrices.""" diff --git a/src/nnnotes/unity/transform.py b/src/nnnotes/unity/transform.py new file mode 100644 index 0000000..2b69bfb --- /dev/null +++ b/src/nnnotes/unity/transform.py @@ -0,0 +1,27 @@ +"""Unity transforms as 4x4 matrices: a quaternion's rotation and a local position, rotation and scale (TRS), in +Unity's left-handed space.""" + +from __future__ import annotations + +import numpy as np + +__all__ = ["quat_matrix", "trs"] + + +def quat_matrix(q: dict) -> np.ndarray: + x, y, z, w = q["x"], q["y"], q["z"], q["w"] + return np.array( + [ + [1 - 2 * (y * y + z * z), 2 * (x * y - z * w), 2 * (x * z + y * w), 0], + [2 * (x * y + z * w), 1 - 2 * (x * x + z * z), 2 * (y * z - x * w), 0], + [2 * (x * z - y * w), 2 * (y * z + x * w), 1 - 2 * (x * x + y * y), 0], + [0, 0, 0, 1], + ], + dtype=np.float64, + ) + + +def trs(pos: dict, rot: dict, scale: dict) -> np.ndarray: + t = np.eye(4) + t[:3, 3] = [pos["x"], pos["y"], pos["z"]] + return t @ quat_matrix(rot) @ np.diag([scale["x"], scale["y"], scale["z"], 1.0]) diff --git a/tests/cli/test_assets.py b/tests/cli/test_assets.py index 185a9b6..f91890d 100644 --- a/tests/cli/test_assets.py +++ b/tests/cli/test_assets.py @@ -1464,6 +1464,31 @@ def assemble(self, task, objects): return assemble.Assembly(files, set()) +class FakeReader(assemble.ProductStage): + """A product per Bg/ key of the selection that reads the master table MasterX: its rows as rows.json.""" + + name = "fake.reader" + version = 1 + KIND = "FakeReader" + MASTER = ("MasterX",) + + def select(self, keys): + return [k for k in keys if k.startswith("Bg/")] + + def keys(self, subject, env): + return [subject] + + def more_inputs(self, subject, env): + from nnnotes.pipeline.views import master_rows + + data = contract.encode(master_rows(env, self.MASTER)) + return [contract.Input("master", env.depot.put(data), len(data), None, ({"kind": "depot"},))] + + def assemble(self, task, objects): + rows = objects.depot.input_bytes(task.input("master")) + return assemble.Assembly({"rows.json": (rows, "json", "fake.rows")}, set()) + + PRODUCT_SOURCES = FAKE_SOURCES + ( assets.StageSource("fake.product", "tests.cli.test_assets:FakeProduct", output=True, product=True), assets.StageSource("live2d.cubism", "tests.cli.test_assets:FakeCubism", output=True, product=True), @@ -1509,6 +1534,26 @@ def test_the_product_layout_places_the_products_of_the_selection(tmp_path, capsy assert (tmp_path / "both" / "product" / "fake.product" / "Bg%2Fx.json").is_file() +def test_products_that_read_master_data_have_subjects_only_with_it(tmp_path, capsys, monkeypatch): + sources = PRODUCT_SOURCES + ( + assets.StageSource("fake.reader", "tests.cli.test_assets:FakeReader", output=True, product=True), + ) + monkeypatch.setattr(assets, "STAGE_SOURCES", sources) + d = setup_data(tmp_path, censuses(broken=False)) + code, out, err = export(capsys, tmp_path, d, "s", "out", "--layout", "product") + assert code == 0, err + run = contract.loads((tmp_path / "out" / assets.REPORTS / "run.json").read_bytes()) + assert [t["id"] for t in run["tasks"] if t["id"].startswith("fake.")] == ["fake.product:Bg/x"] + master = tmp_path / "master" + master.mkdir() + (master / "MasterX.json").write_text('{"_allData": [{"_id": 1}]}', encoding="utf-8") + code, out, err = export(capsys, tmp_path, d, "s", "out2", "--layout", "product", "--master", master) + assert code == 0, err + manifest = contract.loads((tmp_path / "out2" / "fake.reader" / "Bg%2Fx.json").read_bytes()) + rows = (tmp_path / "out2" / manifest["files"]["rows.json"]["asset"]).read_bytes() + assert contract.loads(rows) == {"MasterX": [{"_id": 1}]} + + def test_export_live2d_writes_the_cubism_files_of_one_model(tmp_path, capsys, monkeypatch): monkeypatch.setattr(assets, "STAGE_SOURCES", PRODUCT_SOURCES) monkeypatch.setattr("nnnotes.pipeline.catalogdb.apk_version", lambda apk: None) diff --git a/tests/cli/test_cli.py b/tests/cli/test_cli.py index 9ec5b87..2fa65f5 100644 --- a/tests/cli/test_cli.py +++ b/tests/cli/test_cli.py @@ -368,7 +368,8 @@ def data(catalog_file, tmp_path, cdn, monkeypatch): "argv, message", [ (["bundle", "pull", "Char/A", "Nope/1", "Nope/2", "-o", "b"], "not a key of the catalog: Nope/1, Nope/2"), - (["export", "room", "Nope/1", "-o", "r.glb"], "not a key of the catalog: Nope/1"), + (["export", "room", "Nope/1", "-o", "r"], "Nope/1: not a spot background key"), + (["export", "room", "Spot/a/Background/a", "-o", "r"], "Spot/a/Background/a: not a key of the catalog"), (["export", "shader", "--key", "Nope/1", "-o", "s"], "not a key of the catalog: Nope/1"), (["export", "shader", "--apk-bundle", "zz", "-o", "s"], "--apk-bundle 'zz': 0 APK bundles match"), (["export", "audio", "Nope", "-o", "a"], "cue sheet Nope: no key Cri/Sound/Nope in the catalog"), diff --git a/tests/content/test_hostspot.py b/tests/content/test_hostspot.py new file mode 100644 index 0000000..cabd194 --- /dev/null +++ b/tests/content/test_hostspot.py @@ -0,0 +1,509 @@ +"""hostspot (the spot files of a story's home host, read from bundles): empty material slots and mesh filters, what +draws nothing, materials and textures of a bundle whose serialized files reuse path ids, Spine characters and tap +targets with and without the references the prefab leaves empty, and the Spine files of a skeleton (synthetic +objects).""" + +import copy +import io +import json +import struct +from types import SimpleNamespace + +import numpy as np +import pytest +from PIL import Image +from UnityPy.classes import PPtr as UnityPPtr + +from nnnotes.content import hostspot, room +from nnnotes.unity.reader import DEFAULT_RESOURCES +from tests.helpers.fakeunity import F32, POSITION, UV0, mesh_tt, submesh, unitypy_mesh +from tests.helpers.spotobjects import fv, material, shader + + +class PPtr: + def __init__(self, path_id): + self.path_id = path_id + + +def test_empty_material_slots_are_skipped_and_reported(): + tris = [np.array([[0, 1, 2]]), np.zeros((0, 3), int), np.array([[2, 3, 0], [0, 1, 3]])] + mats = [PPtr(0), PPtr(11), PPtr(12)] + nulls, used = [], [] + + def material_for(p): + used.append(p.path_id) + return p.path_id * 10 + + prims = hostspot.submesh_primitives(tris, mats, material_for, nulls.append) + assert nulls == [0] and used == [12] # the empty submesh 1 gives nothing, silently + assert len(prims) == 1 and prims[0][1] == 120 + assert prims[0][0].tolist() == [2, 0, 3, 0, 3, 1] # winding reversed for glTF + assert hostspot.submesh_primitives(tris[:1], mats[:1], material_for, nulls.append) == [] and nulls == [0, 0] + + +class Obj: + """A component: its typetree and the object UnityPy reads for it.""" + + def __init__(self, tt, **fields): + self.tt, self.fields = tt, fields + + def read_typetree(self): + return self.tt + + def read(self): + return type("Read", (), self.fields)() + + +class Ref(PPtr): + def __init__(self, path_id, target=None): + super().__init__(path_id) + self.target = target + + def read(self): + assert self.path_id, "an empty reference is not read" + return self.target + + +class Graph: + tf_of_go = {1: 11, 2: 12, 3: 13, 4: 14} + + def path(self, tf_pid): + return f"Room/obj{tf_pid}" + + +def test_a_mesh_filter_without_a_mesh_is_skipped_and_reported(): + mesh = object() + mfs = { + ("f", 1): Obj({}, m_Mesh=Ref(21, mesh)), + ("f", 2): Obj({}, m_Mesh=Ref(0)), + ("f", 3): Obj({}, m_Mesh=Ref(0)), + ("f", 4): Obj({}, m_Mesh=Ref(24, mesh)), + } + mrs = { + ("f", 1): Obj({"m_Enabled": 1}, m_Materials=["m1"]), + ("f", 2): Obj({"m_Enabled": 1}, m_Materials=[]), + ("f", 3): Obj({"m_Enabled": 0}, m_Materials=[]), # 3: disabled renderer, 4: no renderer + ("g", 4): Obj({"m_Enabled": 1}, m_Materials=["m4"]), + } # another file's GameObject 4 + nulls = [] + drawn = list(hostspot.drawn_meshes(mfs, mrs, Graph(), nulls.append)) + assert drawn == [(11, mesh, ["m1"])] + assert nulls == ["Room/obj12"] # the disabled one draws nothing either way + + +def test_extract_room_lists_what_draws_nothing(tmp_path, monkeypatch): + """A mesh without triangles, a MeshFilter without a mesh and a submesh without a material are not written but + listed in hostspot.json.""" + + class Mesh: + def __init__(self, name): + self.m_Name = name + + class Cat: + def fetch_key(self, key): + return [] + + class SceneGraphStub(Graph): + def __init__(self, env): + pass + + def world(self, tf_pid): + return np.eye(4) + + def active(self, tf_pid): + return True + + tri = np.array([[0, 1, 2]]) + arrays = { + "quad": (np.zeros((3, 3)), np.empty((0, 3)), np.zeros((3, 2)), [tri]), + "placeholder": (np.zeros((4, 3)), np.empty((0, 3)), np.zeros((4, 2)), [np.zeros((0, 3), int)]), + "bare": (np.zeros((4, 3)), np.empty((0, 3)), np.zeros((4, 2)), []), + } + + def drawn(mf_by_go, mr_by_go, graph, on_null_mesh): + on_null_mesh("Room/obj14") + yield 11, Mesh("quad"), [PPtr(0)] + yield 12, Mesh("placeholder"), [] + yield 13, Mesh("bare"), [] + + monkeypatch.setattr(hostspot.UnityPy, "load", lambda *a: type("Env", (), {"objects": []})()) + monkeypatch.setattr(hostspot, "SceneGraph", SceneGraphStub) + monkeypatch.setattr(hostspot, "drawn_meshes", drawn) + monkeypatch.setattr(hostspot, "mesh_arrays", lambda mesh: arrays[mesh.m_Name]) + doc = hostspot.extract_room(Cat(), "Spot/a/Background/b", tmp_path / "hostspot.glb") + assert doc["meshCount"] == 0 and doc["materials"] == [] + assert doc["nullMeshFilters"] == ["Room/obj14"] + assert doc["meshesWithoutTriangles"] == [ + {"mesh": "placeholder", "path": "Room/obj12"}, + {"mesh": "bare", "path": "Room/obj13"}, + ] + assert doc["nullMaterialSubmeshes"] == [{"mesh": "quad", "path": "Room/obj11", "submesh": 0}] + assert (tmp_path / "hostspot.glb").is_file() and (tmp_path / "hostspot.json").is_file() + + +# ---------------------------------------------------------------- a bundle whose serialized files reuse path ids +V0, V1 = {"x": 0.0, "y": 0.0, "z": 0.0}, {"x": 1.0, "y": 1.0, "z": 1.0} +Q0 = {"x": 0.0, "y": 0.0, "z": 0.0, "w": 1.0} +ROOM_KEY = "Spot/a/Background/b" +PREFAB = "Assets/Room.prefab" +# the quads under the prefab root: (object name, material path id, its file id in CAB-a: 1 is CAB-b) +ROOM_OBJECTS = [("wall", 5, 0), ("wall_b", 5, 1), ("floor", 6, 0), ("wall_c", 5, 0)] + + +def ref(pid, file_id=0): + return {"m_FileID": file_id, "m_PathID": pid} + + +class File: + """A serialized file as UnityPy's PPtr resolves it: name, objects by path id, externals (paths) and the bundle + holding it (`parent`); a file outside the loaded bundles is not found.""" + + def __init__(self, name, bundle, externals=()): + self.name, self.parent, self.objects = name, bundle, {} + self.externals = [SimpleNamespace(path=p) for p in externals] + self.environment = SimpleNamespace(find_file=lambda name: None) + bundle.files[name] = self + + def add(self, kind, pid, tt, obj=None): + o = SimpleNamespace( + type=SimpleNamespace(name=kind), + path_id=pid, + assets_file=self, + read_typetree=lambda: copy.deepcopy(tt), + read=lambda: obj, + ) + self.objects[pid] = o + return o + + def pptr(self, pid, file_id=0): + """A PPtr as UnityPy parses it from an object of this file.""" + return UnityPPtr(m_FileID=file_id, m_PathID=pid, assetsfile=self) + + +def tex_env(pid, file_id=0): + return {"m_Texture": ref(pid, file_id), "m_Scale": {"x": 1.0, "y": 1.0}, "m_Offset": {"x": 0.0, "y": 0.0}} + + +def textured(name, **slots): + """A material typetree of shader 3 with the texture envs `slots` {slot: tex_env}.""" + tt = material(name=name) + tt.update(m_Shader=ref(3), m_ValidKeywords=[], m_CustomRenderQueue=-1) + tt["m_SavedProperties"]["m_TexEnvs"] = [[k, v] for k, v in slots.items()] + return tt + + +def quad(name): + return unitypy_mesh( + mesh_tt(name, 3, {POSITION: (F32, 3, np.eye(3)), UV0: (F32, 2, np.zeros((3, 2)))}, [0, 1, 2], [submesh(0, 3)]) + ) + + +def two_file_room(): + """The environment of a bundle of two serialized files that reuse path ids. CAB-a holds the background prefab + (root `room`, GameObject 100, over ROOM_OBJECTS), shader 3, material 5 `a_wall` (main texture its texture 7, + `_Mask` in unity default resources), material 6 `a_floor` (main texture CAB-b's texture 7, `_Detail` its own + texture 7) and texture 7 `wall_a`. CAB-b holds shader 3, material 5 `b_wall` (main texture its texture 7) and + texture 7 `wall_b`: the pixels of `wall_a` with another filter.""" + bundle = SimpleNamespace(files={}) + a = File("CAB-a", bundle, ["archive:/CAB-b/CAB-b", DEFAULT_RESOURCES]) + b = File("CAB-b", bundle) + pixels = Image.new("RGBA", (2, 2), (200, 120, 40, 255)) + for f in (a, b): + f.add("Shader", 3, shader(name="Unlit/Transparent", srcBlend=fv(5.0), destBlend=fv(10.0))) + a.add("Material", 5, textured("a_wall", _MainTex=tex_env(7), _Mask=tex_env(9, 2))) + a.add("Material", 6, textured("a_floor", _MainTex=tex_env(7, 1), _Detail=tex_env(7))) + b.add("Material", 5, textured("b_wall", _MainTex=tex_env(7))) + for f, name, filter_mode in ((a, "wall_a", 1), (b, "wall_b", 0)): + f.add( + "Texture2D", + 7, + { + "m_Name": name, + "m_MipCount": 1, + "m_TextureSettings": {"m_FilterMode": filter_mode, "m_WrapU": 1, "m_WrapV": 1}, + }, + pixels.copy(), + ) + a.add("AssetBundle", 1, {"m_Container": [[PREFAB.lower(), {"asset": ref(100)}]]}) + a.add("GameObject", 100, {"m_Name": "room", "m_IsActive": 1, "m_Layer": 0}) + a.add( + "Transform", + 200, + { + "m_GameObject": ref(100), + "m_Father": ref(0), + "m_LocalPosition": V0, + "m_LocalRotation": Q0, + "m_LocalScale": V1, + "m_Children": [ref(201 + i) for i in range(len(ROOM_OBJECTS))], + }, + ) + for i, (name, mat, file_id) in enumerate(ROOM_OBJECTS): + go, mesh = 101 + i, quad(f"{name}_mesh") + a.add("GameObject", go, {"m_Name": name, "m_IsActive": 1, "m_Layer": 0}) + a.add( + "Transform", + 201 + i, + { + "m_GameObject": ref(go), + "m_Father": ref(200), + "m_Children": [], + "m_LocalPosition": V0, + "m_LocalRotation": Q0, + "m_LocalScale": V1, + }, + ) + a.add( + "MeshFilter", + 301 + i, + {"m_GameObject": ref(go)}, + SimpleNamespace(m_Mesh=SimpleNamespace(path_id=21 + i, read=lambda mesh=mesh: mesh)), + ) + a.add( + "MeshRenderer", + 401 + i, + {"m_GameObject": ref(go), "m_Enabled": 1}, + SimpleNamespace(m_Materials=[a.pptr(mat, file_id)]), + ) + return SimpleNamespace(objects=[o for f in (a, b) for o in f.objects.values()]) + + +class RoomCatalog: + def fetch_key(self, key): + return [] + + def entry(self, key): + return {"internal_id": PREFAB} + + +def load_room(monkeypatch, env): + """UnityPy.load gives `env`; a texture's image is the object it reads as.""" + monkeypatch.setattr(hostspot.UnityPy, "load", lambda *a: env) + monkeypatch.setattr(hostspot, "texture_image", lambda image: image) + + +def glb_json(path): + data = path.read_bytes() + (n,) = struct.unpack_from(" bytes: + buf = io.BytesIO() + image.save(buf, format="PNG") + return buf.getvalue() + + +def test_write_skeleton_writes_the_skeleton_atlas_and_pages_in_order(): + a, b = images() + shared = SpineRef(40, atlas_asset("chr", [Tex(a), Tex(b)])) + got = [] + written = {} + rec = hostspot.write_skeleton(skeleton("chr", b"\x00binary", [shared]), lambda n, d: got.append((n, d)), written) + assert [n for n, _ in got] == ["chr.skel", "chr.atlas", "p1.png", "p2.png"] + assert got[0][1] == b"\x00binary" and got[1][1] == ATLAS_TEXT.encode() + assert got[2][1] == png_of(a) and got[3][1] == png_of(b) + assert rec == { + "name": "chr_SkeletonData", + "skeleton": "chr.skel", + "atlases": ["chr.atlas"], + "scale": 0.01, + "defaultMix": 0.2, + "mixes": [{"from": "a", "to": "b", "duration": 0.5}], + } + assert written == {40: "chr.atlas"} + got.clear() # a second skeleton on the same atlas: written once + rec2 = hostspot.write_skeleton( + skeleton("chr2", ' {"skeleton": 1}', [shared]), lambda n, d: got.append((n, d)), written + ) + assert got == [("chr2.json", b' {"skeleton": 1}')] and rec2["atlases"] == ["chr.atlas"] + + +def test_write_skeleton_keeps_a_name_that_already_has_the_extension(): + a, b = images() + got = [] + shared = SpineRef(40, atlas_asset("chr.atlas", [Tex(a), Tex(b)])) + rec = hostspot.write_skeleton(skeleton("chr.skel", b"\x00binary", [shared]), lambda n, d: got.append((n, d))) + assert [n for n, _ in got] == ["chr.skel", "chr.atlas", "p1.png", "p2.png"] and rec["skeleton"] == "chr.skel" + + +def test_write_skeleton_checks_pages_against_materials(): + a, _ = images() + with pytest.raises(RuntimeError, match="2 pages vs 1 materials"): + hostspot.write_skeleton( + skeleton("x", b"s", [SpineRef(41, atlas_asset("x.atlas", [Tex(a)]))]), lambda n, d: None + ) diff --git a/tests/content/test_room.py b/tests/content/test_room.py index b01a938..f57a36f 100644 --- a/tests/content/test_room.py +++ b/tests/content/test_room.py @@ -1,79 +1,26 @@ -"""room: material translation from the pass render state (property-driven values), empty material slots, and the -glTF textures, images and materials of a bundle whose serialized files reuse path ids (synthetic shader / material -typetrees and objects).""" +"""room: material translation from the pass render state (property-driven values) on the documents of a Material and +its Shader, the glTF writer, and room.glb / room.json of a synthetic background's object table.""" -import copy import json import struct -from types import SimpleNamespace import numpy as np import pytest -from PIL import Image -from UnityPy.classes import PPtr as UnityPPtr from nnnotes.content import room -from nnnotes.unity.reader import DEFAULT_RESOURCES -from tests.helpers.fakeunity import F32, POSITION, UV0, mesh_tt, submesh, unitypy_mesh +from nnnotes.pipeline.assemble import Objects +from nnnotes.pipeline.depot import Depot +from nnnotes.unity.atoms import gltf +from tests.helpers import spotobjects as so +from tests.helpers.spotobjects import fv, material, shader -FIXED = "" +def translate(mat_tt, shader_tt): + return room.translate_material(so.material_json(mat_tt), so.shader_json(shader_tt)) -def fv(val, name=FIXED): - return {"val": val, "name": name} - -def shader(name="Universal Render Pipeline/Lit", props=None, tags=(), **state): - """A shader typetree: one subshader, one pass with render target 0 and the other states given (defaults: - opaque, back-face culling, depth write on, LEqual, all colour channels).""" - rt = { - "srcBlend": fv(1.0), - "destBlend": fv(0.0), - "srcBlendAlpha": fv(1.0), - "destBlendAlpha": fv(0.0), - "blendOp": fv(0.0), - "blendOpAlpha": fv(0.0), - "colMask": fv(15.0), - } - st = {"culling": fv(2.0), "zWrite": fv(1.0), "zTest": fv(4.0), "alphaToMask": fv(0.0)} - for k, v in state.items(): - (rt if k in rt else st)[k] = v - return { - "m_ParsedForm": { - "m_Name": name, - "m_PropInfo": { - "m_Props": [ - { - "m_Name": n, - "m_Type": 2, - "m_DefValue[0]": d, - "m_DefValue[1]": 0.0, - "m_DefValue[2]": 0.0, - "m_DefValue[3]": 0.0, - } - for n, d in (props or {}).items() - ] - }, - "m_SubShaders": [ - { - "m_Tags": {"tags": list(tags)}, - "m_Passes": [{"m_State": {"rtBlend0": rt, **st}, "m_Tags": {"tags": []}}], - } - ], - } - } - - -def material(floats=None, ints=None, name="m"): - return { - "m_Name": name, - "m_SavedProperties": { - "m_TexEnvs": [], - "m_Ints": list((ints or {}).items()), - "m_Floats": list((floats or {}).items()), - "m_Colors": [], - }, - } +def render_state(mat_tt, shader_tt): + return room.render_state(so.material_json(mat_tt), so.shader_json(shader_tt)) LIT_BLEND = dict( @@ -84,26 +31,26 @@ def material(floats=None, ints=None, name="m"): def test_property_driven_blend_takes_the_material_value(): sh = shader(props=LIT_PROPS, **LIT_BLEND) - opaque = room.translate_material(material({"_SrcBlend": 1.0, "_DstBlend": 0.0, "_Cull": 2.0}), sh) + opaque = translate(material({"_SrcBlend": 1.0, "_DstBlend": 0.0, "_Cull": 2.0}), sh) assert "alphaMode" not in opaque and "doubleSided" not in opaque assert opaque["extras"]["unityRenderState"]["srcBlend"] == 1 and opaque["extras"]["unityRenderState"]["zWrite"] == 1 - blended = room.translate_material(material({"_SrcBlend": 5.0, "_DstBlend": 10.0, "_Cull": 0.0, "_ZWrite": 0.0}), sh) + blended = translate(material({"_SrcBlend": 5.0, "_DstBlend": 10.0, "_Cull": 0.0, "_ZWrite": 0.0}), sh) assert blended["alphaMode"] == "BLEND" and blended["doubleSided"] is True assert blended["extras"]["unityRenderState"]["zWrite"] == 0 with pytest.raises(NotImplementedError, match="cull mode 1"): - room.translate_material(material({"_SrcBlend": 1.0, "_DstBlend": 0.0, "_Cull": 1.0}), sh) + translate(material({"_SrcBlend": 1.0, "_DstBlend": 0.0, "_Cull": 1.0}), sh) with pytest.raises(NotImplementedError, match="blend 1/1"): - room.translate_material(material({"_SrcBlend": 1.0, "_DstBlend": 1.0}), sh) + translate(material({"_SrcBlend": 1.0, "_DstBlend": 1.0}), sh) def test_missing_material_value_falls_back_to_the_shader_default_then_the_stored_value(): sh = shader(props=LIT_PROPS, **LIT_BLEND) - rs = room.render_state(material({}), sh) # the material has none of the properties + rs = render_state(material({}), sh) # the material has none of the properties assert (rs["srcBlend"], rs["dstBlend"], rs["cull"], rs["zWrite"]) == (1, 0, 2, 1) - rs = room.render_state(material({}), shader(**LIT_BLEND)) # no shader default either: the stored value + rs = render_state(material({}), shader(**LIT_BLEND)) # no shader default either: the stored value assert (rs["srcBlend"], rs["dstBlend"], rs["cull"], rs["zWrite"]) == (0, 0, 0, 0) # a fixed value ignores a material property of the same name - rs = room.render_state(material({"culling": 0.0, "_Cull": 0.0}), shader(culling=fv(2.0))) + rs = render_state(material({"culling": 0.0, "_Cull": 0.0}), shader(culling=fv(2.0))) assert rs["cull"] == 2 @@ -125,23 +72,23 @@ def test_missing_material_value_falls_back_to_the_shader_default_then_the_stored ) def test_every_state_field_resolves_through_its_property(key, field): sh = shader(props={"_P": 3.0}, **{field: fv(7.0, "_P")}) - assert room.render_state(material({"_P": 6.0}), sh)[key] == 6 # the material's float - assert room.render_state(material(ints={"_P": 5}), sh)[key] == 5 # or int - assert room.render_state(material(), sh)[key] == 3 # the shader's default - assert room.render_state(material(), shader(**{field: fv(7.0, "_P")}))[key] == 7 # the stored value - assert room.render_state(material({"_P": 6.0}), shader(**{field: fv(7.0)}))[key] == 7 # fixed: stored value + assert render_state(material({"_P": 6.0}), sh)[key] == 6 # the material's float + assert render_state(material(ints={"_P": 5}), sh)[key] == 5 # or int + assert render_state(material(), sh)[key] == 3 # the shader's default + assert render_state(material(), shader(**{field: fv(7.0, "_P")}))[key] == 7 # the stored value + assert render_state(material({"_P": 6.0}), shader(**{field: fv(7.0)}))[key] == 7 # fixed: stored value def test_blend_op_other_than_add_is_not_approximated(): sh = shader(name="Unlit/Transparent", srcBlend=fv(5.0), destBlend=fv(10.0), blendOp=fv(2.0, "_BlendOp")) with pytest.raises(NotImplementedError, match="blend op 2"): - room.translate_material(material(), sh) - assert room.translate_material(material({"_BlendOp": 0.0}), sh)["alphaMode"] == "BLEND" + translate(material(), sh) + assert translate(material({"_BlendOp": 0.0}), sh)["alphaMode"] == "BLEND" -def test_fixed_state_translation_is_unchanged(): +def test_fixed_state_translation(): sh = shader(name="Unlit/Transparent Cutout", tags=[("RenderType", "TransparentCutout")]) - cutout = room.translate_material(material({"_Cutoff": 0.5}), sh) + cutout = translate(material({"_Cutoff": 0.5}), sh) assert cutout["alphaMode"] == "MASK" and cutout["alphaCutoff"] == 0.5 assert cutout["extensions"] == {"KHR_materials_unlit": {}} rs = cutout["extras"]["unityRenderState"] @@ -159,280 +106,24 @@ def test_fixed_state_translation_is_unchanged(): "alphaToMask": 0, } with pytest.raises(NotImplementedError, match="shader not translated"): - room.translate_material(material(), shader(name="Custom/Other")) - - -class PPtr: - def __init__(self, path_id): - self.path_id = path_id - - -def test_empty_material_slots_are_skipped_and_reported(): - tris = [np.array([[0, 1, 2]]), np.zeros((0, 3), int), np.array([[2, 3, 0], [0, 1, 3]])] - mats = [PPtr(0), PPtr(11), PPtr(12)] - nulls, used = [], [] - - def material_for(p): - used.append(p.path_id) - return p.path_id * 10 - - prims = room.submesh_primitives(tris, mats, material_for, nulls.append) - assert nulls == [0] and used == [12] # the empty submesh 1 gives nothing, silently - assert len(prims) == 1 and prims[0][1] == 120 - assert prims[0][0].tolist() == [2, 0, 3, 0, 3, 1] # winding reversed for glTF - assert room.submesh_primitives(tris[:1], mats[:1], material_for, nulls.append) == [] and nulls == [0, 0] - - -class Obj: - """A component: its typetree and the object UnityPy reads for it.""" - - def __init__(self, tt, **fields): - self.tt, self.fields = tt, fields + translate(material(), shader(name="Custom/Other")) - def read_typetree(self): - return self.tt - def read(self): - return type("Read", (), self.fields)() - - -class Ref(PPtr): - def __init__(self, path_id, target=None): - super().__init__(path_id) - self.target = target - - def read(self): - assert self.path_id, "an empty reference is not read" - return self.target - - -class Graph: - tf_of_go = {1: 11, 2: 12, 3: 13, 4: 14} - - def path(self, tf_pid): - return f"Room/obj{tf_pid}" - - -def test_a_mesh_filter_without_a_mesh_is_skipped_and_reported(): - mesh = object() - mfs = { - ("f", 1): Obj({}, m_Mesh=Ref(21, mesh)), - ("f", 2): Obj({}, m_Mesh=Ref(0)), - ("f", 3): Obj({}, m_Mesh=Ref(0)), - ("f", 4): Obj({}, m_Mesh=Ref(24, mesh)), - } - mrs = { - ("f", 1): Obj({"m_Enabled": 1}, m_Materials=["m1"]), - ("f", 2): Obj({"m_Enabled": 1}, m_Materials=[]), - ("f", 3): Obj({"m_Enabled": 0}, m_Materials=[]), # 3: disabled renderer, 4: no renderer - ("g", 4): Obj({"m_Enabled": 1}, m_Materials=["m4"]), - } # another file's GameObject 4 - nulls = [] - drawn = list(room.drawn_meshes(mfs, mrs, Graph(), nulls.append)) - assert drawn == [(11, mesh, ["m1"])] - assert nulls == ["Room/obj12"] # the disabled one draws nothing either way - - -def test_extract_room_lists_what_draws_nothing(tmp_path, monkeypatch): - """A mesh without triangles, a MeshFilter without a mesh and a submesh without a material are not written but - listed in room.json.""" - - class Mesh: - def __init__(self, name): - self.m_Name = name - - class Cat: - def fetch_key(self, key): - return [] - - class SceneGraphStub(Graph): - def __init__(self, env): - pass - - def world(self, tf_pid): - return np.eye(4) - - def active(self, tf_pid): - return True - - tri = np.array([[0, 1, 2]]) - arrays = { - "quad": (np.zeros((3, 3)), np.empty((0, 3)), np.zeros((3, 2)), [tri]), - "placeholder": (np.zeros((4, 3)), np.empty((0, 3)), np.zeros((4, 2)), [np.zeros((0, 3), int)]), - "bare": (np.zeros((4, 3)), np.empty((0, 3)), np.zeros((4, 2)), []), - } - - def drawn(mf_by_go, mr_by_go, graph, on_null_mesh): - on_null_mesh("Room/obj14") - yield 11, Mesh("quad"), [PPtr(0)] - yield 12, Mesh("placeholder"), [] - yield 13, Mesh("bare"), [] - - monkeypatch.setattr(room.UnityPy, "load", lambda *a: type("Env", (), {"objects": []})()) - monkeypatch.setattr(room, "SceneGraph", SceneGraphStub) - monkeypatch.setattr(room, "drawn_meshes", drawn) - monkeypatch.setattr(room, "mesh_arrays", lambda mesh: arrays[mesh.m_Name]) - doc = room.extract_room(Cat(), "Spot/a/Background/b", tmp_path / "room.glb") - assert doc["meshCount"] == 0 and doc["materials"] == [] - assert doc["nullMeshFilters"] == ["Room/obj14"] - assert doc["meshesWithoutTriangles"] == [ - {"mesh": "placeholder", "path": "Room/obj12"}, - {"mesh": "bare", "path": "Room/obj13"}, - ] - assert doc["nullMaterialSubmeshes"] == [{"mesh": "quad", "path": "Room/obj11", "submesh": 0}] - assert (tmp_path / "room.glb").is_file() and (tmp_path / "room.json").is_file() - - -# ---------------------------------------------------------------- a bundle whose serialized files reuse path ids -V0, V1 = {"x": 0.0, "y": 0.0, "z": 0.0}, {"x": 1.0, "y": 1.0, "z": 1.0} -Q0 = {"x": 0.0, "y": 0.0, "z": 0.0, "w": 1.0} -ROOM_KEY = "Spot/a/Background/b" -PREFAB = "Assets/Room.prefab" -# the quads under the prefab root: (object name, material path id, its file id in CAB-a: 1 is CAB-b) -ROOM_OBJECTS = [("wall", 5, 0), ("wall_b", 5, 1), ("floor", 6, 0), ("wall_c", 5, 0)] - - -def ref(pid, file_id=0): - return {"m_FileID": file_id, "m_PathID": pid} - - -class File: - """A serialized file as UnityPy's PPtr resolves it: name, objects by path id, externals (paths) and the bundle - holding it (`parent`); a file outside the loaded bundles is not found.""" - - def __init__(self, name, bundle, externals=()): - self.name, self.parent, self.objects = name, bundle, {} - self.externals = [SimpleNamespace(path=p) for p in externals] - self.environment = SimpleNamespace(find_file=lambda name: None) - bundle.files[name] = self - - def add(self, kind, pid, tt, obj=None): - o = SimpleNamespace( - type=SimpleNamespace(name=kind), - path_id=pid, - assets_file=self, - read_typetree=lambda: copy.deepcopy(tt), - read=lambda: obj, - ) - self.objects[pid] = o - return o - - def pptr(self, pid, file_id=0): - """A PPtr as UnityPy parses it from an object of this file.""" - return UnityPPtr(m_FileID=file_id, m_PathID=pid, assetsfile=self) - - -def tex_env(pid, file_id=0): - return {"m_Texture": ref(pid, file_id), "m_Scale": {"x": 1.0, "y": 1.0}, "m_Offset": {"x": 0.0, "y": 0.0}} - - -def textured(name, **slots): - """A material typetree of shader 3 with the texture envs `slots` {slot: tex_env}.""" - tt = material(name=name) - tt.update(m_Shader=ref(3), m_ValidKeywords=[], m_CustomRenderQueue=-1) - tt["m_SavedProperties"]["m_TexEnvs"] = [[k, v] for k, v in slots.items()] - return tt - - -def quad(name): - return unitypy_mesh( - mesh_tt(name, 3, {POSITION: (F32, 3, np.eye(3)), UV0: (F32, 2, np.zeros((3, 2)))}, [0, 1, 2], [submesh(0, 3)]) - ) - - -def two_file_room(): - """The environment of a bundle of two serialized files that reuse path ids. CAB-a holds the background prefab - (root `room`, GameObject 100, over ROOM_OBJECTS), shader 3, material 5 `a_wall` (main texture its texture 7, - `_Mask` in unity default resources), material 6 `a_floor` (main texture CAB-b's texture 7, `_Detail` its own - texture 7) and texture 7 `wall_a`. CAB-b holds shader 3, material 5 `b_wall` (main texture its texture 7) and - texture 7 `wall_b`: the pixels of `wall_a` with another filter.""" - bundle = SimpleNamespace(files={}) - a = File("CAB-a", bundle, ["archive:/CAB-b/CAB-b", DEFAULT_RESOURCES]) - b = File("CAB-b", bundle) - pixels = Image.new("RGBA", (2, 2), (200, 120, 40, 255)) - for f in (a, b): - f.add("Shader", 3, shader(name="Unlit/Transparent", srcBlend=fv(5.0), destBlend=fv(10.0))) - a.add("Material", 5, textured("a_wall", _MainTex=tex_env(7), _Mask=tex_env(9, 2))) - a.add("Material", 6, textured("a_floor", _MainTex=tex_env(7, 1), _Detail=tex_env(7))) - b.add("Material", 5, textured("b_wall", _MainTex=tex_env(7))) - for f, name, filter_mode in ((a, "wall_a", 1), (b, "wall_b", 0)): - f.add( - "Texture2D", - 7, - { - "m_Name": name, - "m_MipCount": 1, - "m_TextureSettings": {"m_FilterMode": filter_mode, "m_WrapU": 1, "m_WrapV": 1}, - }, - pixels.copy(), - ) - a.add("AssetBundle", 1, {"m_Container": [[PREFAB.lower(), {"asset": ref(100)}]]}) - a.add("GameObject", 100, {"m_Name": "room", "m_IsActive": 1, "m_Layer": 0}) - a.add( - "Transform", - 200, - { - "m_GameObject": ref(100), - "m_Father": ref(0), - "m_LocalPosition": V0, - "m_LocalRotation": Q0, - "m_LocalScale": V1, - "m_Children": [ref(201 + i) for i in range(len(ROOM_OBJECTS))], - }, - ) - for i, (name, mat, file_id) in enumerate(ROOM_OBJECTS): - go, mesh = 101 + i, quad(f"{name}_mesh") - a.add("GameObject", go, {"m_Name": name, "m_IsActive": 1, "m_Layer": 0}) - a.add( - "Transform", - 201 + i, - { - "m_GameObject": ref(go), - "m_Father": ref(200), - "m_Children": [], - "m_LocalPosition": V0, - "m_LocalRotation": Q0, - "m_LocalScale": V1, - }, - ) - a.add( - "MeshFilter", - 301 + i, - {"m_GameObject": ref(go)}, - SimpleNamespace(m_Mesh=SimpleNamespace(path_id=21 + i, read=lambda mesh=mesh: mesh)), - ) - a.add( - "MeshRenderer", - 401 + i, - {"m_GameObject": ref(go), "m_Enabled": 1}, - SimpleNamespace(m_Materials=[a.pptr(mat, file_id)]), - ) - return SimpleNamespace(objects=[o for f in (a, b) for o in f.objects.values()]) - - -class RoomCatalog: - def fetch_key(self, key): - return [] - - def entry(self, key): - return {"internal_id": PREFAB} - - -def load_room(monkeypatch, env): - """UnityPy.load gives `env`; a texture's image is the object it reads as.""" - monkeypatch.setattr(room.UnityPy, "load", lambda *a: env) - monkeypatch.setattr(room, "texture_image", lambda image: image) - - -def glb_json(path): - data = path.read_bytes() - (n,) = struct.unpack_from(" Objects: + depot = Depot(tmp_path / "d") + return Objects(so.table(depot, objs or so.room_objects(), [so.ROOM_ROOT]), depot) + + +def test_room_bakes_the_drawn_meshes_and_lists_what_draws_nothing(tmp_path): + glb, doc = room.room(objects_of(tmp_path), so.BACKGROUND) + gdoc, body = gltf.read(glb) + assert [n["name"] for n in gdoc["nodes"]] == ["quad", "quad2", "quad"] # wall, floor, wall2: hierarchy order + assert [n["extras"]["unityActive"] for n in gdoc["nodes"]] == [True, False, True] + assert doc == { + "backgroundKey": so.BACKGROUND, + "glb": "room.glb", + "meshCount": 3, + "inactiveMeshes": 1, + "nullMaterialSubmeshes": [{"mesh": "quad2", "path": "room/floor", "submesh": 1}], + "nullMeshFilters": ["room/nomesh"], + "emptyMeshes": [{"mesh": "empty_mesh", "path": "room/empty"}, {"mesh": "points", "path": "room/points"}], + **{k: doc[k] for k in ("materials", "textures", "samplers")}, + } + assert gdoc["asset"]["extras"] == {"backgroundKey": so.BACKGROUND} + prim = gdoc["meshes"][0]["primitives"][0] + pos = gltf.accessor_array(gdoc, body, prim["attributes"]["POSITION"]) + # the mesh's vertices under the root at (1, 0, 2), in glTF's space (z negated) + expected = np.array(so.VERTICES) + [1.0, 0.0, 2.0] + expected[:, 2] *= -1.0 + assert np.allclose(pos, expected) + assert np.allclose(gltf.accessor_array(gdoc, body, prim["attributes"]["NORMAL"]), [[0.0, 0.0, 1.0]] * 4) + uv = gltf.accessor_array(gdoc, body, prim["attributes"]["TEXCOORD_0"]) + assert np.allclose(uv, [(u, 1.0 - v) for u, v in so.UVS]) # v flipped once + assert gltf.accessor_array(gdoc, body, prim["indices"]).reshape(-1).tolist() == [0, 2, 1, 0, 3, 2] + assert len(gdoc["meshes"][1]["primitives"]) == 1 # the floor's second submesh has no material + + +def test_room_materials_and_textures_are_shared_by_object(tmp_path): + glb, doc = room.room(objects_of(tmp_path), so.BACKGROUND) + gdoc, _ = gltf.read(glb) + assert [m["name"] for m in gdoc["materials"]] == ["a_wall", "a_floor"] + assert [p["material"] for m in gdoc["meshes"] for p in m["primitives"]] == [0, 1, 0] + wall, floor = gdoc["materials"] + assert wall["alphaMode"] == "BLEND" and wall["extensions"] == {"KHR_materials_unlit": {}} + assert wall["pbrMetallicRoughness"]["baseColorTexture"] == {"index": 0} + assert floor["pbrMetallicRoughness"] == { + "baseColorFactor": [1.0, 0.5, 0.25, 1.0], + "metallicFactor": 0.0, + "roughnessFactor": 0.75, + "baseColorTexture": { + "index": 1, + "extensions": {"KHR_texture_transform": {"scale": [2.0, 2.0], "offset": [0.0, -1.5]}}, + }, + } + assert gdoc["extensionsUsed"] == ["KHR_materials_unlit", "KHR_texture_transform"] + assert doc["textures"] == ["wall_a", "floor_b"] and len(gdoc["images"]) == 2 + assert doc["samplers"] == [ + {"magFilter": room.LINEAR, "minFilter": room.LINEAR, "wrapS": room.CLAMP, "wrapT": room.CLAMP}, + {"magFilter": room.NEAREST, "minFilter": room.NEAREST, "wrapS": room.REPEAT, "wrapT": room.REPEAT}, ] - assert [im["name"] for im in gltf["images"]] == ["wall_a"] - assert [s["magFilter"] for s in gltf["samplers"]] == [room.LINEAR, room.NEAREST] - assert doc["textures"] == ["wall_a", "wall_b"] and doc["samplers"] == gltf["samplers"] + assert [m["name"] for m in doc["materials"]] == ["a_wall", "a_floor"] + assert doc["materials"][1]["alphaMode"] == "OPAQUE" and doc["materials"][0]["unlit"] is True + + +def test_a_mesh_the_conversion_failed_on_is_an_error(tmp_path): + objs = so.room_objects() + objs[so.EMPTY] = ("Mesh", "broken", {"json": {}}, {"status": "generic", "reason": "generic.error"}) + with pytest.raises(ValueError, match="room/empty: mesh 'broken' was not converted"): + room.room(objects_of(tmp_path, objs), so.BACKGROUND) + + +def test_a_material_without_its_shader_is_an_error(tmp_path): + objs = so.room_objects() + del objs[so.LIT] + with pytest.raises(ValueError, match="material 'a_floor' has no shader in the object table"): + room.room(objects_of(tmp_path, objs), so.BACKGROUND) + + +def test_room_names(): + assert room.room_name(so.BACKGROUND) == "home_a" + with pytest.raises(ValueError, match="not a spot background key"): + room.room_name(so.SITUATION) diff --git a/tests/content/test_spot.py b/tests/content/test_spot.py index 3048981..19d928c 100644 --- a/tests/content/test_spot.py +++ b/tests/content/test_spot.py @@ -1,188 +1,134 @@ -"""spot: Spine characters and tap targets, with and without the references the prefab leaves empty -(synthetic components), and the Spine files of a skeleton (synthetic objects).""" - -import io +"""spot: spot.json and the Spine files of a synthetic situation's object table, and the spot's master record.""" import pytest -from PIL import Image from nnnotes.content import spot +from nnnotes.pipeline.assemble import Objects +from nnnotes.pipeline.depot import Depot +from tests.helpers import spotobjects as so -class Ref: - def __init__(self, path_id, target=None): - self.path_id, self.target = path_id, target - - def read(self): - assert self.path_id, "an empty reference is not read" - return self.target - - -class Reader: - def __init__(self, tt): - self.tt = tt - - def read_typetree(self): - return self.tt - - -class Read: - def __init__(self, tt=None, **fields): - self.object_reader = Reader(tt) - self.__dict__.update(fields) - - -class Obj: - def __init__(self, tt, **fields): - self.tt, self.fields = tt, fields - - def read_typetree(self): - return self.tt +def objects_of(tmp_path, objs=None) -> Objects: + depot = Depot(tmp_path / "d") + return Objects(so.table(depot, objs or so.situation_objects(), [so.SIT_ROOT]), depot) - def read(self): - return Read(**self.fields) +def translation(x, y, z): + return [1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, x, y, z, 1.0] # column-major -class Graph: - tf_of_go = {1: 11, 2: 12, 3: 13, 4: 14} - def path(self, tf_pid): - return f"Situation/obj{tf_pid}" - - -def go(pid): - return {"m_FileID": 0, "m_PathID": pid} - - -def world_of(go_ref): - return [float(go_ref["m_PathID"])] - - -ANIM = { - "m_GameObject": go(3), - "_animationName": "idle", - "loop": 1, - "timeScale": 1.0, - "initialSkinName": "default", - "initialFlipX": 0, - "initialFlipY": 0, - "pmaVertexColors": 1, - "tintBlack": 0, - "zSpacing": 0.0, -} - - -def test_spine_characters_keep_one_without_animation_as_null(): - anim = Read(ANIM, skeletonDataAsset=Ref(50)) - objs = [Obj({"m_GameObject": go(1)}, _animation=Ref(30, anim)), Obj({"m_GameObject": go(2)}, _animation=Ref(0))] - got = spot.spine_characters(objs, Graph(), {50: {"name": "chr_sd"}}, world_of) - assert got[0] == { - "path": "Situation/obj11", - "skeletonData": "chr_sd", - "animation": {k: v for k, v in ANIM.items() if k != "m_GameObject"}, - "world": [3.0], +def test_master_record(): + rec = spot.master_record(so.MASTER, so.SITUATION) + assert rec == { + "row": so.MASTER["MasterHomeSpot"][0], + "name": { + "japanese": "カフェ", + "english": "Cafe", + "traditionalChinese": "", + "simplifiedChinese": "", + "korean": "", + }, + "tapTalk": {"5": [51], "7": [71, 72]}, } - assert got[1] == {"path": "Situation/obj12", "skeletonData": None, "animation": None, "world": None} - - -def test_tap_targets_without_focus_have_a_null_focus_world(): - focus = Read({"m_GameObject": go(3)}) - objs = [ - Obj({"m_GameObject": go(2), "m_Enabled": 1, "_characterId": 7, "_focus": go(0)}, _focus=Ref(0)), - Obj({"m_GameObject": go(1), "m_Enabled": 1, "_characterId": 5, "_focus": go(33)}, _focus=Ref(33, focus)), - ] - got = spot.tap_targets(objs, Graph(), world_of) - assert [c["_characterId"] for c in got] == [5, 7] - assert got[0] == {"path": "Situation/obj11", "_characterId": 5, "world": [1.0], "focusWorld": [3.0]} - assert got[1] == {"path": "Situation/obj12", "_characterId": 7, "world": [2.0], "focusWorld": None} - - -# ---------------------------------------------------------------- Spine files (synthetic objects) -class Text: - def __init__(self, name, script): - self.m_Name, self.m_Script = name, script - - -class Tex: - m_TextureFormat = 4 # RGBA32 - - def __init__(self, image): - self.m_Name, self.image = "page", image - - -class TexEnv: - def __init__(self, tex): - self.m_Texture = Ref(1, tex) - - -class Material: - def __init__(self, tex): - self.m_SavedProperties = Read(m_TexEnvs=[("_BumpMap", TexEnv(None)), ("_MainTex", TexEnv(tex))]) - - -ATLAS_TEXT = "\np1.png\nsize: 2,2\nformat: RGBA8888\nr\n bounds: 0,0,1,1\n\np2.png\nsize: 1,1\nr2\n bounds: 0,0,1,1\n" - - -def atlas_asset(name, pages): - return Read( - atlasFile=Ref(20, Text(name, ATLAS_TEXT)), materials=[Ref(30 + i, Material(t)) for i, t in enumerate(pages)] - ) - - -def skeleton(name, script, atlases): - tt = { - "m_Name": name + "_SkeletonData", - "scale": 0.01, - "defaultMix": 0.2, - "fromAnimation": ["a"], - "toAnimation": ["b"], - "duration": [0.5], + with pytest.raises(ValueError, match="0 MasterHomeSpot rows"): + spot.master_record(so.MASTER, "Spot/c/c_01/c_01") + + +def test_spot_document(tmp_path): + doc, files = spot.spot(objects_of(tmp_path), so.SITUATION, spot.master_record(so.MASTER, so.SITUATION)) + assert doc == { + "spotId": 3, + "master": so.MASTER["MasterHomeSpot"][0], + "name": doc["name"], + "tapTalk": {"5": [51], "7": [71, 72]}, + "situationSettings": {"backgroundPosition": so.V0, "cameraFov": 30.0}, + "controller": {"_speed": 2.0}, + "characters": [ # by character id; the root at (0, 0, 1) + { + "path": "sit/chr5", + "_characterId": 5, + "world": translation(2.0, 0.0, 1.0), + "focusWorld": translation(2.0, 1.0, 1.0), + }, + { + "path": "sit/chr7", + "_characterId": 7, + "_radius": 0.5, + "world": translation(0.0, 0.0, 1.0), + "focusWorld": None, + }, + ], + "spineCharacters": [ # hierarchy order + {"path": "sit/chr7", "skeletonData": None, "animation": None, "world": None}, + { + "path": "sit/chr5", + "skeletonData": "chr_SkeletonData", + "animation": so.ANIMATION, + "world": translation(2.0, 0.0, 1.0), + }, + ], + "skeletons": [ # by name; the shared atlas once + { + "name": "chr2_SkeletonData", + "skeleton": "chr2.json", + "atlases": ["chr.atlas"], + "scale": 0.01, + "defaultMix": 0.2, + "mixes": [{"from": "a", "to": "b", "duration": 0.5}], + }, + { + "name": "chr_SkeletonData", + "skeleton": "chr.skel", + "atlases": ["chr.atlas"], + "scale": 0.01, + "defaultMix": 0.2, + "mixes": [{"from": "a", "to": "b", "duration": 0.5}], + }, + ], + "matrixConvention": spot.MATRIX_CONVENTION, + } + objs = so.situation_objects() + assert files == { + "spine/chr2.json": b' {"skeleton": 1}', + "spine/chr.atlas": so.ATLAS_TEXT.encode(), + "spine/p1.png": objs[so.PAGE1][2]["image"], + "spine/p2.png": objs[so.PAGE2][2]["image"], + "spine/chr.skel": b"\x00binary", } - return Obj(tt, skeletonJSON=Ref(9, Text(name, script)), atlasAssets=atlases) -def images(): - return Image.new("RGBA", (2, 2), (1, 2, 3, 4)), Image.new("RGBA", (1, 1), (9, 8, 7, 6)) +def test_a_skeleton_name_with_its_extension_is_kept(tmp_path): + objs = so.situation_objects() + cls, name, files, extra = objs[so.SKEL] + objs[so.SKEL] = (cls, name, {**files, "json": {"m_Name": "chr.skel"}}, extra) + _, files = spot.spot(objects_of(tmp_path, objs), so.SITUATION, spot.master_record(so.MASTER, so.SITUATION)) + assert "spine/chr.skel" in files and "spine/chr.skel.skel" not in files -def png(image) -> bytes: - buf = io.BytesIO() - image.save(buf, format="PNG") - return buf.getvalue() +def test_atlas_pages_match_its_materials(tmp_path): + objs = so.situation_objects() + cls, name, files, extra = objs[so.ATLAS] + objs[so.ATLAS] = (cls, name, {"json": {**files["json"], "materials": files["json"]["materials"][:1]}}, extra) + with pytest.raises(ValueError, match="chr.atlas: 2 pages vs 1 materials"): + spot.spot(objects_of(tmp_path, objs), so.SITUATION, spot.master_record(so.MASTER, so.SITUATION)) -def test_write_skeleton_writes_the_skeleton_atlas_and_pages_in_order(): - a, b = images() - shared = Ref(40, atlas_asset("chr", [Tex(a), Tex(b)])) - got = [] - written = {} - rec = spot.write_skeleton(skeleton("chr", b"\x00binary", [shared]), lambda n, d: got.append((n, d)), written) - assert [n for n, _ in got] == ["chr.skel", "chr.atlas", "p1.png", "p2.png"] - assert got[0][1] == b"\x00binary" and got[1][1] == ATLAS_TEXT.encode() - assert got[2][1] == png(a) and got[3][1] == png(b) - assert rec == { - "name": "chr_SkeletonData", - "skeleton": "chr.skel", - "atlases": ["chr.atlas"], - "scale": 0.01, - "defaultMix": 0.2, - "mixes": [{"from": "a", "to": "b", "duration": 0.5}], - } - assert written == {40: "chr.atlas"} - got.clear() # a second skeleton on the same atlas: written once - rec2 = spot.write_skeleton(skeleton("chr2", ' {"skeleton": 1}', [shared]), lambda n, d: got.append((n, d)), written) - assert got == [("chr2.json", b' {"skeleton": 1}')] and rec2["atlases"] == ["chr.atlas"] +def test_a_situation_without_its_settings_is_an_error(tmp_path): + objs = so.situation_objects() + cls, name, files, extra = objs[so.SIT_ROOT] + doc = files["json"] + root = {**doc["nodes"][0], "components": doc["nodes"][0]["components"][1:]} + objs[so.SIT_ROOT] = (cls, name, {"json": {**doc, "nodes": [root, *doc["nodes"][1:]]}}, extra) + with pytest.raises(ValueError, match="no SpotSituationSettings in the situation prefab"): + spot.spot(objects_of(tmp_path, objs), so.SITUATION, spot.master_record(so.MASTER, so.SITUATION)) -def test_write_skeleton_keeps_a_name_that_already_has_the_extension(): - a, b = images() - got = [] - shared = Ref(40, atlas_asset("chr.atlas", [Tex(a), Tex(b)])) - rec = spot.write_skeleton(skeleton("chr.skel", b"\x00binary", [shared]), lambda n, d: got.append((n, d))) - assert [n for n, _ in got] == ["chr.skel", "chr.atlas", "p1.png", "p2.png"] and rec["skeleton"] == "chr.skel" +def test_situation_names(): + assert spot.situation_name(so.SITUATION) == "home_a_01" + for key in (so.BACKGROUND, "Spot/a/b/c"): + with pytest.raises(ValueError, match="not a spot situation key"): + spot.situation_name(key) -def test_write_skeleton_checks_pages_against_materials(): - a, _ = images() - with pytest.raises(RuntimeError, match="2 pages vs 1 materials"): - spot.write_skeleton(skeleton("x", b"s", [Ref(41, atlas_asset("x.atlas", [Tex(a)]))]), lambda n, d: None) +def test_atlas_pages(): + assert spot.atlas_pages(so.ATLAS_TEXT) == ["p1.png", "p2.png"] diff --git a/tests/content/test_story_host.py b/tests/content/test_story_host.py index f1b01d2..57e9462 100644 --- a/tests/content/test_story_host.py +++ b/tests/content/test_story_host.py @@ -9,9 +9,9 @@ import numpy as np import pytest -from nnnotes.content import advui, room, storyhost +from nnnotes.content import advui, hostspot, storyhost from nnnotes.unity.reader import SceneGraph -from tests.content.test_room import ROOM_KEY, File, RoomCatalog, load_room, two_file_room +from tests.content.test_hostspot import ROOM_KEY, File, RoomCatalog, load_room, two_file_room V0 = {"x": 0.0, "y": 0.0, "z": 0.0} V1 = {"x": 1.0, "y": 1.0, "z": 1.0} @@ -153,7 +153,7 @@ def test_glb_json_reads_the_json_chunk(tmp_path): storyhost.glb_json(tmp_path / "x.glb") -def test_glb_order_follows_room_extract_room(): +def test_glb_order_follows_hostspot_extract_room(): tri = [[0, 1, 2]] drawn = [ (1, Mesh("a", [tri, tri]), [pp(20), pp(10)]), @@ -165,7 +165,7 @@ def test_glb_order_follows_room_extract_room(): ] # path id 10 of another file: another material nodes, mats = storyhost.glb_order(drawn, arrays) assert nodes == [(1, "a"), (5, "b"), (6, "c")] - assert [room.object_key(m.deref()) for m in mats] == [("CAB-a", 20), ("CAB-a", 10), ("CAB-b", 10)] + assert [hostspot.object_key(m.deref()) for m in mats] == [("CAB-a", 20), ("CAB-a", 10), ("CAB-b", 10)] with pytest.raises(NotImplementedError, match="2 submeshes, 1 materials"): storyhost.glb_order([(1, Mesh("c", [tri, tri]), [pp(1)])], arrays) @@ -226,11 +226,11 @@ def test_glb_textures_and_room_materials_keep_the_glb_order(): assert storyhost.room_material(oa, empty, "U", index)["texEnvs"]["_MainTex"]["texture"] is None -def test_background_numbers_the_room_as_room_extract_room(tmp_path, monkeypatch): +def test_background_numbers_the_room_as_hostspot_extract_room(tmp_path, monkeypatch): """Materials and textures of two serialized files that reuse path ids: the background record and room.glb agree (check_room), and a texture slot names the glTF texture, not the image.""" load_room(monkeypatch, two_file_room()) - doc = room.extract_room(RoomCatalog(), ROOM_KEY, tmp_path / "room.glb") + doc = hostspot.extract_room(RoomCatalog(), ROOM_KEY, tmp_path / "room.glb") gltf = storyhost.glb_json(tmp_path / "room.glb") bg = storyhost.background(RoomCatalog(), ROOM_KEY, "situation(Clone)", None) storyhost.check_room(ROOM_KEY, gltf, doc, bg) diff --git a/tests/helpers/spotobjects.py b/tests/helpers/spotobjects.py new file mode 100644 index 0000000..29ff81d --- /dev/null +++ b/tests/helpers/spotobjects.py @@ -0,0 +1,357 @@ +"""A synthetic spot as products' object tables (nnnotes.objects/1) whose files are in a depot: a background prefab +(content.room) with drawn, inactive, hidden, mesh-less and empty meshes, two materials of two shaders and their +textures; a situation prefab (content.spot) with tap targets, Spine characters, two skeletons sharing one atlas; and +the master tables of the spot. Shaders and materials are made as typetrees and turned into their object documents +(formats.shader.parsed_summary, unity.convert.material.material_doc).""" + +from __future__ import annotations + +import io + +import numpy as np +from PIL import Image + +from nnnotes.formats.shader import parsed_summary +from nnnotes.pipeline import contract +from nnnotes.unity.atoms import gltf +from nnnotes.unity.atoms.mesh import arrays +from nnnotes.unity.atoms.reader import mesh_input_from_typetree +from nnnotes.unity.convert.material import material_doc +from tests.helpers.fakeunity import F32, NORMAL, POSITION, UV0, VERSION, mesh_tt, submesh + +FIXED = "" +BACKGROUND = "Spot/home_a/Background/home_a" +SITUATION = "Spot/home_a/home_a_01/home_a_01" +CAB, SIT = "CAB-r", "CAB-s" +ROOM_ROOT, QUAD, QUAD2, POINTS, EMPTY = (f"{CAB}:{i}" for i in (1, 10, 11, 12, 13)) +MAT_WALL, MAT_FLOOR, UNLIT, LIT, TEX_WALL, TEX_FLOOR = (f"{CAB}:{i}" for i in (20, 21, 30, 31, 40, 41)) +SIT_ROOT, SDA, SDA2, ATLAS, SKEL, SKEL2, ATLAS_TEXT_ASSET = (f"{SIT}:{i}" for i in (1, 50, 51, 52, 60, 61, 62)) +PAGE_MAT1, PAGE_MAT2, PAGE1, PAGE2 = (f"{SIT}:{i}" for i in (70, 71, 80, 81)) +SPINE = "spine-unity|Spine.Unity|" +GAME = "Assembly-CSharp||" +ATLAS_TEXT = "\np1.png\nsize: 2,2\nformat: RGBA8888\nr\n bounds: 0,0,1,1\n\np2.png\nsize: 1,1\nr2\n bounds: 0,0,1,1\n" +V0, V1, Q0 = {"x": 0.0, "y": 0.0, "z": 0.0}, {"x": 1.0, "y": 1.0, "z": 1.0}, {"x": 0.0, "y": 0.0, "z": 0.0, "w": 1.0} +ROOM_OFFSET = {"x": 1.0, "y": 0.0, "z": 2.0} # the background root's own position +VERTICES = [(0.0, 0.0, 0.5), (1.0, 0.0, 0.5), (1.0, 1.0, 0.5), (0.0, 1.0, 0.5)] +UVS = [(0.0, 0.0), (1.0, 0.0), (1.0, 1.0), (0.0, 1.0)] + + +def ref(oid: str | None, cls: str | None = None): + if oid is None: + return None + file, _, pid = oid.rpartition(":") + r = {"file": file, "pathId": pid} + if cls: + r["class"] = cls + return {"$ref": r} + + +# ---------------------------------------------------------------- shader and material typetrees +def fv(val, name=FIXED): + return {"val": val, "name": name} + + +def shader(name="Universal Render Pipeline/Lit", props=None, tags=(), **state): + """A shader typetree: one subshader, one pass with render target 0 and the other states given (defaults: + opaque, back-face culling, depth write on, LEqual, all colour channels).""" + rt = { + "srcBlend": fv(1.0), + "destBlend": fv(0.0), + "srcBlendAlpha": fv(1.0), + "destBlendAlpha": fv(0.0), + "blendOp": fv(0.0), + "blendOpAlpha": fv(0.0), + "colMask": fv(15.0), + } + st = {"culling": fv(2.0), "zWrite": fv(1.0), "zTest": fv(4.0), "alphaToMask": fv(0.0)} + for k, v in state.items(): + (rt if k in rt else st)[k] = v + return { + "m_ParsedForm": { + "m_Name": name, + "m_PropInfo": { + "m_Props": [ + { + "m_Name": n, + "m_Type": 2, + "m_DefValue[0]": d, + "m_DefValue[1]": 0.0, + "m_DefValue[2]": 0.0, + "m_DefValue[3]": 0.0, + } + for n, d in (props or {}).items() + ] + }, + "m_SubShaders": [ + { + "m_Tags": {"tags": list(tags)}, + "m_Passes": [{"m_State": {"rtBlend0": rt, **st}, "m_Tags": {"tags": []}}], + } + ], + } + } + + +def material(floats=None, ints=None, name="m", textures=None, colors=None, shader_ref=None): + """A material typetree; `textures` {slot: (texture reference, scale (x, y), offset (x, y))}.""" + return { + "m_Name": name, + "m_Shader": shader_ref, + "m_SavedProperties": { + "m_TexEnvs": [ + [k, {"m_Texture": t, "m_Scale": {"x": s[0], "y": s[1]}, "m_Offset": {"x": o[0], "y": o[1]}}] + for k, (t, s, o) in (textures or {}).items() + ], + "m_Ints": list((ints or {}).items()), + "m_Floats": list((floats or {}).items()), + "m_Colors": list((colors or {}).items()), + }, + } + + +def material_json(tt: dict) -> dict: + """The `json` of a Material of typetree `tt` (its references already {"$ref"}).""" + return material_doc(tt, lambda r: r) + + +def shader_json(tt: dict) -> dict: + return parsed_summary(tt) + + +# ---------------------------------------------------------------- object files +def png(color, size=(2, 2)) -> bytes: + buf = io.BytesIO() + Image.new("RGBA", size, color).save(buf, format="PNG") + return buf.getvalue() + + +def mesh_glb(name: str, indices=(), submeshes=()) -> bytes: + """The `mesh` of a Mesh: four vertices (VERTICES, UVS, normals -z) with `indices` in `submeshes`.""" + ch = { + POSITION: (F32, 3, np.array(VERTICES)), + NORMAL: (F32, 3, np.tile([0.0, 0.0, -1.0], (4, 1))), + UV0: (F32, 2, np.array(UVS)), + } + return gltf.write([arrays(mesh_input_from_typetree(mesh_tt(name, 4, ch, list(indices), list(submeshes)), VERSION))]) + + +def texture(name: str, filter_mode: int, wrap: int) -> dict: + return { + "m_Name": name, + "m_MipCount": 1, + "m_TextureSettings": {"m_FilterMode": filter_mode, "m_WrapU": wrap, "m_WrapV": wrap}, + } + + +def node(path: str, *components, position=V0, active=True) -> dict: + return { + "path": path, + "name": path.rsplit("/", 1)[-1], + "active": active, + "localPosition": position, + "localRotation": Q0, + "localScale": V1, + "components": list(components), + } + + +def component(type_: str, **fields) -> dict: + return {"type": type_, "pathId": "0", "fields": fields} + + +def mono(script: str, **fields) -> dict: + return {"type": "MonoBehaviour", "pathId": "0", "script": script, "fields": fields} + + +def drawn(path: str, mesh: str | None, materials, enabled=1, **kw) -> dict: + return node( + path, + component("MeshFilter", m_Mesh=ref(mesh, "Mesh")), + component("MeshRenderer", m_Enabled=enabled, m_Materials=[ref(m, "Material") for m in materials]), + **kw, + ) + + +# ---------------------------------------------------------------- the background +def room_objects() -> dict: + """{object id: (class, name, files, entry extras)} of the background BACKGROUND: under the root `room` (at + ROOM_OFFSET) `wall` (QUAD, MAT_WALL), `floor` (inactive, QUAD2: two submeshes, the second without a material), + `wall2` (QUAD and MAT_WALL again), `hidden` (a disabled renderer), `nomesh` (no mesh), `empty` (a mesh the + conversion left empty) and `points` (a mesh without an index buffer).""" + prefab = { + "name": "room", + "nodes": [ + node("room", position=ROOM_OFFSET), + drawn("room/wall", QUAD, [MAT_WALL]), + drawn("room/floor", QUAD2, [MAT_FLOOR, None], active=False), + drawn("room/wall2", QUAD, [MAT_WALL]), + drawn("room/hidden", QUAD, [MAT_WALL], enabled=0), + drawn("room/nomesh", None, []), + drawn("room/empty", EMPTY, [MAT_WALL]), + drawn("room/points", POINTS, [MAT_WALL]), + ], + } + unlit = shader(name="Unlit/Transparent", srcBlend=fv(5.0), destBlend=fv(10.0)) + wall = material( + name="a_wall", + shader_ref=ref(UNLIT, "Shader"), + textures={"_MainTex": (ref(TEX_WALL, "Texture2D"), (1.0, 1.0), (0.0, 0.0)), "_Mask": (None, (1, 1), (0, 0))}, + ) + floor = material( + name="a_floor", + shader_ref=ref(LIT, "Shader"), + textures={"_BaseMap": (ref(TEX_FLOOR, "Texture2D"), (2.0, 2.0), (0.0, 0.5))}, + floats={"_Metallic": 0.0, "_Smoothness": 0.25}, + colors={"_BaseColor": {"r": 1.0, "g": 0.5, "b": 0.25, "a": 1.0}}, + ) + return { + ROOM_ROOT: ("GameObject", "room", {"json": prefab}, {}), + QUAD: ("Mesh", "quad", {"mesh": mesh_glb("quad", [0, 1, 2, 0, 2, 3], [submesh(0, 6)])}, {}), + QUAD2: ("Mesh", "quad2", {"mesh": mesh_glb("quad2", [0, 1, 2, 0, 2, 3], [submesh(0, 3), submesh(3, 3)])}, {}), + POINTS: ("Mesh", "points", {"mesh": mesh_glb("points")}, {}), + EMPTY: ( + "Mesh", + "empty_mesh", + {"json": {"m_Name": "empty_mesh"}}, + {"status": "generic", "reason": "empty.mesh"}, + ), + MAT_WALL: ("Material", "a_wall", {"json": material_json(wall)}, {}), + MAT_FLOOR: ("Material", "a_floor", {"json": material_json(floor)}, {}), + UNLIT: ("Shader", "Unlit/Transparent", {"json": shader_json(unlit)}, {}), + LIT: ("Shader", "Universal Render Pipeline/Lit", {"json": shader_json(shader())}, {}), + TEX_WALL: ("Texture2D", "wall_a", {"json": texture("wall_a", 1, 1), "image": png((200, 120, 40, 255))}, {}), + TEX_FLOOR: ("Texture2D", "floor_b", {"json": texture("floor_b", 0, 0), "image": png((9, 8, 7, 255))}, {}), + } + + +# ---------------------------------------------------------------- the situation +ANIMATION = { + "_animationName": "idle", + "loop": 1, + "timeScale": 1.0, + "initialSkinName": "default", + "initialFlipX": 0, + "initialFlipY": 0, + "pmaVertexColors": 1, + "tintBlack": 0, + "zSpacing": 0.0, +} +HEADER = {"m_Enabled": 1, "m_Script": ref("CAB-x:9", "MonoScript"), "m_Name": ""} +SITUATION_OFFSET = {"x": 0.0, "y": 0.0, "z": 1.0} + + +def skeleton_data(name: str, skeleton: str) -> dict: + return { + "$script": SPINE + "SkeletonDataAsset", + "m_Name": name, + "skeletonJSON": ref(skeleton, "TextAsset"), + "atlasAssets": [ref(ATLAS, "MonoBehaviour")], + "scale": 0.01, + "defaultMix": 0.2, + "fromAnimation": ["a"], + "toAnimation": ["b"], + "duration": [0.5], + } + + +def situation_objects() -> dict: + """{object id: (class, name, files, entry extras)} of the situation SITUATION: the root `sit` (at + SITUATION_OFFSET) with SpotSituationSettings and SpotCharacterController; `sit/chr7` (character 7, no focus, a + Spine character without animation) before `sit/chr5` (character 5 at (2, 0, 0), focus `sit/chr5/focus` at + (0, 1, 0), a Spine character driving `sit/chr5/spine`'s SkeletonAnimation of SDA); skeletons SDA (binary .skel) + and SDA2 (JSON) sharing the two-page atlas ATLAS.""" + prefab = { + "name": "sit", + "nodes": [ + node( + "sit", + mono(GAME + "SpotSituationSettings", **HEADER, backgroundPosition=V0, cameraFov=30.0), + mono( + GAME + "SpotCharacterController", + **HEADER, + _spotSpineCharacters=[{"component": "MonoBehaviour", "gameObject": "sit/chr5"}], + _characters=[], + _speed=2.0, + ), + position=SITUATION_OFFSET, + ), + node( + "sit/chr7", + mono(GAME + "SpotCharacter", **HEADER, _characterId=7, _focus=None, _radius=0.5), + mono(GAME + "SpotSpineCharacter", **HEADER, _animation=None), + ), + node( + "sit/chr5", + mono(GAME + "SpotCharacter", **HEADER, _characterId=5, _focus={"transform": "sit/chr5/focus"}), + mono( + GAME + "SpotSpineCharacter", + **HEADER, + _animation={ + "component": "MonoBehaviour", + "gameObject": "sit/chr5/spine", + "class": "SkeletonAnimation", + }, + ), + position={"x": 2.0, "y": 0.0, "z": 0.0}, + ), + node("sit/chr5/focus", position={"x": 0.0, "y": 1.0, "z": 0.0}), + node( + "sit/chr5/spine", + mono(SPINE + "SkeletonAnimation", **HEADER, skeletonDataAsset=ref(SDA, "MonoBehaviour"), **ANIMATION), + ), + ], + } + page = lambda tex: {"json": material_json(material(textures={"_MainTex": (ref(tex, "Texture2D"), (1, 1), (0, 0))}))} # noqa: E731 + return { + SIT_ROOT: ("GameObject", "sit", {"json": prefab}, {}), + SDA: ("MonoBehaviour", "chr_SkeletonData", {"json": skeleton_data("chr_SkeletonData", SKEL)}, {}), + SDA2: ("MonoBehaviour", "chr2_SkeletonData", {"json": skeleton_data("chr2_SkeletonData", SKEL2)}, {}), + ATLAS: ( + "MonoBehaviour", + "chr_Atlas", + { + "json": { + "$script": SPINE + "SpineAtlasAsset", + "m_Name": "chr_Atlas", + "atlasFile": ref(ATLAS_TEXT_ASSET, "TextAsset"), + "materials": [ref(PAGE_MAT1, "Material"), ref(PAGE_MAT2, "Material")], + } + }, + {}, + ), + SKEL: ("TextAsset", "chr", {"json": {"m_Name": "chr"}, "data": b"\x00binary"}, {}), + SKEL2: ("TextAsset", "chr2", {"json": {"m_Name": "chr2"}, "data": b' {"skeleton": 1}'}, {}), + ATLAS_TEXT_ASSET: ("TextAsset", "chr", {"json": {"m_Name": "chr"}, "data": ATLAS_TEXT.encode()}, {}), + PAGE_MAT1: ("Material", "p1", page(PAGE1), {}), + PAGE_MAT2: ("Material", "p2", page(PAGE2), {}), + PAGE1: ("Texture2D", "p1", {"json": texture("p1", 1, 1), "image": png((1, 2, 3, 4))}, {}), + PAGE2: ("Texture2D", "p2", {"json": texture("p2", 1, 1), "image": png((9, 8, 7, 6), (1, 1))}, {}), + } + + +MASTER = { + "MasterHomeSpot": [ + {"_id": 3, "_nameTextId": 900, "_backgroundAssetPath": BACKGROUND, "_situationAssetPath": SITUATION}, + {"_id": 4, "_nameTextId": 901, "_backgroundAssetPath": BACKGROUND, "_situationAssetPath": "Spot/b/b_01/b_01"}, + ], + "MasterText": [{"_id": 900, "_japanese": "カフェ", "_english": "Cafe"}], + "MasterStoryHomeSpotTapTalkEpisode": [ + {"_spotId": 3, "_characterId": 7, "_advId": 71}, + {"_spotId": 4, "_characterId": 7, "_advId": 99}, + {"_spotId": 3, "_characterId": 5, "_advId": 51}, + {"_spotId": 3, "_characterId": 7, "_advId": 72}, + ], +} + + +def table(depot, objs: dict, roots) -> dict: + """The object table of `objs` (room_objects(), situation_objects()), every file put in `depot`.""" + out = {} + for oid, (cls, name, files, extra) in sorted(objs.items()): + recs = {} + for role, body in sorted(files.items()): + data = body if isinstance(body, bytes) else contract.encode(body) + ext = {"image": "png", "mesh": "glb"}.get(role, "bin") if isinstance(body, bytes) else "json" + recs[role] = {"sha256": depot.put(data), "size": len(data), "ext": ext} + out[oid] = {"status": "exported", "class": cls, "name": name, "files": recs, **extra} + return {"schema": contract.OBJECTS, "roots": sorted(roots), "objects": out, "missing": []} diff --git a/tests/pipeline/test_modelstages.py b/tests/pipeline/test_modelstages.py deleted file mode 100644 index 5edc140..0000000 --- a/tests/pipeline/test_modelstages.py +++ /dev/null @@ -1,295 +0,0 @@ -"""The Spine skeleton stage: its subjects, closures, keys and runs with synthetic censuses and objects.""" - -import subprocess -import sys -from pathlib import Path - -import pytest - -from nnnotes.pipeline import census, contract, link, modelstages -from nnnotes.pipeline.census import CensusStage -from nnnotes.pipeline.contract import Input, Task -from nnnotes.pipeline.depot import Depot -from nnnotes.pipeline.modelstages import SpineStage -from nnnotes.pipeline.stages import Env, Pending, describe, execute, registry - - -def test_the_libraries_are_those_of_the_extractor(): - from nnnotes.unity import export - - assert modelstages.LIBRARIES == export._LIBS - - -def test_importing_the_stages_does_not_import_unitypy(): - code = ( - "import sys, nnnotes.pipeline.modelstages as m; m.SpineStage(); " - "assert 'UnityPy' not in sys.modules, 'UnityPy imported'" - ) - r = subprocess.run([sys.executable, "-c", code], capture_output=True, text=True) - assert r.returncode == 0, r.stderr[-2000:] - - -# ---------------------------------------------------------------- spine.skeleton -CAB = {k: f"CAB-{c * 32}" for k, c in zip(("a", "t", "s", "d", "e", "m", "f"), "1234567")} - - -def obj(pid, cls, name=None, script=None): - o = {"pathId": pid, "classId": 0, "class": cls, "byteSize": 10, "typeHash": None} - if name is not None: - o["name"] = name - if script is not None: - o["script"] = {"file": CAB["m"], "pathId": script} - return o - - -def cdoc(files, scripts=()): - """A census document: files [(name, externals, objects)].""" - fs = [ - { - "name": n, - "unityVersion": "6000.0.0f1", - "platform": 13, - "typetree": True, - "externals": [{"file": x, "path": f"archive:/{x}/{x}", "guid": None, "type": 0} for x in ext], - "objects": objs, - } - for n, ext, objs in files - ] - classes = {} - for f in fs: - for o in f["objects"]: - classes[o["class"]] = classes.get(o["class"], 0) + 1 - return { - "schema": contract.CENSUS, - "files": fs, - "resources": [], - "assetBundles": [], - "resourcePaths": [], - "scripts": list(scripts), - "classes": classes, - "objects": sum(classes.values()), - "errors": [], - } - - -def spine_censuses(): - sda, atlas = 1, 2 - return { - "spine_a": cdoc( - [ - ( - CAB["a"], - [CAB["t"], CAB["m"], "unity default resources"], - [ - obj(10, "MonoBehaviour", "chr_SkeletonData", sda), - obj(11, "MonoBehaviour", "chr_Atlas", atlas), - obj(12, "TextAsset", "chr"), - ], - ) - ] - ), - "tex_b": cdoc([(CAB["t"], [CAB["s"]], [obj(1, "Texture2D", "chr"), obj(2, "Material", "chr_Material")])]), - "shader_c": cdoc([(CAB["s"], [], [obj(1, "Shader", "Spine/Skeleton")])]), - "other_d": cdoc([(CAB["d"], [], [obj(5, "MonoBehaviour", "x_SkeletonData", sda)])]), - "scene_e": cdoc( - [ - (CAB["e"], [], [obj(7, "GameObject", "root")]), - (CAB["e"] + ".sharedAssets", [CAB["t"]], [obj(7, "MonoBehaviour", "y_SkeletonData", sda)]), - ] - ), - "scripts": cdoc( - [(CAB["m"], [], [obj(1, "MonoScript", "SkeletonDataAsset"), obj(2, "MonoScript", "SpineAtlasAsset")])], - scripts=[ - { - "file": CAB["m"], - "pathId": 1, - "assembly": "spine-unity.dll", - "namespace": "Spine.Unity", - "class": "SkeletonDataAsset", - "name": "SkeletonDataAsset", - }, - { - "file": CAB["m"], - "pathId": 2, - "assembly": "spine-unity.dll", - "namespace": "Spine.Unity", - "class": "SpineAtlasAsset", - "name": "SpineAtlasAsset", - }, - ], - ), - "unsel_f": cdoc([(CAB["f"], [], [obj(3, "MonoBehaviour", "z_SkeletonData", sda)])]), - } - - -def census_result(depot, env, subject, doc, body): - inp = Input("bundle", contract.sha256(body), len(body), subject + ".bundle", ()) - t = describe(CensusStage(), subject, None, Env(depot, {"bundles": {subject: inp}})) - rec = contract.artifact( - contract.artifact_id(t.id, "census"), - depot.add(contract.encode(doc), "json"), - contract.provenance(t), - {"kind": "unity.census", "format": "json", "facts": census.facts(doc)}, - ) - depot.commit(contract.result(t, [rec], [])) - env.done(t.id, t.key) - return inp - - -def spine_env(depot, docs=None, data=None, selected=("other_d", "scene_e", "spine_a")): - docs = docs or spine_censuses() - data = data or {} - env = Env(depot, {"selected": sorted(selected)}) - env.facts["bundles"] = { - s: census_result(depot, env, s, d, data.get(s, b"UnityFS " + s.encode())) for s, d in sorted(docs.items()) - } - st = describe(link.ScriptsStage(), "all", None, env) - execute(st, depot, registry(link.ScriptsStage())) - env.done(st.id, st.key) - return env - - -def test_spine_subjects_closures_and_context(tmp_path): - stage, depot = SpineStage(), Depot(tmp_path / "s") - env = spine_env(depot) - assert stage.subjects(env) == ["other_d:5", "scene_e.sharedAssets:7", "spine_a:10"] - t = describe(stage, "spine_a:10", None, env) - assert t.id == "spine.skeleton:spine_a:10" and t.atoms == SpineStage.ATOMS and t.params == {} - assert [i.role for i in t.inputs] == ["bundle:scripts", "bundle:shader_c", "bundle:spine_a", "bundle:tex_b"] - assert t.inputs[2].sha256 == contract.sha256(b"UnityFS spine_a") - assert t.context == {"object": CAB["a"] + ":10"} - assert stage.depends("spine_a:10", env) == [ - "link.scripts:all", - "unity.census:scripts", - "unity.census:shader_c", - "unity.census:spine_a", - "unity.census:tex_b", - ] - e = describe(stage, "scene_e.sharedAssets:7", None, env) - assert [i.role for i in e.inputs] == ["bundle:scene_e", "bundle:shader_c", "bundle:tex_b"] - assert e.context == {"object": CAB["e"] + ".sharedAssets:7"} - assert [i.role for i in describe(stage, "other_d:5", None, env).inputs] == ["bundle:other_d"] - with pytest.raises(ValueError, match="no SkeletonDataAsset"): - describe(stage, "unsel_f:3", None, env) - - -def test_spine_keys_follow_the_closure_contents_only(tmp_path): - stage = SpineStage() - key = describe(stage, "spine_a:10", None, spine_env(Depot(tmp_path / "a"))).key - env = spine_env(Depot(tmp_path / "b"), data={"other_d": b"UnityFS other, changed"}) - assert describe(stage, "spine_a:10", None, env).key == key - env = spine_env(Depot(tmp_path / "c"), data={"shader_c": b"UnityFS shader, changed"}) - assert describe(stage, "spine_a:10", None, env).key != key - - -def test_spine_waits_for_censuses_and_the_script_table(tmp_path): - stage, depot = SpineStage(), Depot(tmp_path / "s") - env = spine_env(depot) - env.waiting("unity.census:tex_b") - with pytest.raises(Pending, match="unity.census:tex_b"): - stage.subjects(env) - env.waiting("unity.census:tex_b", failed=True) # a failed census is left out of the closures - assert [i.role for i in describe(stage, "spine_a:10", None, env).inputs] == ["bundle:scripts", "bundle:spine_a"] - env.waiting("link.scripts:all") - with pytest.raises(Pending, match="link.scripts:all"): - stage.subjects(env) - - -class FakeRef: - def __init__(self, path_id, target): - self.path_id, self.target = path_id, target - - def read(self): - return self.target - - -class FakeObj: - def __init__(self, tt, **fields): - self.tt, self.fields = tt, fields - - def read_typetree(self): - return self.tt - - def read(self): - return type("Read", (), self.fields)() - - -class FakeFile: - def __init__(self, objects): - self.objects = objects - - -class FakeEnv: - def __init__(self, files): - self.files = files - - def get_cab(self, name): - return self.files.get(name.lower()) - - -def fake_skeleton(pages=1): - from PIL import Image - - text = lambda name, s: type("T", (), {"m_Name": name, "m_Script": s})() # noqa: E731 - tex = type("Tex", (), {"m_Name": "chr", "m_TextureFormat": 4, "image": Image.new("RGBA", (1, 1))})() - mat = type( - "Mat", - (), - { - "m_SavedProperties": type( - "P", (), {"m_TexEnvs": [("_MainTex", type("E", (), {"m_Texture": FakeRef(1, tex)})())]} - )() - }, - )() - atlas = type( - "A", - (), - {"atlasFile": FakeRef(2, text("chr", "\nchr.png\nsize: 1,1\n")), "materials": [FakeRef(3, mat)] * pages}, - )() - return FakeObj( - {"m_Name": "chr_SkeletonData", "scale": 1.0, "defaultMix": 0.0}, - skeletonJSON=FakeRef(4, text("chr", b"\x01skel")), - atlasAssets=[FakeRef(11, atlas)], - ) - - -def test_spine_run_stores_the_files_of_the_skeleton(tmp_path, monkeypatch): - from nnnotes.unity import reader as unity - - stage, depot = SpineStage(), Depot(tmp_path / "s") - env = spine_env(depot) - for s in ("scripts", "shader_c", "spine_a", "tex_b"): - depot.put(b"UnityFS " + s.encode()) - loaded = [] - - def load_closure(paths): - loaded.append([Path(p).read_bytes() for p in paths]) - return FakeEnv({CAB["a"].lower(): FakeFile({10: sda})}) - - monkeypatch.setattr(unity, "load_closure", load_closure) - sda = fake_skeleton() - t = describe(stage, "spine_a:10", None, env) - assert execute(t, depot, registry(stage)).status == "ran" - assert loaded == [[b"UnityFS scripts", b"UnityFS shader_c", b"UnityFS spine_a", b"UnityFS tex_b"]] - doc = depot.result(t.key) - got = {contract.parse_artifact_id(a["id"])[1]: a for a in doc["artifacts"]} - assert {r: (a["semantics"]["kind"], a["content"]["ext"]) for r, a in got.items()} == { - "chr.skel": ("spine.skeleton", "skel"), - "chr.atlas": ("spine.atlas", "atlas"), - "chr.png": ("spine.page", "png"), - } - assert depot.read(got["chr.skel"]["content"]["sha256"]) == b"\x01skel" - assert got["chr.skel"]["semantics"]["facts"]["atlases"] == ["chr.atlas"] - (item,) = doc["items"] - assert item == { - "object": CAB["a"] + ":10", - "status": "exported", - "class": "MonoBehaviour", - "artifacts": sorted(a["id"] for a in doc["artifacts"]), - } - sda = fake_skeleton(pages=2) # a writer error: a failed item, cached - t2 = Task(t.stage, t.version, t.subject, {}, t.atoms, t.inputs[:3], t.context) # another key - assert execute(t2, depot, registry(stage)).result_status == "partial" - doc2 = depot.result(t2.key) - assert doc2["artifacts"] == [] and doc2["items"][0]["reason"]["code"] == "failed.export" - assert "1 pages vs 2 materials" in doc2["items"][0]["reason"]["message"] diff --git a/tests/pipeline/test_spot.py b/tests/pipeline/test_spot.py new file mode 100644 index 0000000..b34bb30 --- /dev/null +++ b/tests/pipeline/test_spot.py @@ -0,0 +1,103 @@ +"""The spot product stages (pipeline.spot) run on synthetic object tables.""" + +import json +import subprocess +import sys + +import pytest + +from nnnotes.content import spot +from nnnotes.pipeline import contract, layout +from nnnotes.pipeline.contract import Input, Task +from nnnotes.pipeline.depot import Depot +from nnnotes.pipeline.spot import RoomStage, SpotStage +from nnnotes.pipeline.stages import Env, execute, registry +from nnnotes.pipeline.views import ViewError +from tests.helpers import spotobjects as so + + +def master_dir(tmp_path): + md = tmp_path / "master" + md.mkdir() + for name, rows in so.MASTER.items(): + (md / f"{name}.json").write_text(json.dumps({"_allData": rows}), encoding="utf-8") + return md + + +def run(depot, stage, subject, objs, roots, *more): + data = contract.encode(so.table(depot, objs, roots)) + inp = Input("objects", depot.put(data), len(data), None, ({"kind": "depot"},)) + t = Task(stage.name, stage.version, subject, {}, {}, (inp, *more)) + execute(t, depot, registry(stage), force=True) + doc = depot.result(t.key) + arts = {contract.parse_artifact_id(a["id"])[1]: a for a in doc["artifacts"]} + return t, doc, arts + + +def test_the_stages_start_from_their_key(): + rooms, spots = RoomStage(), SpotStage() + assert rooms.keys(so.BACKGROUND, None) == [so.BACKGROUND] and spots.keys(so.SITUATION, None) == [so.SITUATION] + assert rooms.manifest(so.BACKGROUND) == "rooms/home_a.json" + assert spots.manifest(so.SITUATION) == "spots/home_a_01.json" + keys = [so.BACKGROUND, so.SITUATION, "Spot/b/b_01/b_01", "Spot/a/b/c", "Character/x"] + assert rooms.select(keys) == [so.BACKGROUND] + assert spots.select(keys) == ["Spot/b/b_01/b_01", so.SITUATION] + assert rooms.MASTER == () and spots.MASTER == spot.MASTER_TABLES + with pytest.raises(ValueError, match="not a spot situation key"): + spots.keys(so.BACKGROUND, None) + + +def test_the_room_product(tmp_path): + depot = Depot(tmp_path / "d") + t, doc, arts = run(depot, RoomStage(), so.BACKGROUND, so.room_objects(), [so.ROOM_ROOT]) + assert {p: a["semantics"]["kind"] for p, a in arts.items()} == { + "room.glb": "spot.room", + "room.json": "spot.room-summary", + "objects": "product.objects", + } + assert arts["room.glb"]["content"]["ext"] == "glb" + summary = contract.loads(depot.read(arts["room.json"]["content"]["sha256"])) + assert summary["meshCount"] == 3 and summary["glb"] == "room.glb" + table = contract.loads(depot.read(arts["objects"]["content"]["sha256"])) + assert table["objects"] == {} + assert doc["items"][0]["class"] == "SpotRoom" and doc["items"][0]["status"] == "exported" + manifest = layout.product_manifest(t.id, doc, table) + assert manifest["product"] == "room.gltf" and set(manifest["files"]) == {"room.glb", "room.json"} + + +def test_the_spot_product_reads_its_master_record(tmp_path): + depot = Depot(tmp_path / "d") + stage = SpotStage() + env = Env(depot, {"master": master_dir(tmp_path)}) + (master,) = stage.more_inputs(so.SITUATION, env) + assert master.role == "master" + assert contract.loads(depot.input_bytes(master)) == spot.master_record(so.MASTER, so.SITUATION) + t, doc, arts = run(depot, stage, so.SITUATION, so.situation_objects(), [so.SIT_ROOT], master) + assert {p: (a["semantics"]["kind"], a["content"]["ext"]) for p, a in arts.items()} == { + "spot.json": ("spot.situation", "json"), + "spine/chr2.json": ("spine.skeleton", "json"), + "spine/chr.skel": ("spine.skeleton", "skel"), + "spine/chr.atlas": ("spine.atlas", "atlas"), + "spine/p1.png": ("spine.texture", "png"), + "spine/p2.png": ("spine.texture", "png"), + "objects": ("product.objects", "json"), + } + doc_json = contract.loads(depot.read(arts["spot.json"]["content"]["sha256"])) + assert doc_json["spotId"] == 3 and doc_json["tapTalk"] == {"5": [51], "7": [71, 72]} + assert doc["items"][0]["class"] == "SpotSituation" + + +def test_a_master_table_the_master_data_lacks_is_an_error(tmp_path): + md = master_dir(tmp_path) + (md / "MasterText.json").unlink() + with pytest.raises(ViewError, match="master table MasterText is not in the master data"): + SpotStage().more_inputs(so.SITUATION, Env(Depot(tmp_path / "d"), {"master": md})) + + +def test_planning_the_stages_does_not_import_unitypy(): + code = ( + "import sys, nnnotes.pipeline.spot as m; m.RoomStage(); m.SpotStage(); " + "print(sorted(x for x in ('UnityPy', 'numpy', 'PIL') if x in sys.modules))" + ) + r = subprocess.run([sys.executable, "-c", code], capture_output=True, text=True) + assert r.returncode == 0 and r.stdout.strip() == "[]", r.stderr[-2000:] diff --git a/tests/unity/test_speed_exact.py b/tests/unity/test_speed_exact.py index 24fa7c5..45c9a4b 100644 --- a/tests/unity/test_speed_exact.py +++ b/tests/unity/test_speed_exact.py @@ -425,7 +425,7 @@ def ref_dump_objects(shaders, out_dir: Path, source: str, index: list) -> list: continue done.add(name) safe = shader._safe(name) - jsonio.write_json(out_dir / f"{safe}.json", shader._parsed_summary(tt), default=str) + jsonio.write_json(out_dir / f"{safe}.json", shader.parsed_summary(tt), default=str) rec = {"name": name, "source": source, "parsed": f"{safe}.json", "variants": []} for plat, si, pi, stage, n, t, kws, code in shader.variants(tt, shader.platform_blobs(o.read())): tag = shader.PLATFORM.get(plat, str(plat))