feat(convert)!: write path ids as decimal strings in object JSON

docs/stages.md

@@ -49,17 +49,17 @@ with its reason code; any other error of a converter gives it with `generic.erro
 | Class | Converter | Artifacts | Atoms | Notes |
 |---|---|---|---|---|
 | Texture2D | `tex.png/2` | `image`: PNG of the first mip level (HDR ASTC: `.astc`), `json`: the typetree without `image data` (settings, colour space, mip count, stream data) | `texture.decode`, `png.encode`, `astc.container` | further mip levels as facts; a texture without pixels: `empty.texture` |
-| Sprite | `sprite.png/2` | `image`: PNG on the sprite's rect (packing undone, tight mask applied), `json`: the typetree, geometry, render data and image (see below) | `texture.decode`, `sprite.crop`, `png.encode`, `mesh.arrays` | the tasks of a bundle share the textures it decoded last; a sprite whose SpriteAtlas is in another bundle gets its `json` here (with its geometry and object facts) and its `image` from `sprite.crop` |
+| Sprite | `sprite.png/3` | `image`: PNG on the sprite's rect (packing undone, tight mask applied), `json`: the typetree, geometry, render data and image (see below) | `texture.decode`, `sprite.crop`, `png.encode`, `mesh.arrays` | the tasks of a bundle share the textures it decoded last; a sprite whose SpriteAtlas is in another bundle gets its `json` here (with its geometry and object facts) and its `image` from `sprite.crop` |
 | Mesh | `mesh.glb/2` | `mesh`: binary glTF in the mesh's local space, `json`: the typetree without the vertex, index and compressed mesh data | `mesh.arrays`, `gltf.write` | see `gltf.write` below; limits of the skin and blend shapes as issues of the artifact |
-| GameObject (a root) | `prefab.json/2` | `json`: the hierarchy's nodes (path, path ids, transform) with their components' fields | | the objects below it are `contained`; a root of a scene's file names its scene |
-| Material | `material.json/1` | `json`: shader, keywords, textures, floats, colours | | |
-| Shader | `shader.json/1` | `json`: the parsed form (the bytes of `nnnotes export shader`'s `<name>.json`), `program:<platform>/s<S>p<P>_<stage>_<N>`: each compiled sub-program as stored (`.glsl`, `.metal`, `.vkprog`), `index`: the variants (platform, subshader, pass, stage, GPU program type, keywords, program role), `typetree`: the typetree JSON | | a shader without a parsed form: its typetree with `generic.error` |
-| AnimationClip | `clip.json/2` | `json`: the Mecanim clip data and its bindings as stored (see below) | | a legacy or compressed clip: `generic.clip.legacy` |
-| AnimatorController | `controller.json/2` | `json`: name, clips (`$ref`), layers, state machines, parameters, default values | | an AnimatorOverrideController: `generic.controller.override` |
-| MonoBehaviour (not in a hierarchy) | `mono.json/2`, `cri.usm/1` | `json`: `$script` (assembly, namespace, class) and the typetree; a cue sheet asset also `acb`, a movie asset holding its USM also `usm` (see below) | `sniff` | the class from the task's script table |
+| GameObject (a root) | `prefab.json/3` | `json`: the hierarchy's nodes (path, path ids, transform) with their components' fields | | the objects below it are `contained`; a root of a scene's file names its scene |
+| Material | `material.json/2` | `json`: shader, keywords, textures, floats, colours | | |
+| Shader | `shader.json/2` | `json`: the parsed form (the bytes of `nnnotes export shader`'s `<name>.json`), `program:<platform>/s<S>p<P>_<stage>_<N>`: each compiled sub-program as stored (`.glsl`, `.metal`, `.vkprog`), `index`: the variants (platform, subshader, pass, stage, GPU program type, keywords, program role), `typetree`: the typetree JSON | | a shader without a parsed form: its typetree with `generic.error` |
+| AnimationClip | `clip.json/3` | `json`: the Mecanim clip data and its bindings as stored (see below) | | a legacy or compressed clip: `generic.clip.legacy` |
+| AnimatorController | `controller.json/3` | `json`: name, clips (`$ref`), layers, state machines, parameters, default values | | an AnimatorOverrideController: `generic.controller.override` |
+| MonoBehaviour (not in a hierarchy) | `mono.json/3`, `cri.usm/1` | `json`: `$script` (assembly, namespace, class) and the typetree; a cue sheet asset also `acb`, a movie asset holding its USM also `usm` (see below) | `sniff` | the class from the task's script table |
 | TextAsset | `data/2` | `data`: the stored bytes, `json`: the typetree without `m_Script` | `sniff` | extension from the container path, else from the content |
-| Font | `font/2` | `font`: the font file, `json`: the typetree without `m_FontData` | `sniff` | |
-| any other class (a MonoScript, an AssetBundle, ...) | `generic.json/2` | `json`: the typetree | `sniff` | an object that can be read only in part (see below): that part, `unsupported.typetree` |
+| Font | `font/3` | `font`: the font file, `json`: the typetree without `m_FontData` | `sniff` | |
+| any other class (a MonoScript, an AssetBundle, ...) | `generic.json/3` | `json`: the typetree | `sniff` | an object that can be read only in part (see below): that part, `unsupported.typetree` |
 
 Every object a converter takes has a `json` artifact or is contained in another object's. Each item lists in
 `refs` the objects its documents reference (by object id, other bundles' objects included, the objects of its own

src/nnnotes/unity/convert/__init__.py

@@ -27,6 +27,8 @@
 typetree JSON (a Texture2D, Mesh, TextAsset or Font without the bytes its other artifact holds).
 
 Object JSON (canon/1 in the object's own field order):
+- a path id is a 64-bit integer, written as its decimal string (a JSON number of that size does not survive a
+  reader that parses numbers as doubles), so the object a reference names is "<file>:<pathId>" as it is written;
 - a reference is {"$ref": {"file", "pathId"}}, with "class" and "name" when the target is in the bundle; a null
   reference is null. Inside a prefab, a GameObject, Transform or component of the same prefab is named by its path:
   {"gameObject": path}, {"transform": path}, {"component": class, "gameObject": path};
@@ -66,17 +68,17 @@
 # converter -> version
 CONVERTERS = {
     "tex.png": 2,
-    "sprite.png": 2,
+    "sprite.png": 3,
     "mesh.glb": 2,
-    "prefab.json": 2,
-    "material.json": 1,
-    "clip.json": 2,
-    "controller.json": 2,
-    "mono.json": 2,
+    "prefab.json": 3,
+    "material.json": 2,
+    "clip.json": 3,
+    "controller.json": 3,
+    "mono.json": 3,
     "data": 2,
-    "font": 2,
-    "shader.json": 1,
-    "generic.json": 2,
+    "font": 3,
+    "shader.json": 2,
+    "generic.json": 3,
     "cri.usm": 1,
 }
 # class -> the converters its objects may use (every class may fall back to generic.json)

src/nnnotes/unity/convert/objects.py

@@ -454,7 +454,7 @@ def dref(self, key: tuple) -> dict:
         """{"$ref"} of an object, with its class and name when it is in the bundle (recorded in `seen`)."""
         file, pid = key
         self.seen.add(object_id(file, pid))
-        d = {"file": label(file), "pathId": pid}
+        d = {"file": label(file), "pathId": str(pid)}
         r = self.readers.get(key)
         if r is not None:
             d["class"] = r.type.name
@@ -472,7 +472,7 @@ def ref(self, owner, pptr: dict):
         fid = int(pptr.get("m_FileID", 0) or 0)
         file = self.file_of(owner.assets_file.name, fid)
         if file is None:
-            return {"$ref": {"fileId": fid, "pathId": pid}}
+            return {"$ref": {"fileId": fid, "pathId": str(pid)}}
         key = (file, pid)
         if self._local is not None:
             local = self._local(key)

src/nnnotes/unity/convert/prefab.py

@@ -133,7 +133,7 @@ def _subtree(ctx: BundleObjects, h: Hierarchies, f: str, g, root_tf: int):
 def _component(ctx: BundleObjects, h: Hierarchies, o, root: tuple, unresolved: list) -> dict:
     key = ctx.key(o)
     tt = ctx.tt(key, whole=False)
-    out = {"type": o.type.name, "pathId": key[1]}
+    out = {"type": o.type.name, "pathId": str(key[1])}
     if o.type.name == "MonoBehaviour":
         entry = ctx.mono_script(key, tt)
         out["script"] = entry if entry and entry != NO_SCRIPT else None
@@ -164,8 +164,8 @@ def prefab(ctx: BundleObjects, info, h: Hierarchies, root: Root) -> None:
         node = {
             "path": g.path(tf),
             "name": go["m_Name"],
-            "pathId": go_pid,
-            "transformPathId": tf,
+            "pathId": str(go_pid),
+            "transformPathId": str(tf),
             "active": bool(go["m_IsActive"]),
             "layer": go["m_Layer"],
             "tag": go.get("m_Tag"),

tests/golden/sprite.crop.json

@@ -5,8 +5,8 @@
  },
  "fixtures": {
   "atlas": {
-   "key": "b1f819e7c9328fb5accdbd6366070fb94a45f592ae5cc1d1a531d4326c90d873",
-   "result": "6f2bc5adaf8e95fe9a17028d47fdac301436432523758fbfedc8a0034338945c"
+   "key": "8bff1abc736f8e76c8ff92e0f87f28317e25bcd2a59dfbe4b8753144b862566b",
+   "result": "746170244ebd43a73004fc8c369d817d50216f51062ab73ebb9b57fbb7f6ad2f"
   }
  },
  "libraries": {

tests/golden/unity.export.json

@@ -1,79 +1,79 @@
 {
  "atoms": {
   "astc.container": "nnnotes/1",
-  "clip.json": "clip.json/2",
-  "controller.json": "controller.json/2",
+  "clip.json": "clip.json/3",
+  "controller.json": "controller.json/3",
   "cri.usm": "cri.usm/1",
   "data": "data/2",
-  "font": "font/2",
-  "generic.json": "generic.json/2",
+  "font": "font/3",
+  "generic.json": "generic.json/3",
   "gltf.write": "numpy-2.4.6/1",
-  "material.json": "material.json/1",
+  "material.json": "material.json/2",
   "mesh.arrays": "numpy-2.4.6/1",
   "mesh.glb": "mesh.glb/2",
-  "mono.json": "mono.json/2",
+  "mono.json": "mono.json/3",
   "png.encode": "pillow-12.3.0/1",
-  "prefab.json": "prefab.json/2",
+  "prefab.json": "prefab.json/3",
   "reader": "fakeunity/1",
-  "shader.json": "shader.json/1",
+  "shader.json": "shader.json/2",
   "sniff": "nnnotes/1",
   "sprite.crop": "pillow-12.3.0+numpy-2.4.6/1",
-  "sprite.png": "sprite.png/2",
+  "sprite.png": "sprite.png/3",
   "tex.png": "tex.png/2",
   "texture.decode": "unitypy-1.25.3+texture2ddecoder-1.0.6+astc-encoder-py-0.1.12+pillow-12.3.0/1"
  },
  "fixtures": {
   "clip.json": {
-   "key": "c2211c68d3286636469c74023637e15b3c11dade0f858ab2380498aa319171eb",
-   "result": "1f6ad672b184cd8b4cf9eeeb9ee0b8512ba532eaf708cb36b86c538b19910b75"
+   "key": "5174601cce42b801f88496a18f30601c9b96a98415e5ea634b60ef5e6e65cc6e",
+   "result": "2f98d3aa5e65721ad5549b8d39a32d632a4056b62306ecae289a259f8dd73b96"
   },
   "controller.json": {
-   "key": "eaae782443b5874108271db45e6a0da404f91239f700dd827802081ba64dd184",
-   "result": "e63d57f287d5d388e1d6d2269bc03ffbf03165b11cb03a19d2ad003fb00cacf8"
+   "key": "47cb3924c6de41c3b40606809353c7f7b6e615a9fe4924772ceae7f6920274ab",
+   "result": "139a51c4e08d27211fdeb01d27cf9d81d0cd4256e7ee5d382b85ed088f3dea80"
   },
   "cri.usm": {
-   "key": "c69fabc7e24dd038b2375b59a0421cd514a8ea5e5dc3a21c181dda809902dc3f",
-   "result": "d829f22ea9a736a478f2963470f2bdca4d6a3200e40a510606956fe771d63e00"
+   "key": "8a8b0f2d4ef9b6e9bd697efe2b27a8e6a9ae891dfba203c33420fc6eefa241d3",
+   "result": "2031ae1e748492317d8565e02f1a7c745333ff9e026f9334711b0a4f48d64758"
   },
   "data": {
-   "key": "eec1daf7f06a873c434b64b5ab2d0629b61a71c331f16ed5434706b60ee3bee3",
-   "result": "493c173754a5a0088bc972fc8c373415d02b345060ab83df9ce41f8b0d0317f8"
+   "key": "313e7f1b4c9cad52dd5e62069ebf8372536735b7ab6548b41861cb923d7e63d2",
+   "result": "980c3dc7410587621768f517fd8b4fa34756dc10bdb3fbb8c7230e510c0a9f59"
   },
   "font": {
-   "key": "81155f5c29ca33162b4b0ee97d7559fb0460ad02c46a187097bb6e4893792b7c",
-   "result": "d0da497a3f6207d7e92e31ac4313f289eb483b5fcf0496476a3fb50e235b830a"
+   "key": "f75bcfd326253ea01d22297c76e9fd1b209bed5c7470acc7769cd025a5a1c3d6",
+   "result": "b1d520e50ffbc83211fd51c8f7a7e84657ad2e932f4077525f36e9862d8603fb"
   },
   "generic.json": {
-   "key": "76af5e1c7fd2b389a80c4725aa2b561ffb0ae552b0d65ce2eb4ad56710ae3201",
-   "result": "6d6d058a293360ee2065fbb205f35834198e7285a743f2e5b1dc847daa4cdad3"
+   "key": "dea8a6e77d1b1848d43b0ba28d16ca55812c455a0b254bb7a6703ea948e18b99",
+   "result": "097447e4235618804dcee54a5c5e9d27ec77a0012389b1c726fe463dca9694e3"
   },
   "material.json": {
-   "key": "c391d41f8afdc880c74e24f143560ff80c122c97f82241cd2b4f0cdf8f5fad1f",
-   "result": "0990d8b5ccdc04c6335b2779354b3d64727ebccc701490ffe6a46346719e90e2"
+   "key": "7d4b88d018c77afbc9a9c745667d489830c2bbf97a20efd8ca0660d0d35112cf",
+   "result": "f58509825571f63771d27fa675bb98b849a6e3d17afd9047eaf8e5c1ce86034e"
   },
   "mesh.glb": {
-   "key": "1e5a4fee78d3fd98334ce50a6b0b1cc56f5733252f2cb00ff974d515a362b911",
-   "result": "11949923ee05d67811b42f530d84313e9197afd74e203da44291761b8aa87994"
+   "key": "608ccea80ac1758cf022d3a21b9bca4bf61cfc4a716bbb05e58a5db26d531b0e",
+   "result": "04060bd45010e278b6659d5619ae5f9e4207211411f68573c95202119e8b4b44"
   },
   "mono.json": {
-   "key": "0aeed465e4e3f8a310b4a2bd3cc06fa899fcdd5fb0ae04d3a5c2a5ccd85d09ea",
-   "result": "9711bc6d921e7c49ef0b86b487808c12b95e106bb791655a880fef9257ec6da3"
+   "key": "a005d9bbfb99bf1d67498f26ec4fa2d1f59cab6f30361d93d41ae815f4e25311",
+   "result": "0337425cb6d6fb774c3c297756fc97cb6c416471833a5eff3eaa6cc0680874fb"
   },
   "prefab.json": {
-   "key": "0c801dd911545b249a8b629b3dfa05c7ea6c466482d7f93252f126717419dc94",
-   "result": "96507449031bd9d046088f01b64602d7191aa87d7cf5af29ecde677c051d20fe"
+   "key": "b5ae5dcac2a7c2bd8c49995a55397ae4539307c36425286b10c75cbaae4d2451",
+   "result": "c568f4b1773c3437518596374da5aaae7d0d9e5d54d436794410f528277a2c7a"
   },
   "shader.json": {
-   "key": "105e05e73efa131a339c80a936f0e8dc44a2cebe3c25d332f500b51871dfe83e",
-   "result": "0f670ee8b5f4ceb584329ecd3a4695d70dd5bf0b9690b4df63b87963fb83913b"
+   "key": "7b4f858df794ef86cc62c1a5201a630ae91ab86ff0e6dbf946d836cf29a3b309",
+   "result": "aab00b71441e5978a98431d780fa1c4facb394389ca66901414541a238583197"
   },
   "sprite.png": {
-   "key": "ab28c0f55b23cb58aaf5b94f4357231ab1f4be8ce16c6a02e69bfc1917a3509c",
-   "result": "82cf3a23fe661f24f5e9fc3189e2b70db39c287e39392397556e2b081beb4c2c"
+   "key": "f8fcd13c515ca0d0acd08cd983b23e917fbec42e7de0c351980b69cab4f4af09",
+   "result": "d053cdaf856f61c785987e324a847e04c4cdc0e9556059eba66dc737f7809531"
   },
   "tex.png": {
-   "key": "f5b96a51a6de3073767d863f7fbd7f835df241386588ae3fddbfa7eae4d02c5a",
-   "result": "ea26cd3f966ff9ebb9c7491f72e720aa311d5552f5ef0b136d9712958dac6ac7"
+   "key": "5659b2e28ffb247c956a6273a41b1f6c0196e64737060a7228fde0ff1d946c3e",
+   "result": "84ac758d275634016b105b57698e5cac22ac57379ea8783486b833f1650c27f5"
   }
  },
  "libraries": {

tests/helpers/live2dmodel.py

@@ -21,7 +21,7 @@
 
 def ref(oid: str, cls: str | None = None) -> dict:
     file, _, pid = oid.rpartition(":")
-    r = {"file": file, "pathId": int(pid)}
+    r = {"file": file, "pathId": pid}
     if cls:
         r["class"] = cls
     return {"$ref": r}
@@ -38,7 +38,7 @@ def moc3() -> bytes:
 
 
 def mono(script: str, **fields) -> dict:
-    return {"type": "MonoBehaviour", "pathId": 0, "script": script, "fields": fields}
+    return {"type": "MonoBehaviour", "pathId": "0", "script": script, "fields": fields}
 
 
 def node(path: str, *components) -> dict:
@@ -89,7 +89,7 @@ def prefab() -> dict:
                 mono(FRAMEWORK + "CubismPhysicsController", _rig=rig),
                 mono("Live2D.Cubism|Live2D.Cubism.Framework.MotionSync|CubismMotionSyncController"),
                 mono("Assembly-CSharp||Live2DMotionSyncCriAudioInput"),
-                {"type": "Animator", "pathId": 2, "fields": {}},
+                {"type": "Animator", "pathId": "2", "fields": {}},
             ),
             node("m001/Parameters"),
             node(
@@ -106,12 +106,12 @@ def prefab() -> dict:
             node(
                 "m001/Drawables/ArtMesh0",
                 mono("Live2D.Cubism|Live2D.Cubism.Rendering|CubismRenderer", _mainTexture=ref(TEX1, "Texture2D")),
-                {"type": "MeshRenderer", "pathId": 3, "fields": {"m_Materials": [ref(MAT0, "Material")]}},
+                {"type": "MeshRenderer", "pathId": "3", "fields": {"m_Materials": [ref(MAT0, "Material")]}},
             ),
             node(
                 "m001/Drawables/ArtMesh1",
                 mono("Live2D.Cubism|Live2D.Cubism.Rendering|CubismRenderer", _mainTexture=ref(TEX0, "Texture2D")),
-                {"type": "MeshRenderer", "pathId": 4, "fields": {"m_Materials": [ref(MAT1, "Material")]}},
+                {"type": "MeshRenderer", "pathId": "4", "fields": {"m_Materials": [ref(MAT1, "Material")]}},
             ),
         ],
     }