Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/contracts.md
Original file line number Diff line number Diff line change
Expand Up @@ -218,7 +218,7 @@ Documents written by particular stages (artifacts of no object: their ids are `<
| Document | Stage | Schema |
|---|---|---|
| `nnnotes.catalog-index/1` | `catalog.index` | [catalog-index.schema.json](https://github.com/MetaSekaiLab/nnnotes/blob/main/src/nnnotes/schemas/catalog-index.schema.json) |
| `nnnotes.census/1` | `unity.census` | [census.schema.json](https://github.com/MetaSekaiLab/nnnotes/blob/main/src/nnnotes/schemas/census.schema.json) |
| `nnnotes.census/2` | `unity.census` | [census.schema.json](https://github.com/MetaSekaiLab/nnnotes/blob/main/src/nnnotes/schemas/census.schema.json) |
| `nnnotes.scripts/1` | `link.scripts` | [scripts.schema.json](https://github.com/MetaSekaiLab/nnnotes/blob/main/src/nnnotes/schemas/scripts.schema.json) |
| `nnnotes.addresses/1` | `link.addresses` | [addresses.schema.json](https://github.com/MetaSekaiLab/nnnotes/blob/main/src/nnnotes/schemas/addresses.schema.json) |
| `nnnotes.artifacts/1` | `link.artifacts` | [artifacts.schema.json](https://github.com/MetaSekaiLab/nnnotes/blob/main/src/nnnotes/schemas/artifacts.schema.json) |
Expand Down
2 changes: 1 addition & 1 deletion docs/schemas.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ schemas.load("plan") # the schema as a dict
| `catalog-diff` | `nnnotes.catalog-diff/1` | the difference between two catalog versions (`catalog diff`) |
| `catalog-index` | `nnnotes.catalog-index/1` | the index of a catalog pair: remote and APK catalogs, every location in catalog order |
| `catalogs` | `nnnotes.catalogs/1` | the catalog versions of a depot (`catalog versions`) |
| `census` | `nnnotes.census/1` | the census of one bundle: serialized files, type trees and objects, read without decoding |
| `census` | `nnnotes.census/2` | the census of one bundle: serialized files, type trees, objects and what they reference, read without decoding |
| `common` | | definitions shared by the other schemas |
| `cost` | `nnnotes.cost/1` | the measured cost of a task's last run |
| `coverage` | `nnnotes.coverage/1` | the coverage of a run per class and per reason code |
Expand Down
20 changes: 12 additions & 8 deletions docs/stages.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,16 +12,17 @@ The asset export is built from three layers:

## Stages

Every stage is at version 1, except `unity.census` (version 2: it lists the scripts animation clip bindings name)
Every stage is at version 1, except `unity.census` (version 3: it lists what every object references and the
hierarchy holding it)
and `cri.movie` (version 2: every stream kind, codec and channel of a USM), and `catalog.index` (version 2:
gzip input and Japanese remote asset placeholders).

| Stage | Subject | Inputs | Outputs |
|---|---|---|---|
| `catalog.index` | a catalog (remote + APK) | the catalog files | every location (with the bundle hash, crc and size), bundles and raw remote files |
| `unity.census` | a bundle, or a built-in file of the APK | the bundle | serialized files, externals, objects (file, path id, class, size, type hash, name, script, the scripts an AnimationClip's bindings name), container, scene hashes, dependencies, script fields |
| `unity.census` | a bundle, or a built-in file of the APK | the bundle | serialized files, externals, objects (file, path id, class, size, type hash, name, script, the scripts an AnimationClip's bindings name, the objects it references, the hierarchy holding it), container, scene hashes, dependencies, script fields |
| `link.scripts` | global | the censuses holding MonoScripts | serialized file and path id -> assembly, namespace, class |
| `unity.export` | a bundle, or a built-in file of the APK | the bundle, the script entries its MonoBehaviours use (and, for files without type trees, the type trees nnnotes ships for their classes) | the objects' artifacts and one item per object |
| `unity.export` | a bundle, or a built-in file of the APK | the bundle, the script entries its MonoBehaviours use (and, for files without type trees, the type trees nnnotes ships for their type hashes) | the objects' artifacts and one item per object |
| `sprite.crop` | a SpriteAtlas | the atlas' JSON, the images of its textures, the metas of the sprites of other bundles it packs | the sprites' images |
| `cri.audio` | ACB content | `acb` (a raw file of the catalog, or the `acb` artifact of a cue sheet held in a bundle), `awb` (an AWB the catalog pairs with it), `boot` (the game's boot data: the key's source) | FLAC per stream, `cues.json`, `streams.json` |
| `cri.movie` | USM content | `usm` (a raw file of the catalog, or the `usm` artifact of a movie held in a bundle), `boot` | every stream as stored (video, alpha, audio, subtitles), `movie.mkv` (or `movie.webm`) |
Expand Down Expand Up @@ -65,7 +66,9 @@ with its reason code; any other error of a converter gives it with `generic.erro

Every object a converter takes has a `json` artifact or is contained in another object's. Each item lists in
`refs` the objects its documents reference (by object id, other bundles' objects included, the objects of its own
prefab left out): the objects reachable from an object follow from the items alone.
prefab left out): the objects reachable from an object follow from the items alone. The census of a bundle lists the
same before anything is converted, as a superset: per object `refs`, every object its typetree references, and
`unit`, the root of the hierarchy whose prefab holds it.

A sprite's `json` holds `sprite` (its typetree), `mesh` (its geometry decoded: `positions` in the sprite's units,
`submeshes` of triangles, `uv0` when stored), `renderData` (`source`: `renderData` for the sprite's own render
Expand Down Expand Up @@ -93,10 +96,11 @@ subjects. Census always covers them; everything and `--select bundle:apk/*` expo
These files store no type trees. Objects of the engine's classes are read with the type trees UnityPy has for the
file's Unity version; MonoBehaviours with the type trees nnnotes ships for the game's classes it reads there
(`nnnotes.unity.scripttypes`, generated by `scripts/gen_typetrees.py`): the render pipeline assets, renderer data,
renderer features, post-process data and the game's settings objects. A shipped type tree is used only when the type
hash the file stores is the one it was made for and its read consumes exactly the object's bytes; the task's
context holds the shipped type trees its MonoBehaviours use, so a new shipped tree runs those tasks again. A
MonoBehaviour without one is `unsupported.typetree` with its header (`m_GameObject`, `m_Enabled`, `m_Script`,
renderer features, post-process data and the game's settings objects. A MonoBehaviour takes the shipped type tree
made for the type hash the file stores for it (the hash of its class's serialized layout), and the read must consume
exactly the object's bytes. The export task's context holds the shipped type trees its MonoBehaviours use, and the
census of a built-in file has the shipped type trees' id among its atoms, so a new shipped tree runs those tasks
again. A MonoBehaviour without one is `unsupported.typetree` with its header (`m_GameObject`, `m_Enabled`, `m_Script`,
`m_Name`) as its typetree JSON; inside a hierarchy its component keeps that header and the prefab records the
issue. PlayerSettings, whose type tree reads the stored layout right up to `m_ActiveColorSpace` (by its stored
type hash), gives the fields up to it.
Expand Down
Loading
Loading