feat(site): assemble immutable product snapshots

docs/products.md

@@ -542,3 +542,28 @@ every episode whose key the selection holds.
 `ui.pack` assembles one catalog key and its reachable objects. Its `ui.json` (`nnnotes.ui-pack/1`) names root prefabs, controllers, animation bindings and shaders. The product layout writes `index.json` (`nnnotes.ui-library/1`) over the selected UI products; all referenced data lives in their object tables. See [UI libraries](ui.md).
 
 A stage with `INCLUDE_FONTS = True` requests traversal through its referenced font objects. The request records these roots in `font_roots`; unit selection and assembly follow the same rule. Other products keep their own font selection behavior.
+
+## Site snapshots
+
+`nnnotes.site.products.Snapshot` takes a depot and product-layout manifests keyed by path. It builds each product's
+dependencies first, using `site.references` to rewrite the declared `parts` fields, `site.asset` for the files and
+`site.manifest` for their manifest. Every transformation is recorded as a depot task in `snapshot.run`.
+
+The immutable manifest path is `products/<sha256>.json`, where the content id is that of the manifest itself.
+References in a live, story or home host point at those immutable paths. A dependency change gives its parents
+new manifest contents while unrelated products keep their identities. Object ids and file roles stay unchanged.
+Only declared reference fields are rewritten; text and serialized component values retain their values.
+
+Stories retain the selected language groups in their part index. With no explicit language selection, the
+snapshot uses the languages whose UI and font products are available. An explicitly requested missing language
+or a missing required dependency is an error. Public aliases may use region directories; their `root` points
+back to the site root.
+
+Product file entries accept whole files, compressed files and split JSON objects, including object JSON roles.
+A split entry has `{parts, size}`, with `[key, asset, size]` rows, or `[key, asset, size, stored]` for compressed
+parts. Rejoining the key names and value texts reconstructs the exact minified document and its byte count.
+
+`nnnotes.site.publish.materialize` places a complete snapshot and its listing files from the depot. It writes
+assets, immutable manifests, public aliases, then indexes. A failed replacement stops publication before file
+collection and keeps the ownership journal for retry. After every replacement succeeds, files owned by the
+preceding layout that are outside the supplied complete reference closure are removed. Unowned files are kept.

docs/stages.md

@@ -343,3 +343,13 @@ The result's `entry.json` is a product file entry, either `{asset, size, stored?
 use the decoded content id; compressed bytes have their own depot content id. The result records both byte forms,
 and layout entries select the stored form. Replaying a task reuses its result; changing a parameter or encoder
 version gives a different task key.
+
+## site.references and site.manifest
+
+Both document stages take `source` and `spec` JSON inputs and emit `document.json`. `site.references` applies an
+explicit path-to-manifest map to declared reference fields and selects the requested language groups.
+`site.manifest` applies transformed file entries to a product while checking that its logical paths, object ids
+and file roles agree with the source. It also records the relative site root and optional default language.
+
+The snapshot assembler consumes these task results to build immutable product graphs; see
+[site snapshots](products.md#site-snapshots).

src/nnnotes/cli/assets.py

@@ -141,6 +141,8 @@ def matches(self, stage: str) -> bool:
     StageSource("story", "nnnotes.pipeline.story:StoryStage", output=True, product=True),
     StageSource("ui.pack", "nnnotes.pipeline.ui:UiStage", output=True, product=True),
     StageSource("site.asset", "nnnotes.pipeline.siteassets:AssetStage"),
+    StageSource("site.references", "nnnotes.pipeline.siteproducts:ReferencesStage"),
+    StageSource("site.manifest", "nnnotes.pipeline.siteproducts:ManifestStage"),
     StageSource("view.", "nnnotes.pipeline.views:stages", output=True),
     StageSource("view.voices", "nnnotes.pipeline.voices:VoiceStage", output=True),
 )

src/nnnotes/pipeline/layout.py

@@ -499,11 +499,23 @@ def diff(out_dir, entries_: list[dict]) -> dict:
     return {"write": write, "remove": remove, "keep": len(files) - write}
 
 
-def materialize(out_dir, name: str, params: dict, entries_: list[dict], source, *, link: str = "copy") -> dict:
+def materialize(
+    out_dir,
+    name: str,
+    params: dict,
+    entries_: list[dict],
+    source,
+    *,
+    link: str = "copy",
+    write_order=None,
+    fail_fast: bool = False,
+) -> dict:
     """Write layout `name` into `out_dir` (see the module documentation). `source(sha256) -> Path` gives the bytes
     (from_depot, from_layout); `link`: the placement (PLACEMENTS). -> {"manifest": the layout document (without
     the files not placed), "sha256": of its file, "write", "remove", "keep", "cloned", "linked", "copied",
-    "failed": [{path, message}] of the files not placed}."""
+    "failed": [{path, message}] of the files not placed}. `write_order(path)` orders replacements; `fail_fast`
+    raises on a failed replacement before removing owned files or updating the manifest, keeping the journal
+    for a subsequent run."""
     if link not in PLACEMENTS:
         raise ValueError(f"unknown placement {link!r}")
     out = Path(out_dir)
@@ -528,13 +540,15 @@ def restore(p):  # a replaced or removed hard link's depot object: read-only aga
             except (KeyError, OSError):
                 pass
 
-    for p in write:
+    for p in sorted(write, key=write_order) if write_order else write:
         sha, size = files[p]
         dst = out / p
         if not (p not in mine and _present(dst, size) and _same(dst, files[p])):
             try:
                 _place(Path(source(sha)), dst, link, stats, lambda p=p: restore(p))
             except OSError as e:
+                if fail_fast:
+                    raise LayoutError(f"could not publish {p}: {type(e).__name__}: {e.strerror or e}") from e
                 failed.append({"path": p, "message": f"{type(e).__name__}: {e.strerror or e}"})
     for p in remove:
         _remove(out, p, lambda p=p: restore(p))

src/nnnotes/pipeline/siteproducts.py

@@ -0,0 +1,115 @@
+"""Task-backed product documents with immutable references for a static site."""
+
+from __future__ import annotations
+
+import copy
+import json
+
+from ..core import jsonio
+from . import contract
+from .stages import Output, Stage
+
+REFERENCES, MANIFEST = "site.references", "site.manifest"
+
+
+def at(document, path):
+    value = document
+    for key in path:
+        if not isinstance(value, dict) or key not in value:
+            return None
+        value = value[key]
+    return value
+
+
+def select_languages(document, paths, languages):
+    document = copy.deepcopy(document)
+    for path in paths:
+        values = at(document, path)
+        if values is None:
+            continue
+        if not isinstance(values, dict):
+            raise ValueError(f"{'.'.join(path)}: language group must be an object")
+        missing = set(languages) - set(values)
+        if missing:
+            raise ValueError(f"{'.'.join(path)}: missing languages {', '.join(sorted(missing))}")
+        parent = at(document, path[:-1])
+        if not isinstance(parent, dict):
+            raise ValueError("language group parent must be an object")
+        parent[path[-1]] = {code: values[code] for code in languages}
+    return document
+
+
+class _DocumentStage(Stage):
+    def subjects(self, env):
+        return sorted(env.facts.get(self.name, {}))
+
+    def inputs(self, subject, env):
+        return env.fact(self.name)[subject]
+
+    def run(self, task, depot):
+        source = json.loads(depot.input_bytes(task.input("source")))
+        spec = json.loads(depot.input_bytes(task.input("spec")))
+        doc = self.transform(source, spec)
+        content = depot.add(
+            jsonio.dumps(doc, ensure_ascii=False, separators=(",", ":"), sort_keys=True).encode(), "json"
+        )
+        art = contract.artifact(
+            contract.artifact_id(task.id, "document.json"), content, contract.provenance(task), {"kind": self.name}
+        )
+        return Output([art])
+
+    def transform(self, source, spec):
+        raise NotImplementedError
+
+
+class ReferencesStage(_DocumentStage):
+    """Rewrite only declared product-reference fields and language groups."""
+
+    name, version = REFERENCES, 1
+
+    def transform(self, source, spec):
+        doc = select_languages(source, spec["languagePaths"], spec["languages"])
+
+        def rewrite(value):
+            if isinstance(value, str):
+                if value not in spec["routes"]:
+                    raise ValueError(f"product reference has no manifest: {value}")
+                return spec["routes"][value]
+            if isinstance(value, dict):
+                return {key: rewrite(v) for key, v in value.items()}
+            if isinstance(value, list):
+                return [rewrite(v) for v in value]
+            if value is not None:
+                raise ValueError("product reference must be a path, group or null")
+            return None
+
+        for path in spec["paths"]:
+            value = at(doc, path)
+            if value is not None:
+                parent = at(doc, path[:-1])
+                if not isinstance(parent, dict):
+                    raise ValueError("product reference parent must be an object")
+                parent[path[-1]] = rewrite(value)
+        return doc
+
+
+class ManifestStage(_DocumentStage):
+    """Apply transformed file entries while preserving product and object identity."""
+
+    name, version = MANIFEST, 1
+
+    def transform(self, source, spec):
+        if source.get("schema") != contract.PRODUCT:
+            raise ValueError("site manifest requires a product")
+        doc = copy.deepcopy(source)
+        if set(doc["files"]) != set(spec["files"]) or set(doc["objects"]) != set(spec["objects"]):
+            raise ValueError("site manifest file or object identity differs")
+        doc["files"] = spec["files"]
+        for oid, entry in doc["objects"].items():
+            if set(entry["files"]) != set(spec["objects"][oid]):
+                raise ValueError(f"site manifest object {oid}: file roles differ")
+            entry["files"] = spec["objects"][oid]
+        doc["root"] = spec.get("root", "../")
+        if spec.get("language"):
+            doc["language"] = spec["language"]
+        return doc

src/nnnotes/schemas/product.schema.json

@@ -24,7 +24,7 @@
     "files": {
       "type": "object",
       "additionalProperties": {
-        "$ref": "#/$defs/asset"
+        "$ref": "#/$defs/file"
       }
     },
     "objects": {
@@ -41,7 +41,7 @@
           "files": {
             "type": "object",
             "additionalProperties": {
-              "$ref": "#/$defs/asset"
+              "$ref": "#/$defs/file"
             }
           }
         },
@@ -50,10 +50,55 @@
         ],
         "additionalProperties": false
       }
+    },
+    "root": {
+      "type": "string",
+      "minLength": 1
+    },
+    "language": {
+      "enum": [
+        "ja",
+        "en",
+        "zh-Hant",
+        "zh-Hans",
+        "ko"
+      ]
     }
   },
   "$defs": {
+    "file": {
+      "anyOf": [
+        {
+          "$ref": "#/$defs/whole"
+        },
+        {
+          "$ref": "#/$defs/encoded"
+        },
+        {
+          "$ref": "#/$defs/parts"
+        }
+      ]
+    },
     "asset": {
+      "type": "string",
+      "pattern": "^assets/[0-9a-f]{64}\\.[a-z0-9]+$"
+    },
+    "encodedAsset": {
+      "description": "An asset stored gzip- or brotli-encoded: the SHA-256 of its decoded bytes, the file's extension, then .gz or .br.",
+      "type": "string",
+      "pattern": "^assets/[0-9a-f]{64}\\.[a-z0-9]+\\.(gz|br)$"
+    },
+    "size": {
+      "description": "The byte count of the file (decoded).",
+      "type": "integer",
+      "minimum": 0
+    },
+    "stored": {
+      "description": "The byte count of an encoded asset as stored (less than size).",
+      "type": "integer",
+      "minimum": 0
+    },
+    "whole": {
       "type": "object",
       "required": [
         "asset",
@@ -62,12 +107,86 @@
       "additionalProperties": false,
       "properties": {
         "asset": {
-          "type": "string",
-          "pattern": "^assets/[0-9a-f]{64}\\.[0-9a-z]+$"
+          "$ref": "#/$defs/asset"
+        },
+        "size": {
+          "$ref": "#/$defs/size"
+        }
+      }
+    },
+    "encoded": {
+      "type": "object",
+      "required": [
+        "asset",
+        "size",
+        "stored"
+      ],
+      "additionalProperties": false,
+      "properties": {
+        "asset": {
+          "$ref": "#/$defs/encodedAsset"
+        },
+        "size": {
+          "$ref": "#/$defs/size"
+        },
+        "stored": {
+          "$ref": "#/$defs/stored"
+        }
+      }
+    },
+    "parts": {
+      "type": "object",
+      "required": [
+        "parts",
+        "size"
+      ],
+      "additionalProperties": false,
+      "properties": {
+        "parts": {
+          "type": "array",
+          "minItems": 1,
+          "items": {
+            "anyOf": [
+              {
+                "type": "array",
+                "prefixItems": [
+                  {
+                    "type": "string"
+                  },
+                  {
+                    "$ref": "#/$defs/asset"
+                  },
+                  {
+                    "$ref": "#/$defs/size"
+                  }
+                ],
+                "minItems": 3,
+                "maxItems": 3
+              },
+              {
+                "type": "array",
+                "prefixItems": [
+                  {
+                    "type": "string"
+                  },
+                  {
+                    "$ref": "#/$defs/encodedAsset"
+                  },
+                  {
+                    "$ref": "#/$defs/size"
+                  },
+                  {
+                    "$ref": "#/$defs/stored"
+                  }
+                ],
+                "minItems": 4,
+                "maxItems": 4
+              }
+            ]
+          }
         },
         "size": {
-          "type": "integer",
-          "minimum": 0
+          "$ref": "#/$defs/size"
         }
       }
     }