Need an operations runbook for Substrate operators

[git286]

---

Summary

There is no operational documentation for the people running Substrate.

An operator's questions are structurally different from an actor developer's. They are fleet-level and cross-component: is the platform healthy, which component owns this symptom, are we out of capacity, is one tenant hurting everyone, is a node degrading. Answering those means knowing which of seven-plus components to look at, and Substrate currently gives almost no help with that routing.

This issue asks for docs/operations.md: a symptom → component → signal → action runbook.

---

Scope boundary

This is the operator column of the audience split; the actor-developer guide is the other. They must stay separate documents.

	Substrate operator	Actor developer
Unit of concern	fleet, pools, nodes, components	one actor
Placement (pod/node)	primary diagnostic lever	not exposed — see #569
Cluster access	full	limited or none
Typical entry point	an alert, or a tenant complaint	their own app misbehaving

The overlap — "a tenant says their actor is slow, is it us or them?" — is the escalation boundary, and deserves its own section rather than being split across both docs.

---

What an operator has today

Capability	What exists	Coverage
Metrics	rpc.server.call.duration (ateapi, atelet), atenet.router.route.duration, atelet.snapshot.size	three instruments, three components
Dashboards	ate-grpc, ate-e2e-latency, ate-snapshot (tools/setup-gcp/dashboards/)	GCP only; no kind equivalent
Alerts	—	none ship. tools/setup-gcp has create dashboards; there is no create alerts
GKE scrape config	one PodMonitoring: atenet-router-monitoring.yaml	needed for the Envoy round-trip lines
kind scrape annotations	prometheus.io/scrape on ateapi, atelet, atenet-router	3 of 7+ workloads
Fleet CLI	kubectl ate get workers, get actors, get atespaces, top workers
Traces	--trace, Jaeger (kind) / Cloud Trace (GKE)	control-plane operations
Bounded metric labels	ate.workerpool.name, ate.worker.state, ate.sandbox.class (internal/ateattr/ateattr.go:50-56)	see the enum problem below

[HavenXia]

Typical entry point | an alert, or a tenant complaint

I think we has zero alerts today, so nobody enters through one. If the runbook assumes alert-driven operation, a minimal alert set is a prerequisite, should we create an issue to track that?