Skip to content

Dynamo 1.4+ bump: bundled NATS disabled by default (TCP request plane, ZMQ KV events) #1836

Description

@yuanchen8911

Summary

Upstream Dynamo is changing its transport defaults: 1.4+ defaults to TCP for the request plane and ZMQ for KV events, and the platform Helm chart disables bundled NATS by default (ai-dynamo/dynamo#11951, open at filing time). AICR pins dynamo-platform 1.2.0 with global.nats.install: true set explicitly, so nothing breaks today — but the next Dynamo bump should be a coordinated NATS removal, not a bare version bump.

Background

AICR installs the bundled NATS subchart because Dynamo 1.2 defaults the runtime event plane to NATS (recipes/components/dynamo-platform/values.yaml). In 1.4+ the event plane defaults to ZMQ, so keeping install: true would ship a NATS + JetStream StatefulSet (with a 10Gi PVC) that nothing uses by default. Explicit settings are preserved upstream, so the pinned 1.2.0 config is unaffected until we bump.

Scope (at the 1.4+ chart bump)

  • Drop global.nats.install: true and the nats.config.jetstream PVC block from recipes/components/dynamo-platform/values.yaml
  • Remove storageClassPaths: nats.config.jetstream.fileStore.pvc.storageClassName from the dynamo-platform entry in recipes/registry.yaml (currently the component's only storage-class injection point)
  • Remove the NATS nodeSelectorPaths/tolerationPaths under nodeScheduling.system in the same registry entry
  • Update the event-plane description in tests/manifests/dynamo-vllm-smoke-test.yaml (KV events flow over ZMQ end-to-end; no NATS relay)
  • Rewrite docs/integrator/eks-dynamo-networking.md: the TCP 4222 security-group rule becomes obsolete, but the TCP request plane and ZMQ KV events are direct frontend↔worker pod-to-pod connections crossing the same GPU↔system nodegroup SG boundary — verify which ports Dynamo 1.4 uses on a real EKS cluster and document those instead
  • Note the upgrade impact for standing clusters: an in-place helm upgrade accepting the new default removes bundled NATS and drops NATS_SERVER from workload specs, triggering rolling restarts (fresh bundle installs are unaffected)
  • Run make bom-docs and commit the regenerated BOM with the version bump

Non-goals

  • No change at the current 1.2.0 pin.
  • Workloads that explicitly need NATS (legacy transports, NATS event plane, request-trace sinks) can still opt in via global.nats.install=true or dynamo-operator.natsAddr upstream; AICR does not need to model that unless a recipe requires it.

Trigger

Blocked on the upstream chart release containing ai-dynamo/dynamo#11951 (expected in dynamo-platform 1.4.x; ETA requested from the Dynamo team).

Metadata

Metadata

Assignees

Labels

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions