DEP: Documentation Structure Overhaul

[Jont828]

Summary

Restructure the Dynamo documentation site to address a long-standing problem flagged by NVIDIA QA: the docs mix high-level guides with component-specific implementation details and offer no defined path from Concepts to Quick Start to Reference. This DEP proposes a new top-level information architecture built on tab-based navigation — Home, User Guide, Developer Guide, Recipes, Reference, Blog, Community — that cleanly separates tutorial content (how to do things) from knowledge-bank content (architecture, concepts, components, features), surfaces deployment paths (Kubernetes and local) as first-class peers, and promotes runnable Recipes and Feature Benchmarks to their own catalogs. The K8s-specific walkthrough rewrite is covered by ai-dynamo/dynamo#8658 and slots into the Kubernetes deployment section defined here.

Status: in progress, details not finalized. The structure described below is implemented on branch dynamo-full-docs-refactor and is under active iteration in ai-dynamo/dynamo#10855. Section names, groupings, and page placement are still being refined — feedback on the tab boundaries and section grouping is welcome. (This DEP originally proposed a single flat sidebar; it has been updated to reflect the tab-based model adopted during implementation. See the Evolution note under Proposal.)

Motivation

QA feedback: the docs/ directory mixes high-level guides with component-specific implementation details, with no defined path from Concepts → Quick Start → Reference. Critical configuration options are explained in prose across multiple pages rather than consolidated in reference tables. Users cannot tell whether a given page is meant to teach them how to use Dynamo, explain how it works internally, or serve as a reference they look up later.

Concretely, the current sidebar has several structural problems:

1. Local and K8s deployment paths are not clearly separated. It is not obvious that Dynamo can be run either locally or on a Kubernetes cluster, nor which pages are relevant to each path. Kubernetes deployment has its own section, but local deployment content is not elevated to the same level — it is scattered across Getting Started, User Guides, and Backends.

2. No navigation overview within sections. Large sections like "Kubernetes Deployment" present a flat list of pages with no landing page explaining which ones to read, in what order, or what each covers. A new user landing on the section sees, for example:

   Quickstart · Installation Guide · Model Deployment Guide · DGDR Reference · Dynamo Operator · Service Discovery · Webhooks · Minikube Setup · Managing Models with DynamoModel · Autoscaling · Rolling Update · Developing with Tilt · Inference Gateway (GAIE) · Snapshot · Shadow Engine Failover · Disagg Communication

   There is no indication of which pages are part of the happy path and which are advanced or optional.

3. No separation between "do" and "understand". Tutorials and reference/explanation content are interleaved. A user looking for "how do I deploy on K8s?" lands next to "how does the request plane work internally?".

4. No distinction between required and optional content. Tutorials do not signal what is basic/required versus advanced/optional. The K8s section, for example, includes content on optional components (Grove, model caching) and prerequisites (GPU Operator, RDMA) that are never surfaced at the point in the deployment flow where the user needs them.

5. No top-down conceptual path. There is no high-level overview that explains how the pieces fit together without requiring the reader to work through all the component and backend documentation. It is not obvious, for instance, that backends are modular or that the profiler and planner are part of the operator. "Design Docs" exists but mixes a big-picture architecture page with deep dives into individual planes and component internals.

6. "User Guides" is a dumping ground. It mixes backend tutorials (SGLang/TRT-LLM/vLLM examples), feature explanations (fault tolerance, tool calling, reasoning, LoRA), use-case content (multimodal, diffusion, agents), developer content (writing Python workers, mocker), and observability — none of which belong together.

7. Cross-cutting features are scattered. Observability, fault tolerance, benchmarking, tool calling, etc. each have their own ad-hoc home, with no single section a user can scan to see what Dynamo can do.

8. K8s docs read like an encyclopedia, not a walkthrough. This is the specific problem covered by docs: refactor Kubernetes and update to include v1beta1 DGDR dynamo#8658: prerequisites (GPU Operator, RDMA, Grove, model caching) are documented somewhere but never surfaced at the point in the deployment flow where the user needs them. The structural changes in this DEP are a precondition for the docs: refactor Kubernetes and update to include v1beta1 DGDR dynamo#8658 rewrite to have a coherent home.

Proposal

Adopt a tab-based top-level information architecture. Instead of one long sidebar that interleaves every content type, the site is split into top-level tabs that each hold one kind of content. Within the User Guide tab, content flows from getting started → deployment → operations → a knowledge bank (concepts, components, backends) → use cases → integrations.

Evolution from the original proposal

The first version of this DEP proposed a single reorganized sidebar with top-level sections (Usage, Architecture Overview, Advanced, Developer's Guide, Resources). During implementation this evolved into tabs, because a single sidebar deep enough to hold tutorials, the full knowledge bank, developer docs, recipes, and reference became unwieldy to scan. Tabs let each audience (operator vs. contributor vs. someone copy-pasting a recipe) start in the right place, while skip-slug: true keeps page URLs derived from file paths so a page's URL does not depend on which tab it lives in. The design principles below are unchanged from the original proposal; only the top-level container (tabs vs. one sidebar) and some section names changed.

Design principles

1. Tutorials and knowledge bank are clearly separated. Every page has an obvious type: basic tutorial, advanced tutorial/use case, concept/explanation, or reference. This gives structure to where new pages belong and lets us surface the most important content at the top of the site.

2. Both deployment paths are first-class. It should be immediately clear that Dynamo can be run either locally or on a Kubernetes cluster, and that both are supported. Under User Guide → Model Deployment, "Kubernetes" and "Local Environment" are peer sub-sections, each slim — get the user to a working deployment, then link out to knowledge content for depth.

3. Tutorial pages are copy-paste friendly. Commands should work without modification; user-configurable values (names, namespaces, model IDs) should be set as environment variables at the top of the page. Tutorials explain only what you need to know to complete the task and link out to reference pages for depth.

4. Knowledge bank follows a "giving a talk" flow. Knowledge Base Overview (what Dynamo is and what it's made of) → Concepts (what you need to understand first) → Components (the actual pieces of the system) → Backends (engine-specific reference). Each concept or component is introduced at a high level in the Overview landing page, then covered in detail in its own section.

5. Advanced content is organized, not dumped. The old "User Guides" catch-all is split into Operations (disaggregated serving, performance tuning, fault tolerance, observability reference, benchmarking), Use Cases (agents, tool calling & reasoning, multimodal, diffusion, LoRA, DynoSim), and Integrations (LMCache, FlexKV, custom KV events).

6. Developer content is segregated into its own tab. Contributing, building from source, writing custom backends, mocker, and local K8s dev tooling (Minikube, Tilt) live in the Developer Guide tab, not mixed into user-facing tutorials.

7. Runnable content gets its own catalogs. Recipes (per-model deployment recipes) and Feature Benchmarks (reproducible A/B benchmark pages) are catalog-driven and promoted out of the prose docs — Recipes to its own tab, Feature Benchmarks to its own section in User Guide.

Top-level tabs

Tab	Purpose	Content type
Home	Landing page / front door into the docs	Navigation
User Guide	Everything a user needs: getting started, deployment, operations, knowledge bank, use cases, integrations	Tutorial + Explanation + Reference
Developer Guide	Contributing, building, writing backends, mocker, local K8s dev tooling	Tutorial + Reference
Recipes	Per-model deployment recipes (catalog)	Tutorial (runnable)
Reference	Compatibility, release artifacts, examples, glossary, API reference	Reference
Blog	Digest posts	Misc
Community	Community resources	Misc

Proposed navigation tree

This reflects the structure as built on dynamo-full-docs-refactor (docs/index.yml). Names and placement are still being refined.

Home
  └── Introduction                         ← getting-started/introduction.mdx (landing page)

User Guide
  ├── Getting Started
  │   ├── About                            ← getting-started/about.md
  │   ├── K8s Quickstart                   ← kubernetes/README.md
  │   └── Local Quickstart                 ← getting-started/quickstart.mdx
  ├── Model Deployment
  │   ├── Kubernetes
  │   │   ├── Overview                     ← getting-started/kubernetes-deployment.md
  │   │   ├── Installation Guide           ← kubernetes/installation-guide.md
  │   │   ├── Model Deployment Guide       ← kubernetes/model-deployment-guide.md
  │   │   ├── DGDR Reference               ← kubernetes/dgdr.md
  │   │   ├── Model Loading                ← Model Caching, ModelExpress
  │   │   ├── Observability                ← Metrics, Logging, Operator Metrics
  │   │   ├── Multinode                    ← Multinode Deployments, Grove, Topology-Aware Scheduling, Topology-Aware KV Transfer, LWS
  │   │   └── Cloud Providers              ← AWS (EKS, EFA, EFS, ECS), Azure (AKS, RDMA/IB, Storage, Lustre CSI, Spot VMs), GCP (GKE)
  │   └── Local Environment
  │       ├── Installation                 ← getting-started/local-installation.md
  │       ├── SGLang Deployments           ← backends/sglang/sglang-examples.md
  │       ├── TRT-LLM Deployments          ← backends/trtllm/trtllm-examples.md
  │       ├── vLLM Deployments             ← backends/vllm/vllm-examples.md
  │       └── Observability Setup          ← Overview, Prometheus + Grafana
  ├── Operations
  │   ├── Disaggregated Serving            ← features/disaggregated-serving/README.md (+ Sizing with AIConfigurator)
  │   ├── Performance Tuning               ← performance/tuning.md
  │   ├── Fault Tolerance                  ← Overview, Request Migration/Cancellation/Rejection, Graceful Shutdown, Testing
  │   ├── Observability Reference          ← Metrics, Health Checks, Tracing, Logging
  │   └── Benchmarking                     ← Dynamo Benchmarking, Mocker Trace Replay, Planner Replay Benchmarking
  ├── Feature Benchmarks                   ← benchmarks/README.mdx (catalog of reproducible A/B benchmark pages)
  ├── Knowledge Base
  │   ├── Overview                         ← design-docs/architecture.md
  │   ├── Concepts                         ← Architecture Flow, Disaggregated Serving, Distributed Runtime, Communication Planes (Discovery/Request/Event)
  │   ├── Components                        ← Kubernetes Operator, Frontend, Router, Planner, Profiler, KVBM
  │   └── Backends                          ← SGLang, TensorRT-LLM, vLLM
  ├── Use Cases
  │   ├── Overview                         ← features/README.md
  │   ├── Agents                           ← SGLang for Agentic Workloads, Agent Tracing, Agent Hints, Pi-Mono, ThunderAgent
  │   ├── Tool Calling & Reasoning         ← parsing, parser config/fallback, troubleshooting
  │   ├── Fastokens Tokenizer              ← features/tokenizer/README.md
  │   ├── LoRA Adapters                    ← features/lora/README.md
  │   ├── Inference Simulation (DynoSim)   ← Mocker, Runs, Sweeps, Planner Simulation Benchmarking
  │   ├── Multimodal                       ← Overview, Embedding Cache, Encoder Disaggregation, Multimodal KV Routing
  │   └── Diffusion (Preview)              ← Overview, FastVideo, vLLM-Omni, SGLang/TRT-LLM Diffusion
  └── Integrations                          ← LMCache, FlexKV, KV Events for Custom Engines

Developer Guide
  └── Developer's Guide
      ├── Contributing                     ← contribution-guide.md
      ├── Documentation Style Guide        ← documentation-style-guide.md
      ├── Building from Source             ← getting-started/building-from-source.md
      ├── Writing Custom Backends          ← Custom Backend Overview, Unified Backends, Python Workers, Runtime Containers
      ├── Metrics Developer Guide          ← observability/metrics-developer-guide.md
      ├── Mocker                           ← mocker/mocker.md
      ├── Minikube Setup                   ← kubernetes/deployment/minikube.md
      ├── Developing with Tilt             ← kubernetes/tilt-dev-setup.md
      └── Docs Website                     ← README.md

Recipes                                     ← recipes/README.mdx (per-model deployment recipe catalog)

Reference
  ├── Compatibility                        ← getting-started/compatibility.md (Support Matrix + Feature Matrix consolidated)
  ├── Release Artifacts                    ← reference/release-artifacts.md
  ├── Examples                             ← getting-started/examples.md
  ├── Glossary                             ← reference/glossary.md
  └── (hidden) API Reference               ← NIXL Connect API, Kubernetes API Reference

Blog                                        ← digest/index.mdx (Digest posts)

Community                                   ← community/README.md

Where current content moves

Current section	What happens
Getting Started	Split: Introduction → Home tab (landing page). About / K8s Quickstart / Local Quickstart → User Guide → Getting Started. Building from Source → Developer Guide. Compatibility → Reference.
Kubernetes Deployment	Becomes User Guide → Model Deployment → Kubernetes. Dev tools (Minikube/Tilt) → Developer Guide. Operator/component pages → Knowledge Base → Components → Kubernetes Operator. Content rewrite covered by ai-dynamo/dynamo#8658.
User Guides	Eliminated. Backend examples → Model Deployment → Local Environment. Cross-cutting capabilities → Operations. Multimodal/Diffusion/Agents/LoRA/DynoSim → Use Cases. Writing Python Workers / Mocker → Developer Guide.
Backends	Example pages → Local Environment. Reference guides, deep-dives, and observability → Knowledge Base → Backends (vLLM gains parity with SGLang/TRT-LLM).
Components	Move under Knowledge Base → Components. Component design docs move here from Design Docs (e.g., router-design.md → Router).
Integrations	Move under User Guide → Integrations.
Design Docs	Eliminated. architecture.md → Knowledge Base Overview. Conceptual docs (dynamo-flow, disagg-serving, distributed-runtime, communication planes) → Concepts. Component design docs → their respective Components sections.
Resources	Becomes the Reference tab. Support Matrix + Feature Matrix consolidated into a single Compatibility page.
Recipes / Benchmarks	Promoted to catalogs: Recipes tab and User Guide → Feature Benchmarks section.
Blog / Community	Blog (Digest) and Community each become their own tab.

Relationship to ai-dynamo/dynamo#8658

Issue ai-dynamo/dynamo#8658 covers the content rewrite of the K8s docs into a single linear walkthrough (modeled on Cluster API's quickstart), with prerequisites and optional components surfaced at the point in the flow where they become relevant, and DGDR as the primary deployment entrypoint. The draft of that content lives in the dynamo-full-docs-refactor branch.

This DEP is the structural container for that work: it defines the Kubernetes deployment section the rewrite lives in, and the parallel sections (Local Environment, Concepts, Components, Operations, etc.) that the K8s walkthrough links out to instead of duplicating. The two efforts should land together — restructuring without rewriting K8s leaves the walkthrough problem; rewriting K8s without restructuring leaves the new walkthrough in a section that still neighbors "Design Docs" and "User Guides".

Implementation

• PR: docs: restructure docs site into tab-based navigation dynamo#10855 — implements the tab structure and nav changes in docs/index.yml and fern/docs.yml, adds the Home landing page (getting-started/introduction.mdx), consolidates the support/feature matrices into compatibility.md, and adds URL redirects so existing links keep resolving.
• Branch: dynamo-full-docs-refactor in ai-dynamo/dynamo.
• This supersedes the earlier PR #9239, which reflected the pre-rebase single-sidebar approach and has been closed.

Alternate Solutions

1. Keep a single reorganized sidebar (the original proposal in this DEP). Reorganize into top-level sections (Usage, Architecture Overview, Advanced, Developer's Guide, Resources) without tabs. Rejected during implementation: a single sidebar deep enough to hold tutorials, the full knowledge bank, developer docs, recipes, and reference becomes hard to scan, and forces every audience through the same entry point. Tabs let each audience start in the right place.

2. Incremental cleanup of "User Guides" only. Split User Guides into per-topic top-level sections without touching the rest of the sidebar. Rejected: leaves the tutorial/knowledge-bank confusion intact, doesn't address the missing local-deployment track, and doesn't give the K8s rewrite a coherent peer section.

3. Rewrite K8s docs only (just docs: refactor Kubernetes and update to include v1beta1 DGDR dynamo#8658). Land the K8s walkthrough in the current sidebar. Rejected: the walkthrough has to link out for concepts, components, and features, and those targets are currently scattered or duplicated. The reader experience would regress as soon as they followed any link.

4. Adopt Diátaxis (tutorials / how-to / reference / explanation) as the top-level structure. Rejected for now: pure Diátaxis would force four parallel sections per topic, which is a heavier lift and tends to fragment content. The proposed structure is Diátaxis-flavored (tutorial vs. knowledge bank) without the full taxonomy.

5. Defer the restructure and ship K8s rewrite into current sidebar. Rejected: the QA feedback explicitly cites structure as the problem, not just K8s content. Shipping the K8s rewrite alone would leave that critique unaddressed.

Requirements

• The new index.yml MUST map every existing page to a location in the new tab structure with no orphans and no broken internal links.
• URL changes MUST have redirects so that external bookmarks and search results continue to resolve. Tabs MUST use skip-slug: true (except Recipes, which keeps its recipes/ prefix) so page URLs derive from file paths and stay independent of which tab a page lives in.
• Newly introduced collapsible sections (Model Deployment, Operations, Knowledge Base, Use Cases, and their sub-sections) SHOULD each have a landing/overview page.
• The Knowledge Base section MUST have an Overview landing page that gives a big-picture view of what Dynamo is made of and how the pieces relate; the current design-docs/architecture.md SHOULD be used as the basis but will likely need a rewrite.
• Pages that currently appear in multiple sections MUST be deduplicated to a single canonical location, with the other references converted to links.
• The K8s content rewrite from docs: refactor Kubernetes and update to include v1beta1 DGDR dynamo#8658 MUST land inside the new Kubernetes deployment section.

References

• Implementation PR: ai-dynamo/dynamo#10855 — docs: restructure docs site into tab-based navigation
• ai-dynamo/dynamo#8658 — docs: refactor Kubernetes and update to include v1beta1 DGDR
• Branch dynamo-full-docs-refactor in ai-dynamo/dynamo — implementation of the tab-based navigation and K8s walkthrough draft
• Superseded PR: ai-dynamo/dynamo#9239 (closed — pre-rebase single-sidebar approach)
• Cluster API quickstart — model for walkthrough-style K8s docs

[sttts]

Pretty much agreeing with everything written. This would be a giant improvement.

Some thoughts:

• Kube API docs need a place, and probably other component references too (--help output). I was tempted to put it under "Resources" in the current side-bar. Not seeing a canonical place in your new structure either.
• in the proposal, we have a clear separation of kube vs. non-kube only in usage, but not further down. In "Architecture Overview" we could squint and consider Operator the Kube part. Maybe call it Kubernetes Operator. My first thought was whether that isn't more confusing than before.
• An idea: we could prefix every Kube or CLI specific topic with an icon ,
• In Kubernetes-focused docs, I often notice the lack of link to API references when an API is mentioned, e.g. DynamoGraphDeployment in a text about DGDR. There should always be a quick way to look up details when a kind is mentioned.
• What I notice with the Kubernetes docs (the Kube project ones, not Dynamo) is that they are very verbose, lots of prose. If you want to lookup a concept or how things work, they are often not super helpful. I usually avoid them and rather turn to reference docs of any kind because it is fast to navigate. This often means that reading the code, googling or asking the agents is more fruitful than reading the docs. In other words: keep the reference affine reader in mind who wants to hop through docs and who mainly benefits from the connections between pages (vs. reading the raw documents in source code).

[harryskim]

Thank you so much for this DEP. The problems you outlined is spot on. We will be spending this month to restructure our docs. I think some content in the developer's guide might need to be placed elsewhere potentially, but that's pretty minor detail. Let me discuss with team and follow up you on slack.

Thank you again for your contributions!

[Jont828]

Update (2026-06-22): DEP body revised to reflect the implemented structure, and implementation PR linked.

The implementation on branch dynamo-full-docs-refactor evolved from the originally-proposed single flat sidebar into a tab-based navigation model. I've updated the issue body to match what's actually built. Summary of what changed:

• Top-level container is now tabs, not one sidebar: Home · User Guide · Developer Guide · Recipes · Reference · Blog · Community. Each content tab uses skip-slug: true so page URLs derive from file paths and don't depend on which tab a page lives in.
• The original sidebar sections map onto the User Guide tab: Getting Started → Model Deployment (Kubernetes / Local Environment) → Operations → Feature Benchmarks → Knowledge Base (Concepts / Components / Backends) → Use Cases → Integrations.
• Developer content moved to its own Developer Guide tab; Resources became the Reference tab (Support + Feature matrices consolidated into a single Compatibility page).
• Runnable content promoted to catalogs: a dedicated Recipes tab and a Feature Benchmarks section.
• The design principles, the QA-driven motivation, and the relationship to docs: refactor Kubernetes and update to include v1beta1 DGDR dynamo#8658 are unchanged.

Implementation PR: ai-dynamo/dynamo#10855. This supersedes #9239 (pre-rebase single-sidebar approach), which has been closed.

Details are still being finalized — feedback on the tab boundaries and section grouping is welcome. @sttts @harryskim, flagging since you commented on the original proposal.