Add NVIDIA Dynamo#125
Open
wolfgang-desalvador wants to merge 16 commits into
Open
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds an NVIDIA Dynamo AKS inference example and reorganizes the repository’s example structure into clearer training vs inferencing workflows, while also expanding AKS shared-storage Helm charts (dynamic/static provisioning) and updating docs + CI workflow paths accordingly.
Changes:
- Added NVIDIA Dynamo inferencing example for AKS, including a Helm chart to create
DynamoGraphDeploymentRequestresources. - Reorganized/expanded training examples (NeMo Run, Megatron-LM GPT3-175B, LLM Foundry) with new Slurm scripts and multiple AKS Helm charts.
- Enhanced AKS shared storage Helm charts to support dynamic and static provisioning (Blobfuse, AMLFS, Azure Files Premium NFS) and updated docs + build workflows for new paths.
Reviewed changes
Copilot reviewed 34 out of 96 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| storage_references/aks/shared_storage/helm/blob-shared-storage/values.yaml | Adds storage.provisioning and static provisioning inputs for Blobfuse-backed RWX storage. |
| storage_references/aks/shared_storage/helm/blob-shared-storage/templates/storageclass.yaml | Renders StorageClass only for dynamic provisioning. |
| storage_references/aks/shared_storage/helm/blob-shared-storage/templates/pvc.yaml | Switches PVC behavior between dynamic StorageClass and static PV binding. |
| storage_references/aks/shared_storage/helm/blob-shared-storage/templates/pv.yaml | Adds static PV template for Blobfuse (existing storage account/container). |
| storage_references/aks/shared_storage/helm/blob-shared-storage/templates/NOTES.txt | Documents dynamic vs static Blob provisioning outputs and usage. |
| storage_references/aks/shared_storage/helm/blob-shared-storage/Chart.yaml | Updates chart description and bumps version. |
| storage_references/aks/shared_storage/helm/azurefiles-shared-storage/values.yaml | Introduces Azure Files Premium NFS shared storage chart values (dynamic/static). |
| storage_references/aks/shared_storage/helm/azurefiles-shared-storage/templates/storageclass.yaml | Dynamic StorageClass template for Azure Files NFS via CSI. |
| storage_references/aks/shared_storage/helm/azurefiles-shared-storage/templates/pvc.yaml | PVC template supporting both dynamic StorageClass and static PV binding. |
| storage_references/aks/shared_storage/helm/azurefiles-shared-storage/templates/pv.yaml | Static PV template for Azure Files NFS existing share. |
| storage_references/aks/shared_storage/helm/azurefiles-shared-storage/templates/_helpers.tpl | Standard Helm helper templates (names/labels). |
| storage_references/aks/shared_storage/helm/azurefiles-shared-storage/templates/NOTES.txt | Post-install notes for dynamic/static Azure Files NFS modes. |
| storage_references/aks/shared_storage/helm/azurefiles-shared-storage/Chart.yaml | Adds new chart metadata for Azure Files NFS shared storage. |
| storage_references/aks/shared_storage/helm/amlfs-shared-storage/values.yaml | Adds storage.provisioning and static provisioning inputs for AMLFS. |
| storage_references/aks/shared_storage/helm/amlfs-shared-storage/templates/storageclass.yaml | Renders StorageClass only for dynamic provisioning. |
| storage_references/aks/shared_storage/helm/amlfs-shared-storage/templates/pvc.yaml | Switches PVC behavior between dynamic StorageClass and static PV binding. |
| storage_references/aks/shared_storage/helm/amlfs-shared-storage/templates/pv.yaml | Adds static PV template for mounting an existing AMLFS instance. |
| storage_references/aks/shared_storage/helm/amlfs-shared-storage/templates/NOTES.txt | Documents dynamic vs static AMLFS provisioning behavior. |
| storage_references/aks/shared_storage/helm/amlfs-shared-storage/Chart.yaml | Updates chart description and bumps version. |
| storage_references/aks/shared_storage/README.md | Documents shared-storage options and usage across dynamic/static + Azure Files NFS. |
| infrastructure_references/aks/scripts/deploy-aks.sh | Adds Dynamo install/precheck/uninstall commands and env vars to AKS deployment script. |
| infrastructure_references/aks/README.md | Documents Dynamo support + updates command/environment variable listings. |
| examples/training/nemo-run/slurm/utils/run_tools.py | Adds helper for creating configured nemo_run.SlurmExecutor instances. |
| examples/training/nemo-run/slurm/utils/init.py | Marks the NeMo Run utils directory as a Python package. |
| examples/training/nemo-run/slurm/setup.sh | Adds a setup script to install uv + create env + install NeMo Run dependencies. |
| examples/training/nemo-run/slurm/images/user-create.png | Adds documentation image asset. |
| examples/training/nemo-run/slurm/images/select-kernel.png | Adds documentation image asset. |
| examples/training/nemo-run/slurm/images/ood-shell.png | Adds documentation image asset. |
| examples/training/nemo-run/slurm/README.md | Updates path references to new examples/training/... structure. |
| examples/training/nemo-run/slurm/.env-sample | Adds sample environment variables for NeMo Run Slurm usage. |
| examples/training/megatron-lm/GPT3-175B/slurm/download_slimpajama.py | Adds SlimPajama download helper for Slurm workflow. |
| examples/training/megatron-lm/GPT3-175B/slurm/README.md | Adds end-to-end Slurm training documentation for GPT3-175B. |
| examples/training/megatron-lm/GPT3-175B/slurm/04-gpt175B.sh | Adds Slurm training launcher script for GPT3-175B. |
| examples/training/megatron-lm/GPT3-175B/slurm/03-preprocess_dataset.sh | Adds preprocessing stage script using NeMo Framework Launcher. |
| examples/training/megatron-lm/GPT3-175B/slurm/02-extract_and_concat_dataset.sh | Adds extract/concat stage script using NeMo Framework Launcher. |
| examples/training/megatron-lm/GPT3-175B/slurm/00-setup_environment.sh | Adds environment setup script (images + repos) for Slurm training workflow. |
| examples/training/megatron-lm/GPT3-175B/docker/README.md | Documents Megatron-LM Docker image contents/build/usage. |
| examples/training/megatron-lm/GPT3-175B/docker/Dockerfile | Adds Megatron-LM container build definition. |
| examples/training/megatron-lm/GPT3-175B/aks/helm/prepare-data/values.yaml | Adds AKS Helm values for SlimPajama data prep pipeline. |
| examples/training/megatron-lm/GPT3-175B/aks/helm/prepare-data/templates/job-preprocess.yaml | Adds preprocessing Job (indexed parallelism) with dependency wait logic. |
| examples/training/megatron-lm/GPT3-175B/aks/helm/prepare-data/templates/job-extract.yaml | Adds extraction Job with dependency wait logic. |
| examples/training/megatron-lm/GPT3-175B/aks/helm/prepare-data/templates/job-download.yaml | Adds download Job (indexed parallelism) for SlimPajama shards. |
| examples/training/megatron-lm/GPT3-175B/aks/helm/prepare-data/templates/job-concatenate.yaml | Adds concatenation Job with dependency wait logic. |
| examples/training/megatron-lm/GPT3-175B/aks/helm/prepare-data/templates/configmap-scripts.yaml | Packages data prep Python scripts into a ConfigMap. |
| examples/training/megatron-lm/GPT3-175B/aks/helm/prepare-data/templates/_helpers.tpl | Adds Helm helpers for naming/labels. |
| examples/training/megatron-lm/GPT3-175B/aks/helm/prepare-data/scripts/preprocess.py | Implements preprocessing logic (BPE download + NeMo preprocessing invocation). |
| examples/training/megatron-lm/GPT3-175B/aks/helm/prepare-data/scripts/extract.py | Implements zstd extraction logic with worker sharding. |
| examples/training/megatron-lm/GPT3-175B/aks/helm/prepare-data/scripts/download_slimpajama.py | Implements SlimPajama download with parallel worker support for AKS jobs. |
| examples/training/megatron-lm/GPT3-175B/aks/helm/prepare-data/scripts/concatenate.py | Implements JSONL concatenation stage for downstream preprocessing. |
| examples/training/megatron-lm/GPT3-175B/aks/helm/prepare-data/Chart.yaml | Adds chart metadata for data prep pipeline. |
| examples/training/megatron-lm/GPT3-175B/aks/helm/megatron-training/values.yaml | Adds AKS values for Megatron-LM training job. |
| examples/training/megatron-lm/GPT3-175B/aks/helm/megatron-training/templates/pytorchjob.yaml | Adds Kubeflow PyTorchJob for distributed Megatron-LM training. |
| examples/training/megatron-lm/GPT3-175B/aks/helm/megatron-training/templates/configmap.yaml | Adds ConfigMap packaging training scripts. |
| examples/training/megatron-lm/GPT3-175B/aks/helm/megatron-training/templates/_helpers.tpl | Adds model size mapping helpers and standard labels. |
| examples/training/megatron-lm/GPT3-175B/aks/helm/megatron-training/templates/NOTES.txt | Adds post-install notes for monitoring/scaling. |
| examples/training/megatron-lm/GPT3-175B/aks/helm/megatron-training/scripts/train_megatron.sh | Adds training entrypoint script for the AKS PyTorchJob. |
| examples/training/megatron-lm/GPT3-175B/aks/helm/megatron-training/Chart.yaml | Adds chart metadata for Megatron training. |
| examples/training/megatron-lm/GPT3-175B/aks/README.md | Updates AKS documentation paths for reorganized training example location. |
| examples/training/llm-foundry/slurm/launch.sb | Adds Slurm launcher for LLM Foundry distributed training. |
| examples/training/llm-foundry/slurm/download_c4_dataset.sb | Adds Slurm dataset conversion job submission script for C4. |
| examples/training/llm-foundry/slurm/blobfuse.template.yaml | Adds Blobfuse2 config template for training data/checkpointing. |
| examples/training/llm-foundry/slurm/README.md | Adds end-to-end Slurm guide for LLM Foundry training + storage options. |
| examples/training/llm-foundry/docker/README.md | Updates build instructions path for reorganized directory structure. |
| examples/training/llm-foundry/docker/Dockerfile | Adds container build for LLM Foundry with DOCA tooling + topology file. |
| examples/training/llm-foundry/aks/helm/llm-training/values.yaml | Adds Helm values for LLM Foundry training PyTorchJob. |
| examples/training/llm-foundry/aks/helm/llm-training/templates/pytorchjob.yaml | Adds distributed training PyTorchJob template using composer. |
| examples/training/llm-foundry/aks/helm/llm-training/templates/_helpers.tpl | Adds Helm helpers and env var rendering. |
| examples/training/llm-foundry/aks/helm/llm-training/templates/NOTES.txt | Updates docs link to new training example path. |
| examples/training/llm-foundry/aks/helm/llm-training/Chart.yaml | Adds chart metadata for LLM Foundry training chart. |
| examples/training/llm-foundry/aks/helm/dataset-download/values.yaml | Adds Helm values for dataset download/convert job. |
| examples/training/llm-foundry/aks/helm/dataset-download/templates/job.yaml | Adds job template for dataset download/convert. |
| examples/training/llm-foundry/aks/helm/dataset-download/templates/_helpers.tpl | Adds Helm helpers for dataset-download chart. |
| examples/training/llm-foundry/aks/helm/dataset-download/templates/NOTES.txt | Adds post-install notes for dataset download chart. |
| examples/training/llm-foundry/aks/helm/dataset-download/Chart.yaml | Adds chart metadata for dataset-download chart. |
| examples/training/llm-foundry/aks/README.md | Updates helm install paths for reorganized example location. |
| examples/training/dgx_benchmarking/slurm/download.sh | Adds script to pull DGX benchmarking resources from NGC. |
| examples/training/dgx_benchmarking/slurm/README.md | Adds guidance for DGX benchmark tuning/affinity on Azure. |
| examples/inferencing/nvidia-dynamo/aks/helm/dynamo-deployment/values.yaml | Adds values for DynamoGraphDeploymentRequest creation. |
| examples/inferencing/nvidia-dynamo/aks/helm/dynamo-deployment/templates/deployment.yaml | Adds template for a DynamoGraphDeploymentRequest resource. |
| examples/inferencing/nvidia-dynamo/aks/helm/dynamo-deployment/Chart.yaml | Adds chart metadata for Dynamo inference deployment chart. |
| examples/inferencing/nvidia-dynamo/aks/README.md | Adds end-to-end AKS deployment guide for Dynamo inference. |
| README.md | Updates top-level catalog to split Training vs Inferencing examples and adjusts paths. |
| .github/workflows/build_megatron_image.yaml | Updates docker build paths to new training example directory structure. |
| .github/workflows/build_llm_foundry_image.yaml | Updates docker build paths to new training example directory structure. |
Contributor
|
@wolfgang-desalvador I've opened a new pull request, #126, to work on those changes. Once the pull request is ready, I'll request review from you. |
2 tasks
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: wolfgang-desalvador <118554802+wolfgang-desalvador@users.noreply.github.com> Agent-Logs-Url: https://github.com/Azure/ai-infrastructure-on-azure/sessions/49c8a1e2-9f5d-4f8b-a5c5-3b074b48b7ee
edwardsp
approved these changes
Mar 23, 2026
Sync DYNAMO_VERSION help text default with actual default value
wolfgang-desalvador
temporarily deployed
to
testing
March 23, 2026 17:18 — with 
GitHub Actions
Inactive
GitHub Actions
Inactive
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Contributor
|
@wolfgang-desalvador I've opened a new pull request, #127, to work on those changes. Once the pull request is ready, I'll request review from you. |
Co-authored-by: wolfgang-desalvador <118554802+wolfgang-desalvador@users.noreply.github.com> Agent-Logs-Url: https://github.com/Azure/ai-infrastructure-on-azure/sessions/1e1e7e15-f086-482c-b959-74d3518641d3
Pin Dynamo precheck script download to release tag instead of main
wolfgang-desalvador
temporarily deployed
to
testing
March 25, 2026 10:48 — with
GitHub Actions
Inactive
GitHub Actions
Inactive
edwardsp
approved these changes
Mar 25, 2026
Collaborator
Author
|
@copilot resolve the merge conflicts in this pull request |
# Conflicts: # infrastructure_references/aks/README.md Co-authored-by: wolfgang-desalvador <118554802+wolfgang-desalvador@users.noreply.github.com>
Contributor
Resolved the merge conflicts and pushed the merge commit in |
edwardsp
approved these changes
Jun 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces a new example for LLM inference using NVIDIA Dynamo on AKS, reorganizes the example directory structure for both training and inferencing workflows, and updates related documentation and CI workflows. The changes improve clarity between training and inference examples, add a ready-to-use Helm chart for NVIDIA Dynamo, and ensure all references and automation scripts point to the new structure.
Directory and Documentation Organization:
examples/training/, movingllm-foundryandmegatron-lmsubdirectories accordingly; updated all documentation and workflow references to match the new structure. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]New Inferencing Example:
examples/inferencing/nvidia-dynamo/aks/, including a comprehensiveREADME.mdwith setup, deployment, and testing instructions.Helm Chart for Inference Deployment:
dynamo-deployment) for deploying NVIDIA Dynamo LLM inference on AKS, with customizable values and deployment templates. [1] [2] [3]These changes make it easier to distinguish between training and inference workflows, streamline onboarding for new users, and provide a robust starting point for deploying LLM inference services on Azure Kubernetes Service.