From c4bd56143ddd6cfe5d29497043ac4b3dbfaa0145 Mon Sep 17 00:00:00 2001 From: Kurt Heiss Date: Tue, 7 Jul 2026 16:04:54 -0700 Subject: [PATCH] docs(26.05): split Julio Helm and VDB reconcile updates Port Julio-owned Helm README and VDB reconcile changes from PR 2311 onto 26.05. Includes Greptile fixes for OCR v1, reranker image tag, and the answer_llm 26.05 chart caveat. --- docs/docs/extraction/vdbs.md | 26 ++- nemo_retriever/helm/README.md | 370 +++++++++++++++++++--------------- 2 files changed, 220 insertions(+), 176 deletions(-) diff --git a/docs/docs/extraction/vdbs.md b/docs/docs/extraction/vdbs.md index 67f3b64a3f..d1b6e4de1e 100644 --- a/docs/docs/extraction/vdbs.md +++ b/docs/docs/extraction/vdbs.md @@ -60,7 +60,7 @@ Pass `vdb_op="lancedb"` to `vdb_upload`, or construct a `LanceDB` instance and p For parameter details, refer to the [Python API guide](nemo-retriever-api-reference.md). ```python -from nemo_retriever.vdb.lancedb import LanceDB +from nemo_retriever.common.vdb.lancedb import LanceDB vdb = LanceDB( uri="./lancedb_data", # Path to LanceDB database directory @@ -83,7 +83,7 @@ When using the `Ingestor` with `vdb_upload`, pass `vdb_op="lancedb"` or a `Lance Semantic retrieval uses dense embeddings to find content that is similar in meaning to a query. In NeMo Retriever Library, the default vector path is LanceDB. Use these resources together with the sections on this page: -- [Metadata and filtering](#metadata-and-filtering) for sidecar metadata at ingest and query-time filters +- [Metadata and filtering](#metadata-and-filtering) for custom metadata at ingest and filtered retrieval - [Concepts](concepts.md) for broader pipeline and search patterns - [Use the NeMo Retriever Library Python API](nemo-retriever-api-reference.md) for `Retriever.query` and `LanceDB.retrieval` parameters @@ -91,10 +91,7 @@ Semantic retrieval uses dense embeddings to find content that is similar in mean ## Metadata and filtering { #metadata-and-filtering } -This page covers LanceDB upload and retrieval. **Metadata is not duplicated here.** - -- **Published guide** — [Custom metadata and filtering](custom-metadata.md) (sidecar `meta_*` on `vdb_upload`, compact JSON in LanceDB, server-side `where` on `Retriever.query`, and client-side `filter_hits_by_content_metadata`). -- **Canonical reference** — [Vector DB operators and LanceDB — Metadata filtering](https://github.com/NVIDIA/NeMo-Retriever/tree/main/nemo_retriever/src/nemo_retriever/vdb#metadata-filtering) in `nemo_retriever/src/nemo_retriever/vdb/README.md` (operator behavior and examples). +Refer to the [metadata filtering notebook](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/nemo_retriever_retriever_query_metadata_filter.ipynb) for an end-to-end example of adding custom metadata fields to your documents and filtering retrieval results with that metadata. ## LanceDB deployment characteristics { #lancedb-deployment-characteristics } @@ -120,19 +117,19 @@ NeMo Retriever Library integrates with vector databases used for RAG collections ### Backends with `VDB` implementations (retriever adapters) { #vdb-backends-implementations } -NeMo Retriever graph operators [`IngestVdbOperator`](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/src/nemo_retriever/vdb/operators.py) and [`RetrieveVdbOperator`](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/src/nemo_retriever/vdb/operators.py) wrap concrete classes that implement the [`VDB`](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/src/nemo_retriever/vdb/adt_vdb.py) interface (`run` for ingest, `retrieval` for search). The library ships one first-party backend: +NeMo Retriever graph operators [`IngestVdbOperator`](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/src/nemo_retriever/operators/vdb.py) and [`RetrieveVdbOperator`](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/src/nemo_retriever/operators/vdb.py) wrap concrete classes that implement the [`VDB`](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/src/nemo_retriever/common/vdb/adt_vdb.py) interface (`run` for ingest, `retrieval` for search). The library ships one first-party backend: | Backend | Project | Implementation | |---------|---------|----------------| -| **LanceDB** | [LanceDB](https://lancedb.com/) · [documentation](https://lancedb.github.io/lancedb/) | [`lancedb.py`](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/src/nemo_retriever/vdb/lancedb.py) — pass `vdb_op="lancedb"` (recommended). | +| **LanceDB** | [LanceDB](https://lancedb.com/) · [documentation](https://lancedb.github.io/lancedb/) | [`lancedb.py`](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/src/nemo_retriever/common/vdb/lancedb.py) — pass `vdb_op="lancedb"` (recommended). | On `GraphIngestor.vdb_upload`, omitting `vdb_op` does not select LanceDB; refer to [Upload to LanceDB](#upload-to-lancedb). -Pass `vdb_op="lancedb"` or a `LanceDB` instance. To integrate another vector database, subclass [`VDB`](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/src/nemo_retriever/vdb/adt_vdb.py) and pass your operator instance as `vdb` (refer to [Build a Custom Vector Database Operator](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/building_vdb_operator.ipynb)). +Pass `vdb_op="lancedb"` or a `LanceDB` instance. To integrate another vector database, subclass [`VDB`](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/src/nemo_retriever/common/vdb/adt_vdb.py) and pass your operator instance as `vdb` (refer to [Build a Custom Vector Database Operator](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/building_vdb_operator.ipynb)). ### RAG Blueprint and partner vector stores { #rag-blueprint-and-partner-vector-stores } -Some deployments use a different vector store than the default LanceDB path on this page—for example the [NVIDIA RAG Blueprint](https://docs.nvidia.com/rag/latest/index.html) (Docker Compose or Helm) or a partner package that subclasses the same [`VDB`](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/src/nemo_retriever/vdb/adt_vdb.py) interface. Use the following public references when you wire those stacks to ingestion and retrieval: +Some deployments use a different vector store than the default LanceDB path on this page—for example the [NVIDIA RAG Blueprint](https://docs.nvidia.com/rag/latest/index.html) (Docker Compose or Helm) or a partner package that subclasses the same [`VDB`](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/src/nemo_retriever/common/vdb/adt_vdb.py) interface. Use the following public references when you wire those stacks to ingestion and retrieval: | Vector store | Where to configure or implement | |--------------|--------------------------------| @@ -144,7 +141,7 @@ Testing and release cadence for these integrations follow the owning project (RA ### More information (embeddings & custom `VDB`) { #vector-database-partners-more-info } -- [Custom metadata and filtering](custom-metadata.md) and the package [VDB README (metadata filtering)](https://github.com/NVIDIA/NeMo-Retriever/tree/main/nemo_retriever/src/nemo_retriever/vdb#metadata-filtering) +- [Metadata filtering notebook](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/nemo_retriever_retriever_query_metadata_filter.ipynb) and the package [VDB README (metadata filtering)](https://github.com/NVIDIA/NeMo-Retriever/tree/main/nemo_retriever/src/nemo_retriever/common/vdb#metadata-filtering) - [Multimodal embeddings (VLM)](embedding.md) - [NeMo Retriever Text Embedding NIM](https://docs.nvidia.com/nim/nemo-retriever/text-embedding/latest/overview.html) - [NVIDIA NIM catalog](https://build.nvidia.com/) for embedding and retrieval-related NIMs @@ -153,12 +150,13 @@ Testing and release cadence for these integrations follow the owning project (RA NVIDIA documents and validates the first-party LanceDB operator for this library. If you integrate a different vector store, you are responsible for testing and maintaining that integration. -To implement a custom operator, follow the `VDB` abstract interface described in [Build a Custom Vector Database Operator](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/building_vdb_operator.ipynb). +To implement a custom operator, follow the `VDB` abstract interface described in [Build a Custom Vector Database Operator](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/building_vdb_operator.ipynb). For an overview of all customization paths (UDFs, graph pipelines, and embeddings), refer to [Customize & extend](customize-extend.md). ## Related Topics { #related-topics } -- [Custom metadata and filtering](custom-metadata.md) -- [Vector DB operators and LanceDB (source)](https://github.com/NVIDIA/NeMo-Retriever/tree/main/nemo_retriever/src/nemo_retriever/vdb) +- [Metadata and filtering](#metadata-and-filtering) +- [Customize & extend](customize-extend.md) +- [Vector DB operators and LanceDB (source)](https://github.com/NVIDIA/NeMo-Retriever/tree/main/nemo_retriever/src/nemo_retriever/common/vdb) - [Use the NeMo Retriever Library Python API](nemo-retriever-api-reference.md) - [Store Extracted Images](nemo-retriever-api-reference.md) - [Environment Variables](environment-config.md) diff --git a/nemo_retriever/helm/README.md b/nemo_retriever/helm/README.md index 121a41fdd5..d1fadf7543 100644 --- a/nemo_retriever/helm/README.md +++ b/nemo_retriever/helm/README.md @@ -50,6 +50,7 @@ nemo_retriever/helm/ ├── Chart.yaml ├── values.yaml ├── README.md <-- this file +├── openshift.md <-- OpenShift restricted-v2 install guide ├── .helmignore └── templates/ ├── _helpers.tpl @@ -78,15 +79,15 @@ nemo_retriever/helm/ ## Quick start -### 1. Service image +### 1. Service image { #1-service-image } -The chart defaults to the staging image published to NGC: +The chart defaults to the GA image published to NGC: ``` -nvcr.io/nvstaging/nim/nemo-retriever-service:043020205-001 +nvcr.io/nvidia/nemo-microservices/nrl-service:26.5.0 ``` -Pulling from `nvcr.io/nvstaging` requires an NGC pull secret — either set +Pulling from `nvcr.io` requires an NGC pull secret — either set `ngcImagePullSecret.create=true` (see below) or pre-create one in the namespace named `ngc-secret`. @@ -188,7 +189,7 @@ NIM (the VL reranker `rerankqa`, Nemotron Parse, Omni 30B, and the Parakeet `audio` ASR NIM) is **disabled by default** to honor the "optional and disabled by default" contract in [deployment-options.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/deployment-options.md); -see [Recommended minimal install (26.05)](#recommended-minimal-install-2605) +refer to [Recommended minimal install](#recommended-minimal-install-2605) for the opt-in `--set` flags that turn any of them on. ```bash @@ -211,14 +212,15 @@ helm install retriever ./nemo_retriever/helm \ --set ngcApiSecret.password=$NGC_API_KEY ``` -> The VL reranker (`rerankqa`), Nemotron Parse, the Nemotron 3 Nano Omni 30B caption NIM, and the Parakeet `audio` ASR NIM are **all off by default** in 26.05 — they only reconcile when you explicitly opt in. Opt-in flags: +> The VL reranker (`rerankqa`), Nemotron Parse, the Nemotron 3 Nano Omni 30B caption NIM, the generic answer-generation LLM (`answer_llm`, Super-49B defaults), and the Parakeet `audio` ASR NIM are **all off by default** — they only reconcile when you explicitly opt in. Opt-in flags: > > * VL reranker — `--set nimOperator.rerankqa.enabled=true` > * Nemotron Parse — `--set nimOperator.nemotron_parse.enabled=true` > * Omni 30B captioner — `--set nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning.enabled=true` +> * Answer generation LLM — `--set nimOperator.answer_llm.enabled=true` (**not in the 26.05 chart**; refer to [Answer generation (operator-managed LLM)](#answer-generation-llm)) > * Parakeet ASR — `--set nimOperator.audio.enabled=true` (also set `serviceConfig.nimEndpoints.audioGrpcEndpoint=audio:50051` to wire ASR into the service, plus `service.installFfmpeg=true` if your image does not bundle ffmpeg) > -> This matches the "optional and disabled by default" contract in [deployment-options.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/deployment-options.md) and avoids silently pulling ≈ 62 GiB of Omni weights or claiming a second dedicated GPU on a "default" install. See the [model hardware requirements](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#model-hardware-requirements) table for per-NIM GPU and disk costs. +> This matches the "optional and disabled by default" contract in [deployment-options.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/deployment-options.md) and avoids silently pulling ≈ 62 GiB of Omni weights, loading a large two-GPU LLM, or claiming extra dedicated GPUs on a "default" install. Refer to the [model hardware requirements](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#model-hardware-requirements) table for per-NIM GPU and disk costs. The chart auto-wires the operator-managed in-cluster URLs of the four "core" NIMs into the service's `nim_endpoints` block: @@ -286,8 +288,8 @@ short list of knobs you'll touch first. | Path | Default | Notes | |-------------------------------|------------------------------------|-------| -| `service.image.repository` | `localhost:32000/nemo-retriever-service` | Override to a published image. | -| `service.image.tag` | `latest` | | +| `service.image.repository` | `nvcr.io/nvidia/nemo-microservices/nrl-service` | GA NGC image; override to pin a different build or use a local registry. | +| `service.image.tag` | `26.5.0` | | | `service.replicas` | `1` | Hard cap = 1 while SQLite is the backend. | | `service.installFfmpeg` | `false` | Install `ffmpeg`/`ffprobe` at container startup by setting `INSTALL_FFMPEG=true`. Requires network egress, writable root filesystem, and sudo/setuid allowed. Not for air-gapped clusters — use a custom image instead. | | `service.resources.requests` | `16 / 16Gi` | Tune in tandem with `serviceConfig.pipeline.*Workers`. | @@ -295,19 +297,20 @@ short list of knobs you'll touch first. | `service.gpu.enabled` | `false` | The service does **not** need a GPU. | For audio and video extraction, set `service.installFfmpeg=true` when your -cluster allows runtime package installation. For air-gapped clusters, see -[Deployment options — Air-gapped and disconnected deployment](https://docs.nvidia.com/nemo/retriever/latest/extraction/deployment-options/#air-gapped-deployment). +cluster allows runtime package installation. **OpenShift restricted-v2** blocks +that path — use a prebuilt service image instead; refer to [Audio and video on restricted OpenShift](./openshift.md#audio-and-video-ffmpeg-on-restricted-openshift). +For air-gapped clusters, refer to [Deployment options — Air-gapped and disconnected deployment](https://docs.nvidia.com/nemo/retriever/latest/extraction/deployment-options/#air-gapped-deployment). ### Audio and video (Parakeet ASR) { #audio-video-parakeet } To run self-hosted Parakeet for [audio and video extraction](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/audio-video.md): -1. Set `nimOperator.audio.enabled=true` (it is on by default; disable other optional NIMs you do not need per [Recommended minimal install (26.05)](#recommended-minimal-install-2605)). -2. Pin the ASR `NIMService` to a **dedicated GPU** with `nimOperator.audio.resources`, `nodeSelector`, or `tolerations` (see [NIM Operator](https://docs.nvidia.com/nim-operator/latest/index.html)). +1. Set `nimOperator.audio.enabled=true` (it is on by default; disable other optional NIMs you do not need per [Recommended minimal install](#recommended-minimal-install-2605)). +2. Pin the ASR `NIMService` to a **dedicated GPU** with `nimOperator.audio.resources`, `nodeSelector`, or `tolerations` (refer to [NIM Operator](https://docs.nvidia.com/nim-operator/latest/index.html)). 3. Confirm the GPU SKU in [Model hardware requirements](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#model-hardware-requirements) (footnote ⁴ lists Blackwell limitations). -4. Set `service.installFfmpeg=true` when the retriever service will process audio or video (see `service.installFfmpeg` above). +4. Set `service.installFfmpeg=true` when the retriever service will process audio or video on clusters that allow runtime package install (refer to `service.installFfmpeg` above). On **OpenShift restricted-v2**, use a [prebuilt service image](./openshift.md#audio-and-video-ffmpeg-on-restricted-openshift) instead. -The retriever service picks up the in-cluster ASR endpoint when `nimOperator.audio` is enabled; see [NIM Operator sub-stack](#nim-operator-sub-stack). +The retriever service picks up the in-cluster ASR endpoint when `nimOperator.audio` is enabled; refer to [NIM Operator sub-stack](#nim-operator-sub-stack). ### Service configuration (rendered into `retriever-service.yaml`) @@ -315,12 +318,20 @@ The retriever service picks up the in-cluster ASR endpoint when `nimOperator.aud |---------------------------------------------------|---------|-------| | `serviceConfig.server.port` | `7670` | Container + Service port. | | `serviceConfig.pipeline.realtimeWorkers` | `24` | Per-pod realtime worker count. | -| `serviceConfig.pipeline.batchWorkers` | `48` | Per-pod batch worker count. See [Timeouts and alleviating ingest failures](#timeouts-and-alleviating-ingest-failures) if embed or pool errors appear under load. | -| `serviceConfig.nimEndpoints.*InvokeUrl` | `""` | Override the auto-resolved NIM Operator URL. Available knobs: `pageElementsInvokeUrl`, `tableStructureInvokeUrl`, `ocrInvokeUrl`, `embedInvokeUrl`, and `captionInvokeUrl` (see [Image captioning (Omni 30B)](#image-captioning-omni-30b)). | +| `serviceConfig.pipeline.batchWorkers` | `48` | Per-pod batch worker count. Refer to [Timeouts and alleviating ingest failures](#timeouts-and-alleviating-ingest-failures) if embed or pool errors appear under load. | +| `serviceConfig.nimEndpoints.*InvokeUrl` | `""` | Override the auto-resolved NIM Operator URL. Available knobs: `pageElementsInvokeUrl`, `tableStructureInvokeUrl`, `ocrInvokeUrl`, `embedInvokeUrl`, and `captionInvokeUrl` (refer to [Image captioning (Omni 30B)](#image-captioning-omni-30b)). | | `serviceConfig.nimEndpoints.captionModelName` | `""` | Model id sent to the remote VLM. Auto-set to `nvidia/nemotron-3-nano-omni-30b-a3b-reasoning` whenever a caption URL is resolved. | -| `serviceConfig.vectordb.enabled` | `true` | Deploy the LanceDB vectordb Pod. When `true` the chart **requires** a resolvable embed endpoint (see [VectorDB and the embed endpoint](#vectordb-and-the-embed-endpoint)); `helm install` / `helm upgrade` fails fast otherwise. | +| `serviceConfig.llm.enabled` | `false` | Enables `POST /v1/answer`. Auto-flips to true when `nimOperator.answer_llm` is enabled and the operator URL resolves. | +| `serviceConfig.llm.apiBase` | `""` | OpenAI-compatible LLM base URL. Explicit value wins; otherwise `answer_llm` opt-in resolves to `http://answer-llm:8000/v1` by default. | +| `serviceConfig.llm.apiKeySecret.name` | `""` | Optional Secret name for external LLM credentials. Explicit values win; otherwise operator-managed `answer_llm` mounts its `authSecret` as `NEMO_RETRIEVER_LLM_API_KEY` so LiteLLM/OpenAI has a credential value without writing it to the ConfigMap. | +| `serviceConfig.llm.apiKeySecret.key` | `api_key` | Secret key for external LLM credentials. Operator-managed `answer_llm` uses `NGC_API_KEY` from `nimOperator.answer_llm.authSecret` when no explicit LLM Secret is set. | +| `serviceConfig.llm.model` | `""` | Optional explicit LiteLLM model id. Leave empty to inherit `nimOperator.answer_llm.model` when using the operator-managed answer LLM; set it for external endpoints. | +| `serviceConfig.llm.ragSystemPromptPrefix` | `""` | Optional explicit RAG prompt prefix. Leave empty unless an endpoint needs model-specific prompt directives. | +| `serviceConfig.llm.reasoningEnabled` | `true` | Request-level reasoning toggle for `/v1/answer`. Defaults to true for external OpenAI-compatible providers; set false for Nemotron endpoints that should receive portable no-reasoning controls. | +| `serviceConfig.vectordb.enabled` | `true` | Deploy the LanceDB vectordb Pod. When `true` the chart **requires** a resolvable embed endpoint (refer to [VectorDB and the embed endpoint](#vectordb-and-the-embed-endpoint)); `helm install` / `helm upgrade` fails fast otherwise. | | `serviceConfig.vectordb.lancedbUri` | `/data/vectordb` | LanceDB on the vectordb Pod's PVC. | | `serviceConfig.vectordb.embedModel` | `nvidia/llama-nemotron-embed-vl-1b-v2` | Passed to vectordb + worker `embed_model_name`. | +| `serviceConfig.vectordb.embedModelProviderPrefix` | `""` | Optional LiteLLM provider prefix prepended to the remote embed model name. | #### VectorDB and the embed endpoint { #vectordb-and-the-embed-endpoint } @@ -345,7 +356,7 @@ resolved. Pick one of: 3. --set serviceConfig.vectordb.enabled=false ``` -Resolution order matches the rest of the chart (see [Mix and match NIM +Resolution order matches the rest of the chart (refer to [Mix and match NIM sources](#3-install-with-the-nim-operator-in-cluster-nims)): 1. Explicit `serviceConfig.nimEndpoints.embedInvokeUrl` always wins. @@ -355,6 +366,93 @@ sources](#3-install-with-the-nim-operator-in-cluster-nims)): `apps.nvidia.com/v1alpha1` CRDs are installed in the cluster. 3. Otherwise the chart fails the install. +#### Answer generation (operator-managed LLM) { #answer-generation-llm } + +> **Note:** Operator-managed answer generation is not included in the 26.05 chart. This branch ships no `answer_llm` values block or NIM template and no `serviceConfig.llm` keys, so `--set nimOperator.answer_llm.enabled=true` has no effect on this chart. The configuration described in this section applies to a later chart release; do not rely on it for a 26.05 deployment. + +Enable the generic `answer_llm` NIM slot to add service-mode answer +generation on top of the VectorDB query path. The slot defaults to the +Super-49B NIM, but the image, model id, service name, resources, +profile filter, and environment can be overridden for another +OpenAI-compatible LLM NIM. + +```bash +helm upgrade --install retriever ./nemo_retriever/helm \ + --set nimOperator.answer_llm.enabled=true +``` + +When the NIM Operator CRDs are present, the chart renders an `answer-llm` +NIMCache/NIMService by default and writes this block into +`retriever-service.yaml`: + +```yaml +llm: + enabled: true + model: "openai/nvidia/llama-3.3-nemotron-super-49b-v1.5" + api_base: "http://answer-llm:8000/v1" + rag_system_prompt_prefix: null + reasoning_enabled: true +``` + +The retriever service then exposes `POST /v1/answer`, which calls the +VectorDB pod's `/v1/query` endpoint for context and sends those chunks to +the configured LLM endpoint. The `answer_llm` NIM deployment leaves +reasoning defaults model-neutral; `/v1/answer` controls reasoning per +request. By default, `serviceConfig.llm.reasoningEnabled=true`, so requests +leave reasoning behavior to the LLM endpoint defaults and avoid sending +provider-specific `chat_template_kwargs` to external OpenAI-compatible +endpoints. Set `serviceConfig.llm.reasoningEnabled=false` for Nemotron +endpoints that should skip reasoning; the service then adds both `/no_think` +and `chat_template_kwargs.enable_thinking=false`. The default Super-49B NIMService +resources request two GPUs (`nvidia.com/gpu: 2`) to match the bundled +tensor-parallel NIM profile. Override `resources`, `modelProfile`, or +`env` for deployments that use a different profile or hardware topology. +When `answer_llm` is enabled and no explicit `serviceConfig.llm.apiKeySecret` +is set, the service also mounts `nimOperator.answer_llm.authSecret` as +`NEMO_RETRIEVER_LLM_API_KEY`; OpenAI-compatible clients require a +credential value even for in-cluster NIM endpoints, and the key is never +rendered into the ConfigMap. + +For example, to try Nemotron 3 Nano as the answer LLM on an A100 80GB +node, override the operator-managed slot instead of adding a second +hard-coded LLM service: + +```bash +helm upgrade --install retriever ./nemo_retriever/helm \ + --set nimOperator.answer_llm.enabled=true \ + --set nimOperator.answer_llm.nimServiceName=nemotron-3-nano \ + --set nimOperator.answer_llm.image.repository=nvcr.io/nim/nvidia/nemotron-3-nano \ + --set nimOperator.answer_llm.image.tag=1.7.0-variant \ + --set nimOperator.answer_llm.model=openai/nvidia/nemotron-3-nano-30b-a3b \ + --set-json nimOperator.answer_llm.modelProfile='{"profiles":["5f89f01a0af587fd8bae50c611b1f358f92effdb9fb29362e1af0a986e5561c3"]}' \ + --set-json nimOperator.answer_llm.resources='{"limits":{"nvidia.com/gpu":1},"requests":{"nvidia.com/gpu":1}}' \ + --set nimOperator.answer_llm.env[0].name=NIM_HTTP_API_PORT \ + --set-string nimOperator.answer_llm.env[0].value=8000 \ + --set nimOperator.answer_llm.env[1].name=NIM_SERVED_MODEL_NAME \ + --set-string nimOperator.answer_llm.env[1].value=nvidia/nemotron-3-nano-30b-a3b \ + --set nimOperator.answer_llm.env[2].name=NIM_TENSOR_PARALLEL_SIZE \ + --set-string nimOperator.answer_llm.env[2].value=1 +``` + +Use the repository and tag available in your NGC environment; staging +registries can use the same override shape with `nvstaging` image names +or tags. `nimOperator.answer_llm.model` is the LiteLLM model id used by +the retriever service; for an OpenAI-compatible in-cluster NIM, keep the +`openai/` prefix there and set `NIM_SERVED_MODEL_NAME` to the raw model +name advertised by the NIM. Replace the default Super-49B `modelProfile`, +`resources`, and `env` when the target model requires a different +GPU/profile setup. Leaving `modelProfile` empty preserves NIM +Operator auto-discovery, but for Nano it can cache every advertised +profile on first reconciliation; pin a known-compatible profile when you +know the target GPU topology. + +`serviceConfig.llm.apiBase` and `serviceConfig.llm.model` can be set +explicitly to point `/v1/answer` at an external OpenAI-compatible LLM +instead of deploying an answer LLM in-cluster. For external credentials, +create a Kubernetes Secret and set `serviceConfig.llm.apiKeySecret.name` +plus `serviceConfig.llm.apiKeySecret.key`; Helm mounts the Secret as an +environment variable instead of writing the key into the ConfigMap. + ### NIM Operator sub-stack Each NIM block under `nimOperator.` renders a `NIMCache` + `NIMService` @@ -374,19 +472,22 @@ pair gated on three conditions ALL holding: | `nimOperator.vlm_embed.enabled` | `true` | Multimodal embedding NIM (also used by the vectordb Pod). | | `nimOperator.vlm_embed.nimServiceName` | `llama-nemotron-embed-vl-1b-v2` | NIMService / in-cluster DNS name. | | `nimOperator.vlm_embed.image` | `nvcr.io/nim/nvidia/llama-nemotron-embed-vl-1b-v2:1.12.0` | Default VLM embed NIM image. | -| `nimOperator.rerankqa.enabled` | `false` | VL reranker NIM (optional; not auto-wired). Set `true` to opt in. Default `false` so 26.05 installs honor the "optional and disabled by default" contract in [deployment-options.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/deployment-options.md) and do not silently provision an extra ≈ 3.1 GiB GPU NIM. The image points at the **VL** SKU (`llama-nemotron-rerank-vl-1b-v2`) per [prerequisites-support-matrix.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#default-helm-nims) — the text-only `llama-nemotron-rerank-1b-v2` silently degrades multimodal reranking and is not the documented POR. | -| `nimOperator.nemotron_parse.enabled` | `false` | Structured-parse NIM (optional). Set `true` when using `extract_method="nemotron_parse"`. Default `false` so 26.05 installs honor the "optional and disabled by default" contract in [deployment-options.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/deployment-options.md). Image tag follows the [image tag conventions](#image-tag-conventions). | -| `nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning.enabled` | `false` | Omni 30B caption NIM (optional). Set `true` to enable image captioning — see [Image captioning (Omni 30B)](#image-captioning-omni-30b). Default `false` so 26.05 installs do not silently pull ≈ 62 GiB of BF16 weights or claim a second dedicated GPU. Image tag follows the [image tag conventions](#image-tag-conventions). | +| `nimOperator.rerankqa.enabled` | `false` | VL reranker NIM (optional; not auto-wired). Set `true` to opt in. Default `false` so chart installs honor the "optional and disabled by default" contract in [deployment-options.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/deployment-options.md) and do not silently provision an extra ≈ 3.1 GiB GPU NIM. The image points at the **VL** SKU (`llama-nemotron-rerank-vl-1b-v2`) per [prerequisites-support-matrix.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#default-helm-nims) — the text-only `llama-nemotron-rerank-1b-v2` silently degrades multimodal reranking and is not the documented POR. | +| `nimOperator.nemotron_parse.enabled` | `false` | Structured-parse NIM (optional). Set `true` when using `extract_method="nemotron_parse"`. Default `false` so chart installs honor the "optional and disabled by default" contract in [deployment-options.md](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/deployment-options.md). Image tag follows the [image tag conventions](#image-tag-conventions). | +| `nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning.enabled` | `false` | Omni 30B caption NIM (optional). Set `true` to enable image captioning — refer to [Image captioning (Omni 30B)](#image-captioning-omni-30b). Default `false` so chart installs do not silently pull ≈ 62 GiB of BF16 weights or claim a second dedicated GPU. Image tag follows the [image tag conventions](#image-tag-conventions). | +| `nimOperator.answer_llm.enabled` | `false` | Generic answer-generation LLM NIM (optional; Super-49B defaults). Set `true` to enable `/v1/answer` — refer to [Answer generation (operator-managed LLM)](#answer-generation-llm). Default `false` so installs do not silently claim answer-generation GPUs. | +| `nimOperator.answer_llm.model` | `openai/nvidia/llama-3.3-nemotron-super-49b-v1.5` | LiteLLM/OpenAI model id inherited by `serviceConfig.llm.model` when the operator-managed answer LLM is enabled and no explicit service model is set. | +| `nimOperator.answer_llm.ragSystemPromptPrefix` | `""` | Optional prompt prefix inherited by `serviceConfig.llm.ragSystemPromptPrefix` only when explicitly set. Leave empty to keep the operator-managed LLM model-neutral and use `serviceConfig.llm.reasoningEnabled` for request-level reasoning control. | | `nimOperator.audio.enabled` | `false` | Parakeet ASR NIM (optional). Set `true` for audio/video transcription; pair with `serviceConfig.nimEndpoints.audioGrpcEndpoint=audio:50051` so the retriever-service can reach it. | | `nimOperator..image.repository` | `nvcr.io/nim/nvidia/...` | Per-NIM image. | | `nimOperator..image.pullSecrets` | `[ngc-secret]` | Referenced by the NIMService CR. | | `nimOperator..authSecret` | `ngc-api` | NIM auth Secret name. | | `nimOperator..storage.pvc.size` | `25Gi` (50Gi for vlm_embed/rerankqa, 100Gi parse, 300Gi VL) | NIMCache PVC size. | | `nimOperator..replicas` | `1` | Per-NIMService replica count. | -| `nimOperator.nimServiceGpuLimit` | `1` | Default `nvidia.com/gpu` limit on every NIMService when per-NIM `resources` is `{}`. Set to `null` for operator-only reconciliation (not reliable on all NIM Operator versions — see [GPU limits and `helm upgrade`](#gpu-limits-and-helm-upgrade)). | +| `nimOperator.nimServiceGpuLimit` | `1` | Default `nvidia.com/gpu` limit on every NIMService when per-NIM `resources` is `{}`. Set to `null` for operator-only reconciliation (not reliable on all NIM Operator versions — refer to [GPU limits and `helm upgrade`](#gpu-limits-and-helm-upgrade)). | | `nimOperator..resources` | `{}` | Per-NIM override of the whole `resources` block. Empty uses `nimServiceGpuLimit`; non-empty replaces the chart default (may require `--force-conflicts` on later `helm upgrade`). | -| `nimOperator.modelProfile` | `{}` | Chart-wide NIMCache GPU/profile filter. Applied to every NIMCache that does not have its own override. See [Filtering cached GPU profiles](#filtering-cached-gpu-profiles). | -| `nimOperator..modelProfile` | `{}` | Per-NIM NIMCache GPU/profile filter. Non-empty values REPLACE the chart-wide default (no merge). See [Filtering cached GPU profiles](#filtering-cached-gpu-profiles). | +| `nimOperator.modelProfile` | `{}` | Chart-wide NIMCache GPU/profile filter. Applied to every NIMCache that does not have its own override. Refer to [Filtering cached GPU profiles](#filtering-cached-gpu-profiles). | +| `nimOperator..modelProfile` | `{}` | Per-NIM NIMCache GPU/profile filter. Non-empty values REPLACE the chart-wide default (no merge). Refer to [Filtering cached GPU profiles](#filtering-cached-gpu-profiles). | | `nimOperator..expose.service.port` | `8000` (9000 for audio) | HTTP port. | | `nimOperator..expose.service.grpcPort` | `8001` (50051 for audio) | gRPC port. | @@ -394,7 +495,7 @@ pair gated on three conditions ALL holding: > are auto-wired into the retriever-service config. Optional NIMs may reconcile > when `nimOperator..enabled` is `true` in `values.yaml`, but the > retriever-service won't call them unless you wire your pipeline to use them. -> For 26.05, prefer the [minimal install](#recommended-minimal-install-2605) overrides. +> For minimal installs, prefer the [minimal install](#recommended-minimal-install-2605) overrides. #### Filtering cached GPU profiles { #filtering-cached-gpu-profiles } @@ -465,7 +566,7 @@ Every NIM in this chart pins an exact NGC image tag in `values.yaml` | Family | Example | Meaning | | ------ | ------- | ------- | | Plain semver | `nemotron-page-elements-v3:1.8.0` | A standard NIM release, identical bytes on every pull. Used by the four core NIMs and the reranker / ASR NIMs. | -| `-variant` | `nemotron-parse-v1.2:1.7.0-variant`, `nemotron-3-nano-omni-30b-a3b-reasoning:1.7.0-variant` | The Nemotron Parse and Nemotron 3 Nano Omni 30B builds that ship per-GPU TensorRT engine variants the NIM Operator selects from at reconciliation time (see the Omni and Parse rows in the [model hardware requirements](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#model-hardware-requirements) table). The `-variant` suffix is the NGC tag that ships alongside the 26.05 chart and matches footnote ³ of the support matrix. | +| `-variant` | `nemotron-parse-v1.2:1.7.0-variant`, `nemotron-3-nano-omni-30b-a3b-reasoning:1.7.0-variant` | The Nemotron Parse and Nemotron 3 Nano Omni 30B builds that ship per-GPU TensorRT engine variants the NIM Operator selects from at reconciliation time (refer to the Omni and Parse rows in the [model hardware requirements](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#model-hardware-requirements) table). The `-variant` suffix is the NGC tag that ships alongside this chart and matches footnote ³ of the support matrix. | For air-gapped mirror pipelines: mirror the *exact* tag — both the plain semver and the `-variant` form — and do not substitute `:latest`. @@ -483,18 +584,18 @@ helm upgrade --install retriever ./nemo_retriever/helm \ and validate against the same release of the retriever service before production rollout. -**Charts and captioning (26.05).** Charts and infographics use **page_elements** +**Charts and captioning.** Charts and infographics use **page_elements** and **ocr** (no `graphic_elements` operator NIM in this chart). For image -captioning, set `nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning.enabled=true` — see +captioning, set `nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning.enabled=true` — refer to [Image captioning (Omni 30B)](#image-captioning-omni-30b) for the chart-side wiring and -[Image captioning (26.05)](https://docs.nvidia.com/nemo/retriever/latest/extraction/prerequisites-support-matrix/#image-captioning-2605) +[Image captioning](https://docs.nvidia.com/nemo/retriever/latest/extraction/prerequisites-support-matrix/#image-captioning) for the product matrix. #### Image captioning (Omni 30B) { #image-captioning-omni-30b } The Nemotron 3 Nano Omni VLM is the canonical image-caption NIM for -26.05. When you enable it, +this chart. When you enable it, ```bash helm upgrade --install retriever ./nemo_retriever/helm \ @@ -531,7 +632,7 @@ Resolution order mirrors every other NIM endpoint (see the `serviceConfig.nimEndpoints.captionModelName` follows the same order — it defaults to the canonical Omni remote model id (`nvidia/nemotron-3-nano-omni-30b-a3b-reasoning`, matching -`nemo_retriever.caption.model_profiles.OMNI_REMOTE_MODEL_ID`) whenever +`nemo_retriever.common.modality.caption.model_profiles.OMNI_REMOTE_MODEL_ID`) whenever the chart resolves any caption URL. Override only when pointing at a different VLM SKU. @@ -583,7 +684,7 @@ block). Confirm `image.repository` and `image.tag` before you upgrade. |------|------| | `nimOperator.nimCache.keepOnUninstall` | When `true`, NIMCache CRs survive `helm uninstall` (`helm.sh/resource-policy: keep`). NIMService CRs are always removed. Set `false` for dev clusters that should fully tear down on uninstall. | | `nimOperator.ocr.enabled` | Reconcile the OCR `NIMService` | -| `nimOperator.ocr.image.repository` | NIM image (default `nvcr.io/nim/nvidia/nemotron-ocr-v1`) | +| `nimOperator.ocr.image.repository` | NIM image (default `nvcr.io/nim/nvidia/nemotron-ocr-v2`) | | `nimOperator.ocr.image.tag` | Pin the image tag for reproducible upgrades | Override the auto-wired in-cluster URL with `serviceConfig.nimEndpoints.ocrInvokeUrl` @@ -937,163 +1038,106 @@ sanity check before opening Grafana. --- -## OpenShift deployment { #openshift-deployment } - -The chart defaults target generic Kubernetes clusters that allow fixed numeric -UIDs (`runAsUser` / `runAsGroup` / `fsGroup` **1000**). **OpenShift 4.x** -namespaces under the default **restricted-v2** Security Context Constraint (SCC) -and **Pod Security Admission (PSA) `restricted`** profile assign a per-namespace -UID/GID range instead. A stock `helm install` without overrides therefore fails -SCC validation, emits PSA warnings, or crashes on log paths the random UID cannot -write. - -We do **not** change chart defaults for OpenShift-only behavior (that would affect -other platforms). Use the overrides below on OpenShift, or save the YAML block -into a local values file and pass `-f `. - -### Cluster posture (typical QA / hardened namespaces) - -| Control | Typical default on a new OpenShift project | -| --- | --- | -| SCC | **restricted-v2** (first match in priority order) | -| PSA | `pod-security.kubernetes.io/warn=restricted` (and often `audit=restricted`; `enforce` may be unset on dev clusters) | -| UID assignment | SCC injects `runAsUser` / `fsGroup` from the namespace range (for example `1000750000–1000759999`) | - -On clusters with **PSA `enforce=restricted`**, missing container `securityContext` -fields become hard rejections, not warnings. - -### Override reference (maps to chart limitations) +## Tracing and Zipkin -| Symptom on stock install | Cause | Helm override | -| --- | --- | --- | -| `FailedCreate`: UID/GID **1000** not in namespace range | Hardcoded `service.podSecurityContext` UID/GID/fsGroup | Omit `runAsUser`, `runAsGroup`, and `fsGroup`; keep only `runAsNonRoot: true` | -| PSA warning: `allowPrivilegeEscalation`, capabilities, `seccompProfile` | Empty `service.securityContext` | Set restricted baseline on `service.securityContext` (see sample below) | -| `PermissionError` on `/var/lib/nemo-retriever/retriever-service.log` when `persistence.enabled=false` | Default log path is image-owned; random UID cannot write without a PVC | Point `serviceConfig.logging.file` at `/tmp/...` (chart mounts `emptyDir` at `/tmp`) | -| `CreateContainerConfigError`: non-numeric image `USER nemo` on **vectordb** | Vectordb container has no `securityContext` block for SCC to annotate | Disable vectordb for smoke tests, or patch the vectordb Deployment after install (below) | -| PSA warnings on **otel-collector** | Otel Deployment has no `securityContext` in the chart | `topology.otel.enabled=false` unless you patch that Deployment | - -### Recommended value overrides +Helm installs the chart-owned OpenTelemetry Collector and Zipkin backend on by +default. This is intentional: the legacy 26.1.2 Helm chart shipped with a +managed Zipkin deployment enabled, so the new chart keeps a default trace +backend available for functional parity. Pod trace export is also enabled by +default for retriever service pods and chart-managed NIMs: ```yaml -# OpenShift overrides for nemo-retriever Helm chart (restricted-v2 / PSA restricted). -# Save locally, then: helm install retriever ./nemo_retriever/helm -f .yaml ... +topology: + otel: + enabled: true + zipkin: + enabled: true service: - podSecurityContext: - runAsNonRoot: true - # Do NOT set runAsUser, runAsGroup, or fsGroup — OpenShift SCC assigns them. - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: ["ALL"] - seccompProfile: - type: RuntimeDefault - -serviceConfig: - logging: - # Writable without persistence PVC (chart always mounts emptyDir at /tmp). - file: /tmp/retriever-service.log - vectordb: - # Set false for minimal service-only validation; see vectordb patch below if enabled. - enabled: false + otel: + enabled: true -topology: +nimOperator: otel: - enabled: false + enabled: true ``` -When **`persistence.enabled=true`**, you can keep the default log path under -`persistence.mountPath` (`/var/lib/nemo-retriever`) because the PVC is mounted and -SCC-assigned `fsGroup` applies. When persistence is off, always relocate logs to -`/tmp` (or another path backed by `service.extraVolumes`). - -### Example install on OpenShift 4.20 (service-only smoke test) +Because Zipkin is chart-owned by default, an upgrade with default values can +create a Zipkin Deployment and Service. Set `topology.zipkin.enabled=false` +before upgrading if your deployment uses an external backend or should not run +chart-owned Zipkin. -Matches QA validation with external NIMs disabled, no persistence, and no results -PVC: +With default values, retriever service pods and chart-managed NIMs emit OTLP to +the chart's OpenTelemetry Collector, which exports traces to the chart-owned +Zipkin service. Set `service.otel.enabled=false` or +`nimOperator.otel.enabled=false` to opt out by surface. Open a job and read the +Zipkin lookup key from either the JSON body or the `x-trace-id` response header: ```bash -oc new-project nemo-retriever - -oc create secret docker-registry ngc-secret -n nemo-retriever \ - --docker-server=nvcr.io --docker-username='$oauthtoken' \ - --docker-password="$NGC_API_KEY" +kubectl port-forward svc/tracing-smoke-nemo-retriever 7670:80 -oc create secret generic ngc-api -n nemo-retriever \ - --from-literal=NGC_API_KEY="$NGC_API_KEY" \ - --from-literal=NGC_CLI_API_KEY="$NGC_API_KEY" +curl -s -D headers.txt -o job.json \ + -X POST http://localhost:7670/v1/ingest/job \ + -H 'content-type: application/json' \ + -d '{"expected_documents":1}' -helm install retriever ./nemo_retriever/helm -n nemo-retriever \ - -f .yaml \ - --set ngcImagePullSecret.create=false \ - --set ngcApiSecret.create=false \ - --set nims.enabled=false \ - --set persistence.enabled=false \ - --set retrieverResults.enabled=false +TRACE_ID=$(jq -r .trace_id job.json) +grep -i x-trace-id headers.txt ``` -Verify pods: +Port-forward Zipkin and query the trace directly: ```bash -oc get pods -n nemo-retriever -oc describe pod -l app.kubernetes.io/name=nemo-retriever -n nemo-retriever +kubectl port-forward svc/tracing-smoke-nemo-retriever-zipkin 9411:9411 +curl "http://localhost:9411/api/v2/trace/${TRACE_ID}" ``` -You should see SCC-assigned numeric `runAsUser` on containers that declare a -`securityContext` block, and no PSA warnings once overrides are applied. +Common opt-out and override knobs: -### Enabling the vectordb Deployment on OpenShift +```yaml +topology: + zipkin: + enabled: false # do not deploy chart-owned Zipkin + exporter: + enabled: false # keep Zipkin deployed, but do not export traces to it + endpoint: http://external-zipkin:9411/api/v2/spans -`serviceConfig.vectordb.enabled=true` renders a **vectordb** container from the -same image (`USER nemo`, non-numeric). The chart does not yet expose a -`securityContext` value for that container. After `helm install`, patch the -Deployment so OpenShift can inject a numeric UID into the container spec: +service: + otel: + enabled: false # do not inject service pod instrumentation env -```bash -RELEASE=retriever -NS=nemo-retriever -VDB_DEPLOY="${RELEASE}-nemo-retriever-vectordb" - -oc patch deployment "$VDB_DEPLOY" -n "$NS" --type=json -p='[ - {"op": "add", "path": "/spec/template/spec/containers/0/securityContext", "value": { - "allowPrivilegeEscalation": false, - "capabilities": {"drop": ["ALL"]}, - "runAsNonRoot": true, - "seccompProfile": {"type": "RuntimeDefault"} - }} -]' +nimOperator: + otel: + enabled: false # do not inject inherited NIM OTLP env + page_elements: + otel: + enabled: false # per-NIM opt-out + ocr: + otel: + env: + TRITON_OTEL_RATE: "10" # per-NIM Triton OTel override ``` -Re-apply the patch after `helm upgrade` if the Deployment is recreated. A future -chart release may add first-class `topology.vectordb.securityContext` values. +Set `topology.zipkin.exporter.endpoint` when you run your own Zipkin-compatible +collector. Set `topology.otel.enabled=false` to disable the chart-owned collector +and all chart-rendered collector wiring. -### Enabling the OpenTelemetry collector on OpenShift - -The chart’s otel-collector Deployment likewise lacks `securityContext` fields. -Prefer `topology.otel.enabled=false` (as in the sample values) unless you operate -your own collector or patch `*-otel` the same way as vectordb. - -### What we intentionally do not require on OpenShift - -Do **not** bind the namespace to **anyuid** SCC or set PSA `enforce=privileged` -unless your security team explicitly approves it. The overrides above are intended -to keep **restricted-v2** / PSA **restricted** posture. +--- -### Related documentation +## OpenShift deployment { #openshift-deployment } -- [Pre-Requisites & Support Matrix](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md) -- [Deployment options](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/deployment-options.md) +OpenShift install procedures, **restricted-v2** / PSA **restricted** value overrides, prebuilt `ffmpeg` images, internal registry pull secrets, optional NIM `LD_LIBRARY_PATH` tuning, and install examples are in **[OpenShift deployment](./openshift.md)**. Pass `-f openshift-restricted.yaml` from that guide when you install on OpenShift. --- ## Air-gapped deployment { #air-gapped-deployment } -See [Deployment options — Air-gapped and disconnected deployment](https://docs.nvidia.com/nemo/retriever/latest/extraction/deployment-options/#air-gapped-deployment) for overview and workflow. Chart-specific reference for mirroring: +Refer to [Deployment options — Air-gapped and disconnected deployment](https://docs.nvidia.com/nemo/retriever/latest/extraction/deployment-options/#air-gapped-deployment) for overview and workflow. Chart-specific reference for mirroring: -### Container images to mirror (26.05 chart defaults) +### Container images to mirror (chart defaults) -Verify tags on the Git branch or tag you ship (for example `26.05` or -`26.5.0`). Defaults below match +Verify tags on the Git branch or tag you ship (for example `main` or +your release tag). Defaults below match [`values.yaml`](./values.yaml) on the current chart. | Role | `nimOperator` key | Default image (`repository:tag`) | @@ -1103,9 +1147,10 @@ Verify tags on the Git branch or tag you ship (for example `26.05` or | Table structure | `table_structure` | `nvcr.io/nim/nvidia/nemotron-table-structure-v1:1.8.0` | | OCR | `ocr` | `nvcr.io/nim/nvidia/nemotron-ocr-v1:1.3.0` | | VL embed | `vlm_embed` | `nvcr.io/nim/nvidia/llama-nemotron-embed-vl-1b-v2:1.12.0` | -| VL reranker (optional) | `rerankqa` | `nvcr.io/nim/nvidia/llama-nemotron-rerank-vl-1b-v2:1.10.0` | +| VL reranker (optional) | `rerankqa` | `nvcr.io/nim/nvidia/llama-nemotron-rerank-vl-1b-v2:1.11.0` | | Nemotron Parse (optional) | `nemotron_parse` | `nvcr.io/nim/nvidia/nemotron-parse-v1.2:1.7.0-variant` | | Omni caption (optional) | `nemotron_3_nano_omni_30b_a3b_reasoning` | `nvcr.io/nim/nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:1.7.0-variant` | +| Answer LLM (optional, Super-49B default) | `answer_llm` | `nvcr.io/nim/nvidia/llama-3.3-nemotron-super-49b-v1.5:2.0.5` | | Parakeet ASR (optional) | `audio` | `nvcr.io/nim/nvidia/parakeet-1-1b-ctc-en-us:1.5.0` | GPU SKU support for `audio` is in [Model hardware requirements](https://github.com/NVIDIA/NeMo-Retriever/blob/main/docs/docs/extraction/prerequisites-support-matrix.md#model-hardware-requirements). @@ -1135,7 +1180,8 @@ imagePullSecrets: - name: my-private-registry ngcImagePullSecret: - create: false # use secrets that authenticate to YOUR mirror + create: false + name: "" # Explicitly empty — clears the default "ngc-secret" nimOperator: page_elements: @@ -1146,15 +1192,15 @@ nimOperator: # Repeat for table_structure, ocr, vlm_embed, and any optional keys you enable. ``` -- Set `nimOperator..image.pullSecrets` to the Secret name your - `NIMService` resources should use (defaults to `ngc-secret`). +- Set `nimOperator..image.pullSecrets` to your mirror pull secret + (for example `my-private-registry`; chart default is `ngc-secret`). - Leave `serviceConfig.nimEndpoints.*` empty when operator-managed NIMs are in-cluster; set explicit URLs only for external or mirrored services outside the chart. - For **offline captioning**, enable `nimOperator.nemotron_3_nano_omni_30b_a3b_reasoning` and point the pipeline - caption endpoint at the in-cluster NIM URL (see - [Image captioning (26.05)](https://docs.nvidia.com/nemo/retriever/latest/extraction/prerequisites-support-matrix/#image-captioning-2605)). + caption endpoint at the in-cluster NIM URL (refer to + [Image captioning](https://docs.nvidia.com/nemo/retriever/latest/extraction/prerequisites-support-matrix/#image-captioning)). ### Mirroring pattern @@ -1209,7 +1255,7 @@ helm template r nemo_retriever/helm \ ``` Both renders should succeed cleanly and parse as valid Kubernetes manifests -(`kubectl apply --dry-run=client -f /tmp/r.yaml`). See [VectorDB and the +(`kubectl apply --dry-run=client -f /tmp/r.yaml`). Refer to [VectorDB and the embed endpoint](#vectordb-and-the-embed-endpoint) for why `helm template r nemo_retriever/helm` without flags is rejected as a misconfiguration.