Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 8 additions & 16 deletions docs/docs/extraction/deployment-options.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,24 +19,16 @@ Build and run the NeMo Retriever service image with the [Docker service image gu

1. [Pre-Requisites & Support Matrix](prerequisites-support-matrix.md)
2. **NeMo Retriever Helm chart (supported):** [Deploy (Helm chart)](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md) — sources in [`nemo_retriever/helm`](https://github.com/NVIDIA/NeMo-Retriever/tree/main/nemo_retriever/helm) on GitHub
3. **Published Library Helm charts (supported):** cluster install and upgrade procedures are covered in the [NeMo Retriever Library](https://docs.nvidia.com/nemo/retriever/latest/extraction/overview/) — use alongside the NeMo Retriever chart README for your release
3. **Published Library Helm charts (supported):** cluster install and upgrade procedures are covered in [About getting started](getting-started-about.md) — use alongside the NeMo Retriever chart README for your release
4. [Environment variables](environment-config.md) and [Troubleshoot](troubleshoot.md) as needed

**Core NIMs for the default extraction pipeline** (26.05): `page_elements`, `table_structure`, `ocr`, and `vlm_embed` (`llama-nemotron-embed-vl-1b-v2:1.12.0`). These four are auto-wired into the retriever service. **Nemotron Parse**, **Nemotron 3 Nano Omni**, the **VL reranker**, and **Parakeet ASR** are optional and not auto-wired. For a minimal GPU footprint, disable optional keys you do not need (see [Recommended minimal install (26.05)](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#recommended-minimal-install-2605)). See [Pre-Requisites & Support Matrix — Default Helm NIMs](prerequisites-support-matrix.md#default-helm-nims).
**Core NIMs for the default extraction pipeline:** `page_elements`, `table_structure`, `ocr`, and `vlm_embed` (`llama-nemotron-embed-vl-1b-v2:1.12.0`). These four are auto-wired into the retriever service. **Nemotron Parse**, **Nemotron 3 Nano Omni**, the **VL reranker**, and **Parakeet ASR** are optional and not auto-wired. For a minimal GPU footprint, disable optional keys you do not need (refer to [Recommended minimal install](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#recommended-minimal-install-2605)). Refer to [Pre-Requisites & Support Matrix — Default Helm NIMs](prerequisites-support-matrix.md#default-helm-nims).


For audio and video extraction in Kubernetes, set `service.installFfmpeg=true`
so the service container installs `ffmpeg` and `ffprobe` at startup. This
runtime install requires package-repository network egress, a writable root
filesystem, and security policy that allows the image's scoped sudo use. If
your cluster blocks startup package installation (for example air-gapped
environments), use a custom service image that already contains `ffmpeg` and
`ffprobe`, then set `service.image.repository` and `service.image.tag`.
For audio and video extraction in Kubernetes, set `service.installFfmpeg=true` so the service container installs `ffmpeg` and `ffprobe` at startup. This runtime install requires package-repository network egress, a writable root filesystem, and security policy that allows the image's scoped sudo use. If your cluster blocks startup package installation, use a custom service image that already contains `ffmpeg` and `ffprobe`, then set `service.image.repository` and `service.image.tag`. For Parakeet ASR chart values, OpenShift-specific Helm configuration, and air-gapped alternatives, refer to [Audio and video (Parakeet ASR)](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#audio-video-parakeet) and [OpenShift deployment](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/openshift.md) in the Helm chart directory.

### I want examples and notebooks

1. [Jupyter Notebooks](notebooks/index.md)
2. [Integrate with LangChain, LlamaIndex, Haystack](integrations-langchain-llamaindex-haystack.md)
1. [Jupyter Notebooks](starter-kits.md)
Comment thread
greptile-apps[bot] marked this conversation as resolved.

### I need API details and keys

Expand All @@ -49,7 +41,7 @@ environments), use a custom service image that already contains `ffmpeg` and
2. [Throughput is dataset-dependent](multimodal-extraction.md#extraction-limitations-and-quality)
3. [Evaluate on your data](evaluate-on-your-data.md)

## When to use NVIDIA-hosted NIMs
## When to use NVIDIA-hosted NIMs { #when-to-use-nvidia-hosted-nims }

[NVIDIA-hosted NIMs](https://build.nvidia.com/) run inference on NVIDIA-managed infrastructure. You call models with API keys (refer to [Get your API key](api-keys.md)) without operating GPU nodes yourself.

Expand All @@ -61,7 +53,7 @@ Consider hosted NIMs when:

**Also refer to:** [NVIDIA NIM catalog](https://build.nvidia.com/)

## When to self-host NIMs
## When to self-host NIMs { #when-to-self-host-nims }

Self-hosted NIMs run on your GPUs or air-gapped hardware, typically with Kubernetes and the [NIM Operator](https://docs.nvidia.com/nim-operator/latest/index.html).

Expand All @@ -77,7 +69,7 @@ Consider self-hosting when:

The **default document extraction pipeline** (page elements, table structure, OCR, and VL embed) runs disconnected when you mirror images and models into a private registry and configure the [NIM Operator for air-gapped environments](https://docs.nvidia.com/nim-operator/latest/air-gap.html).

On a staging host with internet access, pull from NGC, retag to your private registry, stage chart archives, then install in the enclave with registry overrides. Procedures, the 26.05 image inventory, and Helm value patterns are in [Helm — Air-gapped deployment](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#air-gapped-deployment).
On a staging host with internet access, pull from NGC, retag to your private registry, stage chart archives, then install in the enclave with registry overrides. Procedures, the chart image inventory, and Helm value patterns are in [Helm — Air-gapped deployment](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#air-gapped-deployment).

!!! warning "Audio and video extraction"

Expand All @@ -88,6 +80,6 @@ For offline image captioning, deploy the in-cluster [Nemotron 3 Nano Omni](prere
**Related**

- [Deploy (Helm chart)](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md) ([`nemo_retriever/helm`](https://github.com/NVIDIA/NeMo-Retriever/tree/26.05/nemo_retriever/helm) on GitHub) — [air-gapped deployment](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md#air-gapped-deployment)
- [NeMo Retriever Library — prerequisites / deployment](https://docs.nvidia.com/nemo/retriever/latest/extraction/overview/) (supported **Helm** handoff)
- [About getting started](getting-started-about.md) (prerequisites through first deployment)
- [Pre-Requisites & Support Matrix](prerequisites-support-matrix.md)
- [Audio and video](audio-video.md)
13 changes: 6 additions & 7 deletions docs/docs/extraction/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,16 +21,17 @@ For more information, refer to [Vector databases](vdbs.md).
For images that `nemoretriever-page-elements-v3` does not classify as tables, charts, or infographics,
you can use our VLM caption task to create a dense caption of the detected image.
That caption is then embedded along with the rest of your content.
For chart-labeled PDF regions and other caption scope limits, see [Are PDF chart or figure regions captioned when Omni is enabled?](#are-pdf-chart-or-figure-regions-captioned-when-omni-is-enabled). For more information, refer to [Extract Captions from Images](nemo-retriever-api-reference.md).
For chart-labeled PDF regions and other caption scope limits, refer to [Are PDF chart or figure regions captioned when Omni is enabled?](#are-pdf-chart-or-figure-regions-captioned-when-omni-is-enabled). For more information, refer to [Extract Captions from Images](nemo-retriever-api-reference.md).

## Are PDF chart or figure regions captioned when Omni is enabled?

No. Chart-labeled PDF regions are not routed through Omni captioning. See [Image captioning](prerequisites-support-matrix.md#image-captioning-2605) for scope, validation, and what the caption stage covers.
No. Chart-labeled PDF regions are not routed through Omni captioning. Refer to [Charts and infographics](multimodal-extraction.md#charts-and-infographics) and [Image captioning](multimodal-extraction.md#image-captioning) for caption scope and validation.

## When should I consider advanced visual parsing?

For scanned documents, or documents with complex layouts,
you can use [nemotron-parse](https://build.nvidia.com/nvidia/nemotron-parse) as an alternate PDF extraction method by setting `extract_method="nemotron_parse"`.
For scanned documents, or documents with complex layouts,
you can use [nemotron-parse](https://build.nvidia.com/nvidia/nemotron-parse) as an alternate PDF extraction method by setting `extract_method="nemotron_parse"`.
Nemotron Parse does not produce chart modality rows. For chart detection and chart-filtered retrieval, use the default **pdfium** layout path instead (refer to [Charts and infographics](multimodal-extraction.md#charts-and-infographics)).
For more information, refer to [Nemotron Parse](https://build.nvidia.com/nvidia/nemotron-parse).

## Why are the environment variables different between library mode and self-hosted mode?
Expand All @@ -40,11 +41,9 @@ For more information, refer to [Nemotron Parse](https://build.nvidia.com/nvidia/
For [self-hosted deployments](deployment-options.md#when-to-self-host-nims), you should set the environment variables `NGC_API_KEY` and `NIM_NGC_API_KEY`.
For more information, refer to [Authentication and API keys](api-keys.md).

For advanced scenarios, you might want to set environment variables for NIM container paths, tags, and batch sizes on the ingestion runtime. Configure them in your Helm values, Kubernetes `Secret`/`ConfigMap`, or follow [Environment variables](environment-config.md).

### Library Mode

For production environments, you should use the provided Helm charts. When you run the NeMo Retriever Library from Python (without those charts), you should set the environment variable `NVIDIA_API_KEY`. This is because the NeMo Retriever containers and the NeMo Retriever services running inside them do not have access to arbitrary variables on your laptop or jump host unless you inject them into the workload (for example via Helm, `Secret`, or the client environment as documented on [Deployment options](deployment-options.md) and [Authentication and API keys](api-keys.md)).
For production environments, you should use the provided Helm charts. When you run the NeMo Retriever Library from Python without those charts, set `NVIDIA_API_KEY` only when you call [build.nvidia.com](https://build.nvidia.com/) hosted inference—it is not required for locally deployed Hugging Face models or self-hosted NIM endpoints. For more information, refer to [Deployment options](deployment-options.md) and [Authentication and API keys](api-keys.md).

For advanced scenarios, you might want to use library mode with self-hosted NIM instances.
You can set custom endpoints for each NIM.
Expand Down
6 changes: 3 additions & 3 deletions docs/docs/extraction/getting-started-about.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ Typical order:
1. [Get your API key](api-keys.md) (NGC / API access as required by your workflow).
2. Confirm the [Pre-Requisites & Support Matrix](prerequisites-support-matrix.md) for your OS, GPU, and software stack.
3. Deploy using one of:
- [Deployment options](deployment-options.md) for how to run NeMo Retriever Library
- **Supported:** [Helm chart](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md) for Kubernetes, plus [NeMo Retriever Library install docs](https://docs.nvidia.com/nemo/retriever/latest/extraction/overview/) for the published charts
4. Explore [Jupyter Notebooks](notebooks/index.md) for end-to-end examples.
- [Deployment options](deployment-options.md) for library, hosted NIMs, and Kubernetes paths
- **Supported:** [Helm chart](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md) for Kubernetes cluster install and upgrade

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Pin release Helm link

This getting-started page is for the 26.05 release flow, but the supported Helm chart link still opens the main branch. A new Kubernetes user can follow this first-run path, copy chart values or upgrade guidance added after 26.05, and apply settings that do not match the 26.05 chart. Point this link at the 26.05 branch so the release docs hand off to release-matched Helm docs.

Suggested change
- **Supported:** [Helm chart](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md) for Kubernetes cluster install and upgrade
- **Supported:** [Helm chart](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md) for Kubernetes cluster install and upgrade
Prompt To Fix With AI
This is a comment left during a code review.
Path: docs/docs/extraction/getting-started-about.md
Line: 11

Comment:
**Pin release Helm link**

This getting-started page is for the 26.05 release flow, but the supported Helm chart link still opens the `main` branch. A new Kubernetes user can follow this first-run path, copy chart values or upgrade guidance added after 26.05, and apply settings that do not match the 26.05 chart. Point this link at the 26.05 branch so the release docs hand off to release-matched Helm docs.

```suggestion
    - **Supported:** [Helm chart](https://github.com/NVIDIA/NeMo-Retriever/blob/26.05/nemo_retriever/helm/README.md) for Kubernetes cluster install and upgrade
```

How can I resolve this? If you propose a fix, please make it concise.

4. Explore [Jupyter Notebooks](starter-kits.md) for end-to-end examples.
Comment thread
greptile-apps[bot] marked this conversation as resolved.

If you are new to the product, read [What is NeMo Retriever Library?](overview.md) and [Concepts](concepts.md) under **Introduction** first.
5 changes: 2 additions & 3 deletions docs/docs/extraction/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,5 @@ NeMo Retriever Library supports the following file types:
- [Pre-Requisites & Support Matrix](prerequisites-support-matrix.md)
- [Deployment options](deployment-options.md) — library, Helm, hosted vs self-hosted NIMs in one place
- [Deploy on Kubernetes with Helm](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md)
- [NeMo Retriever Library — prerequisites / deployment](https://docs.nvidia.com/nemo/retriever/latest/extraction/overview/) (supported Helm charts)
- [Notebooks](notebooks/index.md)
- [NVIDIA AI Blueprints catalog](https://build.nvidia.com/explore/discover) — solution cards, enterprise RAG blueprints, and end-to-end patterns (including [Enterprise RAG — multimodal PDF data extraction](https://build.nvidia.com/nvidia/multimodal-pdf-data-extraction-for-enterprise-rag)); for integration pathways, refer to [Integrations](integrations-langchain-llamaindex-haystack.md).
- [Notebooks](starter-kits.md)
- [NVIDIA AI Blueprints catalog](https://build.nvidia.com/explore/discover) — solution cards, enterprise RAG blueprints, and end-to-end patterns (including [Enterprise RAG — multimodal PDF data extraction](https://build.nvidia.com/nvidia/multimodal-pdf-data-extraction-for-enterprise-rag)); for integration pathways, refer to [Starter kits](starter-kits.md).
Comment on lines +51 to +52

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Overview Links Target Missing Page

Both changed overview links now reference starter-kits.md, which does not exist in docs/docs/extraction/. This can break the strict docs build and sends overview readers to a missing page instead of the existing notebooks and integration guidance.

Suggested change
- [Notebooks](starter-kits.md)
- [NVIDIA AI Blueprints catalog](https://build.nvidia.com/explore/discover) — solution cards, enterprise RAG blueprints, and end-to-end patterns (including [Enterprise RAG — multimodal PDF data extraction](https://build.nvidia.com/nvidia/multimodal-pdf-data-extraction-for-enterprise-rag)); for integration pathways, refer to [Starter kits](starter-kits.md).
- [Notebooks](notebooks/index.md)
- [NVIDIA AI Blueprints catalog](https://build.nvidia.com/explore/discover) — solution cards, enterprise RAG blueprints, and end-to-end patterns (including [Enterprise RAG — multimodal PDF data extraction](https://build.nvidia.com/nvidia/multimodal-pdf-data-extraction-for-enterprise-rag)); for integration pathways, refer to [Integrations](integrations-langchain-llamaindex-haystack.md).
Prompt To Fix With AI
This is a comment left during a code review.
Path: docs/docs/extraction/overview.md
Line: 51-52

Comment:
**Overview Links Target Missing Page**

Both changed overview links now reference `starter-kits.md`, which does not exist in `docs/docs/extraction/`. This can break the strict docs build and sends overview readers to a missing page instead of the existing notebooks and integration guidance.

```suggestion
- [Notebooks](notebooks/index.md)
- [NVIDIA AI Blueprints catalog](https://build.nvidia.com/explore/discover) — solution cards, enterprise RAG blueprints, and end-to-end patterns (including [Enterprise RAG — multimodal PDF data extraction](https://build.nvidia.com/nvidia/multimodal-pdf-data-extraction-for-enterprise-rag)); for integration pathways, refer to [Integrations](integrations-langchain-llamaindex-haystack.md).
```

How can I resolve this? If you propose a fix, please make it concise.

Loading