Skip to content

docs(26.05): split Jacob CLI and README reconcile updates #2315

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
kheiss-uwzoo wants to merge 1 commit into from
Closed

docs(26.05): split Jacob CLI and README reconcile updates #2315

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions docs/docs/extraction/releasenotes.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,8 @@ Highlights for the 26.05 release include:

### CLI

- Root CLI adds `retriever ingest` and `retriever query` with NIM URL flags, batch tuning, and LanceDB overwrite/append controls, plus `retriever pipeline` for graph execution
- For product use, only `retriever ingest`, `retriever query`, and `retriever pipeline` (for example `retriever pipeline run`) are supported; other top-level subcommands—including `pdf`, `html`, `eval`, `benchmark`, `harness`, `online`, `compare`, `image`, and `skill-eval`—are development and experimental
- Root CLI adds first-class `retriever ingest` and `retriever query` commands with NIM URL flags, batch tuning, and LanceDB overwrite/append controls
- For product ingest and retrieval, prefer `retriever ingest` and `retriever query`; `retriever pipeline run` remains available for compatibility and development workflows. Other top-level subcommands—including `pdf`, `html`, `eval`, `benchmark`, `harness`, `online`, `compare`, `image`, and `skill-eval`—are development and experimental

### Retriever Service and deployment

Expand Down Expand Up @@ -64,7 +64,6 @@ Highlights for the 26.05 release include:

### Packaging and platform

- GA PyPI install: `uv pip install nemo-retriever==26.5.0` (refer to the library [quickstart](https://github.com/NVIDIA/NeMo-Retriever/tree/26.05/nemo_retriever#setup-your-environment))
- Optional install extras (`[local]`, `[multimedia]`, `[llm]`, `[tabular]`, `[nemotron-parse]`, `[service]`, and others), including slim remote/NIM-only installs on Mac and Windows

### Helm chart
Expand Down
40 changes: 22 additions & 18 deletions docs/docs/extraction/troubleshoot.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ Use this documentation to troubleshoot issues that arise when you use [NeMo Retr

## Can't process long, non-language text strings

NeMo Retriever Library is designed to process language and language-length strings.
If you submit a document that contains extremely long, or non-language text strings,
NeMo Retriever Library is designed to process language and language-length strings.
If you submit a document that contains extremely long, or non-language text strings,
such as a DNA sequence, errors or unexpected results occur.

## Can't process malformed input files
Expand All @@ -17,7 +17,7 @@ When you run a job you might see errors similar to the following:
- File may be malformed
- Failed to format paragraph

These errors can occur when your input file is malformed.
These errors can occur when your input file is malformed.
Verify or fix the format of your input file, and try resubmitting your job.

## Audio or video extraction reports missing media dependencies { #audio-or-video-extraction-reports-missing-media-dependencies }
Expand All @@ -33,7 +33,7 @@ VideoFrameActor requires media dependencies; missing: ffprobe.
The `ffmpeg-python` wrapper and `nemo-retriever[multimedia]` do not install the
`ffmpeg` or `ffprobe` binaries the pipeline executes.

For air-gapped or locked-down clusters, see [Air-gapped and disconnected deployment](deployment-options.md#air-gapped-deployment).
For air-gapped or locked-down clusters, refer to [Air-gapped and disconnected deployment](deployment-options.md#air-gapped-deployment).

**Connected environments:**

Expand All @@ -60,7 +60,7 @@ This path fails with `allowPrivilegeEscalation: false` or `readOnlyRootFilesyste

## Can't start new thread error

In rare cases, when you run a job you might an see an error similar to `can't start new thread`.
In rare cases, when you run a job you might an see an error similar to `can't start new thread`.
This error occurs when the maximum number of processes available to a single user is too low.
To resolve the issue, set or raise the maximum number of processes (`-u`) by using the [ulimit](https://ss64.com/bash/ulimit.html) command.
Before you change the `-u` setting, consider the following:
Expand All @@ -76,26 +76,30 @@ ulimit -u 10000

## Out-of-Memory (OOM) Error when Processing Large Datasets

When you process a very large dataset with thousands of documents, you might encounter an Out-of-Memory (OOM) error.
This happens because, by default, NeMo Retriever Library stores the results from every document in system memory (RAM).
When you process a very large dataset with thousands of documents, you might encounter an Out-of-Memory (OOM) error.
This happens because NeMo Retriever Library materializes extraction results in system memory (RAM) while the job runs.
If the total size of the results exceeds the available memory, the process fails.

To resolve this issue, use the `save_to_disk` method.
For details, refer to [Working with Large Datasets: Saving to Disk](nemo-retriever-api-reference.md).
To reduce memory pressure, try one or more of the following:

- Process documents in smaller batches instead of submitting the entire corpus in one job.
- Route outputs to a sink (for example, `.vdb_upload(...)`, `.webhook(...)`, or `.store(...)`) so results are written out instead of held in memory until the job finishes.
- In `run_mode="service"`, pass `return_results=False` to `.ingest(...)` when you do not need the full result payload returned to the client. For parameter details, refer to the [Python API guide](nemo-retriever-api-reference.md).
- Increase available host or pod memory for the ingest workload.



## Embedding service fails to start with an unsupported batch size error

On certain hardware, for example RTX 6000,
On certain hardware, for example RTX 6000,
the embedding service might fail to start and you might see an error similar to the following.

```bash
ValueError: Configured max_batch_size (30) is larger than the model''s supported max_batch_size (3).
```

If you are using hardware where the embedding NIM uses the ONNX model profile,
you must set `EMBEDDER_BATCH_SIZE=3` in your environment.
If you are using hardware where the embedding NIM uses the ONNX model profile,
you must set `EMBEDDER_BATCH_SIZE=3` in your environment.
You can set the variable in your .env file or directly in your environment.


Expand All @@ -122,18 +126,18 @@ For local GPU inference with Nemotron Parse, combine extras:
pip install "nemo-retriever[local,nemotron-parse]"
```

See also [What is NeMo Retriever Library?](overview.md) and [Pre-Requisites & Support Matrix](prerequisites-support-matrix.md#software-requirements).
Also refer to [What is NeMo Retriever Library?](overview.md) and [Pre-Requisites & Support Matrix](prerequisites-support-matrix.md#software-requirements).

## Extract method nemotron-parse doesn't support image files

Currently, extraction with Nemotron parse doesn't support image files, only scanned PDFs.
Currently, extraction with Nemotron parse doesn't support image files, only scanned PDFs.
To work around this issue, convert image files to PDFs before you use `extract_method="nemotron_parse"`.



## Too many open files error

In rare cases, when you run a job you might an see an error similar to `too many open files` or `max open file descriptor`.
In rare cases, when you run a job you might an see an error similar to `too many open files` or `max open file descriptor`.
This error occurs when the open file descriptor limit for your service user account is too low.
To resolve the issue, set or raise the maximum number of open file descriptors (`-n`) by using the [ulimit](https://ss64.com/bash/ulimit.html) command.
Before you change the `-n` setting, consider the following:
Expand All @@ -149,8 +153,8 @@ ulimit -n 10000

## Triton server INFO messages incorrectly logged as errors

Sometimes messages are incorrectly logged as errors, when they are information.
When this happens, you can ignore the errors, and treat the messages as information.
Sometimes messages are incorrectly logged as errors, when they are information.
When this happens, you can ignore the errors, and treat the messages as information.
For example, you might see log messages that look similar to the following.

```bash
Expand Down Expand Up @@ -185,4 +189,4 @@ ERROR 2025-04-24 22:49:44.434 nimutils.py:68] }
- [Pre-Requisites & Support Matrix](prerequisites-support-matrix.md)
- [Deployment options](deployment-options.md)
- [Deploy with Helm](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md)
- [NeMo Retriever Library — prerequisites / deployment](https://docs.nvidia.com/nemo/retriever/latest/extraction/overview/) (supported **Helm** charts)
- [About getting started](getting-started-about.md) (prerequisites and deployment)
65 changes: 29 additions & 36 deletions nemo_retriever/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Before starting, make sure your system meets the following requirements:
- The host is running CUDA 13.x so that `libcudart.so.13` is available.
- Your GPUs are visible to the system and compatible with CUDA 13.x.
​
If optical character recognition (OCR) fails with a `libcudart.so.13` error, install the CUDA 13 runtime for your platform and update `LD_LIBRARY_PATH` to include the CUDA lib64 directory, then rerun the pipeline.
If optical character recognition (OCR) fails with a `libcudart.so.13` error, install the CUDA 13 runtime for your platform and update `LD_LIBRARY_PATH` to include the CUDA lib64 directory, then rerun the pipeline.

For example, the following command can be used to update the `LD_LIBRARY_PATH` value.

Expand All @@ -44,7 +44,15 @@ For **local GPU inference** (Nemotron models running on your GPU), install with
```bash
uv venv retriever --python 3.12
source retriever/bin/activate
uv pip install "nemo-retriever[local]==26.5.0"
uv pip install "nemo-retriever[local]"
```

The `[local]` extra resolves stable Nemotron extraction packages by default. To
try prerelease/nightly Nemotron packages from PyPI within the same supported
major-version windows, opt in with `--pre`:

```bash
uv pip install --pre "nemo-retriever[local]==26.05-RC1"
```

Install matching **ingestion client** and **ingestion runtime** wheels at the same version when your workflow expects them (refer to the [NeMo Retriever Library prerequisites](https://docs.nvidia.com/nemo/retriever/latest/extraction/overview/) for the exact PyPI coordinates for your release).
Expand All @@ -55,7 +63,7 @@ For **remote NIM inference only** (no local GPU required), the base package is s
uv python install 3.12
uv venv retriever --python 3.12
source retriever/bin/activate
uv pip install nemo-retriever==26.5.0
uv pip install nemo-retriever
```

Install matching **ingestion client** and **ingestion runtime** wheels at the same version when your workflow expects them (refer to the [NeMo Retriever Library prerequisites](https://docs.nvidia.com/nemo/retriever/latest/extraction/overview/) for the exact PyPI coordinates for your release).
Expand All @@ -65,7 +73,7 @@ This creates a dedicated Python environment and installs the `nemo-retriever` Py
If your PDF pipeline uses `extract_method="nemotron_parse"`, install the Nemotron Parse client dependencies with the `nemotron-parse` extra:

```bash
uv pip install "nemo-retriever[nemotron-parse]==26.5.0"
uv pip install "nemo-retriever[nemotron-parse]"
```

For local GPU inference with Nemotron Parse, combine the extras as `nemo-retriever[local,nemotron-parse]`.
Expand All @@ -87,9 +95,10 @@ Skip this step if you are using remote NIM inference only.

The [test PDF](../data/multimodal_test.pdf) contains text, tables, charts, and images. Additional test data resides [here](../data/).

> **Note:** `batch` is the primary intended run_mode of operation for this library. Other modes are experimental and subject to change or removal.
> **Note:** `retriever ingest` defaults to local, in-process execution. Use `retriever ingest batch ...` for Ray Data scale-out on larger workloads.
> `retriever pipeline run` keeps its legacy `--run-mode` flag for compatibility and development workflows.

The examples below use default local GPU inference (no `invoke_url` specified) and require the `[local]` extra and the CUDA 13 torch override from the setup steps above. For remote NIM inference without a local GPU, see [Run with remote inference](#run-with-remote-inference-no-local-gpu-required).
The examples below use default local GPU inference (no `invoke_url` specified) and require the `[local]` extra and the CUDA 13 torch override from the setup steps above. For remote NIM inference without a local GPU, refer to [Run with remote inference](#run-with-remote-inference-no-local-gpu-required).

### Ingest a test pdf
```python
Expand Down Expand Up @@ -159,7 +168,7 @@ used in [Run a recall query](#run-a-recall-query) below. With the
and embedding. For a realistic retrieval corpus, see
[QA evaluation -- Step 1](./src/nemo_retriever/evaluation/README.md#step-1-ingest-and-embed-pdfs-nemo-retriever).

**No local GPU?** Set [`NVIDIA_API_KEY`](https://nvidia.github.io/NeMo-Retriever/extraction/api-keys/#nvidia-api-key) (see [Authentication and API keys](https://nvidia.github.io/NeMo-Retriever/extraction/api-keys/)) and route extraction and embedding
**No local GPU?** Set [`NVIDIA_API_KEY`](https://nvidia.github.io/NeMo-Retriever/extraction/api-keys/#nvidia-api-key) (refer to [Authentication and API keys](https://nvidia.github.io/NeMo-Retriever/extraction/api-keys/)) and route extraction and embedding
through [build.nvidia.com](https://build.nvidia.com/) NIMs instead:

```bash
Expand Down Expand Up @@ -426,7 +435,7 @@ ingestor = (
### Render results as markdown

If you want a readable markdown view of extracted results, pass a single document's extraction
records to `nemo_retriever.io.to_markdown`. The helper returns one markdown string (or `None`
records to `nemo_retriever.common.io.to_markdown`. The helper returns one markdown string (or `None`

@greptile-apps greptile-apps Bot Jul 7, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Markdown Import Path Missing

to_markdown is exported from nemo_retriever.io, and the package layout inspected for this branch does not include a nemo_retriever.common package. A reader copying this changed path as from nemo_retriever.common.io import to_markdown gets ImportError: No module named 'nemo_retriever.common' before any markdown rendering can run.

Suggested change
records to `nemo_retriever.common.io.to_markdown`. The helper returns one markdown string (or `None`
records to `nemo_retriever.io.to_markdown`. The helper returns one markdown string (or `None`
Prompt To Fix With AI 
This is a comment left during a code review.
Path: nemo_retriever/README.md
Line: 438

Comment:
**Markdown Import Path Missing**

`to_markdown` is exported from `nemo_retriever.io`, and the package layout inspected for this branch does not include a `nemo_retriever.common` package. A reader copying this changed path as `from nemo_retriever.common.io import to_markdown` gets `ImportError: No module named 'nemo_retriever.common'` before any markdown rendering can run.

```suggestion
records to `nemo_retriever.io.to_markdown`. The helper returns one markdown string (or `None`
```

How can I resolve this? If you propose a fix, please make it concise.

if there is no content), with per-page sections joined under a single document heading.

For multi-document runs, pass one document at a time—for example, `to_markdown(results[0])`.
Expand Down Expand Up @@ -508,7 +517,7 @@ ingestor = (
.embed(
model_name="nvidia/llama-nemotron-embed-vl-1b-v2",
#works with plain "text"s, "image"s, and "text_image" pairs
embed_modality="text_image"
embed_modality="text_image"
)
)
```
Expand All @@ -520,7 +529,7 @@ ingestor = ingestor.files(documents).extract(method="nemotron_parse")

## Run with remote inference, no local GPU required:

For build.nvidia.com hosted inference, set [`NVIDIA_API_KEY`](https://nvidia.github.io/NeMo-Retriever/extraction/api-keys/#nvidia-api-key) as an environment variable (see [Authentication and API keys](https://nvidia.github.io/NeMo-Retriever/extraction/api-keys/)).
For build.nvidia.com hosted inference, set [`NVIDIA_API_KEY`](https://nvidia.github.io/NeMo-Retriever/extraction/api-keys/#nvidia-api-key) as an environment variable (refer to [Authentication and API keys](https://nvidia.github.io/NeMo-Retriever/extraction/api-keys/)).

```python
ingestor = (
Expand Down Expand Up @@ -625,38 +634,22 @@ sudo apt install python3.12-dev

After installing the headers, restart the pipeline.

## ViDoRe Harness Sweep

The harness includes BEIR-style ViDoRe dataset presets in `nemo_retriever/harness/test_configs.yaml` and a ready-made sweep definition in `nemo_retriever/harness/vidore_sweep.yaml`.

The ViDoRe harness datasets are configured to:

- read PDFs from `/datasets/retrieval-eval/vidore_v3_corpus_pdf/...`
- ingest with `embed_modality: text_image`
- embed at `embed_granularity: page`
- enable `extract_page_as_image: true` and `extract_infographics: true`
- evaluate with BEIR-style `ndcg` and `recall` metrics

To run the full ViDoRe sweep:

```bash
cd ~/NeMo-Retriever/nemo_retriever
retriever-harness sweep --runs-config harness/vidore_sweep.yaml
```
## Retriever Harness

The same commands also work under the main CLI as `retriever harness ...` if you prefer a single top-level command namespace.
The developer harness runs code-owned benchmarks through `retriever harness`.
Use `retriever harness list --runsets` to see available benchmark names and
runsets, then run one benchmark with `retriever harness run <benchmark>`.

### Pipeline image storage
### Ingest image storage

Use the pipeline CLI to persist extracted image assets to local storage or any
Use root ingest to persist extracted image assets to local storage or any
fsspec-compatible URI:

```bash
retriever pipeline run ./data \
retriever ingest ./data \
--store-images-uri ./processed_docs/images
Comment on lines +649 to 650

@greptile-apps greptile-apps Bot Jul 7, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Image Store Flag Misplaced

The current root ingest command does not expose --store-images-uri; that flag was found on the pipeline compatibility command. A user following this new root-ingest image-storage example gets an unknown-option error instead of persisting extracted images.

Prompt To Fix With AI 
This is a comment left during a code review.
Path: nemo_retriever/README.md
Line: 649-650

Comment:
**Image Store Flag Misplaced**

The current root ingest command does not expose `--store-images-uri`; that flag was found on the pipeline compatibility command. A user following this new root-ingest image-storage example gets an unknown-option error instead of persisting extracted images.

How can I resolve this? If you propose a fix, please make it concise.

```

The store stage writes the image payloads produced by the configured pipeline.
With `--embed-granularity page`, stored assets are page images. With
`--embed-granularity element`, stored assets are element images. Store is not
currently configured through the harness.
The store stage writes the image payloads produced by ingest. With
`--embed-granularity page`, stored assets are page images. With
`--embed-granularity element`, stored assets are element images.
Loading
Loading