diff --git a/docs/docs/extraction/releasenotes.md b/docs/docs/extraction/releasenotes.md index e8c5b34b86..dd1a2b483a 100644 --- a/docs/docs/extraction/releasenotes.md +++ b/docs/docs/extraction/releasenotes.md @@ -25,8 +25,8 @@ Highlights for the 26.05 release include: ### CLI -- Root CLI adds `retriever ingest` and `retriever query` with NIM URL flags, batch tuning, and LanceDB overwrite/append controls, plus `retriever pipeline` for graph execution -- For product use, only `retriever ingest`, `retriever query`, and `retriever pipeline` (for example `retriever pipeline run`) are supported; other top-level subcommands—including `pdf`, `html`, `eval`, `benchmark`, `harness`, `online`, `compare`, `image`, and `skill-eval`—are development and experimental +- Root CLI adds first-class `retriever ingest` and `retriever query` commands with NIM URL flags, batch tuning, and LanceDB overwrite/append controls +- For product ingest and retrieval, prefer `retriever ingest` and `retriever query`; `retriever pipeline run` remains available for compatibility and development workflows. Other top-level subcommands—including `pdf`, `html`, `eval`, `benchmark`, `harness`, `online`, `compare`, `image`, and `skill-eval`—are development and experimental ### Retriever Service and deployment @@ -64,7 +64,6 @@ Highlights for the 26.05 release include: ### Packaging and platform -- GA PyPI install: `uv pip install nemo-retriever==26.5.0` (refer to the library [quickstart](https://github.com/NVIDIA/NeMo-Retriever/tree/26.05/nemo_retriever#setup-your-environment)) - Optional install extras (`[local]`, `[multimedia]`, `[llm]`, `[tabular]`, `[nemotron-parse]`, `[service]`, and others), including slim remote/NIM-only installs on Mac and Windows ### Helm chart diff --git a/docs/docs/extraction/troubleshoot.md b/docs/docs/extraction/troubleshoot.md index 4d0bc9c8d9..127fee07a6 100644 --- a/docs/docs/extraction/troubleshoot.md +++ b/docs/docs/extraction/troubleshoot.md @@ -4,8 +4,8 @@ Use this documentation to troubleshoot issues that arise when you use [NeMo Retr ## Can't process long, non-language text strings -NeMo Retriever Library is designed to process language and language-length strings. -If you submit a document that contains extremely long, or non-language text strings, +NeMo Retriever Library is designed to process language and language-length strings. +If you submit a document that contains extremely long, or non-language text strings, such as a DNA sequence, errors or unexpected results occur. ## Can't process malformed input files @@ -17,7 +17,7 @@ When you run a job you might see errors similar to the following: - File may be malformed - Failed to format paragraph -These errors can occur when your input file is malformed. +These errors can occur when your input file is malformed. Verify or fix the format of your input file, and try resubmitting your job. ## Audio or video extraction reports missing media dependencies { #audio-or-video-extraction-reports-missing-media-dependencies } @@ -33,7 +33,7 @@ VideoFrameActor requires media dependencies; missing: ffprobe. The `ffmpeg-python` wrapper and `nemo-retriever[multimedia]` do not install the `ffmpeg` or `ffprobe` binaries the pipeline executes. -For air-gapped or locked-down clusters, see [Air-gapped and disconnected deployment](deployment-options.md#air-gapped-deployment). +For air-gapped or locked-down clusters, refer to [Air-gapped and disconnected deployment](deployment-options.md#air-gapped-deployment). **Connected environments:** @@ -60,7 +60,7 @@ This path fails with `allowPrivilegeEscalation: false` or `readOnlyRootFilesyste ## Can't start new thread error -In rare cases, when you run a job you might an see an error similar to `can't start new thread`. +In rare cases, when you run a job you might an see an error similar to `can't start new thread`. This error occurs when the maximum number of processes available to a single user is too low. To resolve the issue, set or raise the maximum number of processes (`-u`) by using the [ulimit](https://ss64.com/bash/ulimit.html) command. Before you change the `-u` setting, consider the following: @@ -76,26 +76,30 @@ ulimit -u 10000 ## Out-of-Memory (OOM) Error when Processing Large Datasets -When you process a very large dataset with thousands of documents, you might encounter an Out-of-Memory (OOM) error. -This happens because, by default, NeMo Retriever Library stores the results from every document in system memory (RAM). +When you process a very large dataset with thousands of documents, you might encounter an Out-of-Memory (OOM) error. +This happens because NeMo Retriever Library materializes extraction results in system memory (RAM) while the job runs. If the total size of the results exceeds the available memory, the process fails. -To resolve this issue, use the `save_to_disk` method. -For details, refer to [Working with Large Datasets: Saving to Disk](nemo-retriever-api-reference.md). +To reduce memory pressure, try one or more of the following: + +- Process documents in smaller batches instead of submitting the entire corpus in one job. +- Route outputs to a sink (for example, `.vdb_upload(...)`, `.webhook(...)`, or `.store(...)`) so results are written out instead of held in memory until the job finishes. +- In `run_mode="service"`, pass `return_results=False` to `.ingest(...)` when you do not need the full result payload returned to the client. For parameter details, refer to the [Python API guide](nemo-retriever-api-reference.md). +- Increase available host or pod memory for the ingest workload. ## Embedding service fails to start with an unsupported batch size error -On certain hardware, for example RTX 6000, +On certain hardware, for example RTX 6000, the embedding service might fail to start and you might see an error similar to the following. ```bash ValueError: Configured max_batch_size (30) is larger than the model''s supported max_batch_size (3). ``` -If you are using hardware where the embedding NIM uses the ONNX model profile, -you must set `EMBEDDER_BATCH_SIZE=3` in your environment. +If you are using hardware where the embedding NIM uses the ONNX model profile, +you must set `EMBEDDER_BATCH_SIZE=3` in your environment. You can set the variable in your .env file or directly in your environment. @@ -122,18 +126,18 @@ For local GPU inference with Nemotron Parse, combine extras: pip install "nemo-retriever[local,nemotron-parse]" ``` -See also [What is NeMo Retriever Library?](overview.md) and [Pre-Requisites & Support Matrix](prerequisites-support-matrix.md#software-requirements). +Also refer to [What is NeMo Retriever Library?](overview.md) and [Pre-Requisites & Support Matrix](prerequisites-support-matrix.md#software-requirements). ## Extract method nemotron-parse doesn't support image files -Currently, extraction with Nemotron parse doesn't support image files, only scanned PDFs. +Currently, extraction with Nemotron parse doesn't support image files, only scanned PDFs. To work around this issue, convert image files to PDFs before you use `extract_method="nemotron_parse"`. ## Too many open files error -In rare cases, when you run a job you might an see an error similar to `too many open files` or `max open file descriptor`. +In rare cases, when you run a job you might an see an error similar to `too many open files` or `max open file descriptor`. This error occurs when the open file descriptor limit for your service user account is too low. To resolve the issue, set or raise the maximum number of open file descriptors (`-n`) by using the [ulimit](https://ss64.com/bash/ulimit.html) command. Before you change the `-n` setting, consider the following: @@ -149,8 +153,8 @@ ulimit -n 10000 ## Triton server INFO messages incorrectly logged as errors -Sometimes messages are incorrectly logged as errors, when they are information. -When this happens, you can ignore the errors, and treat the messages as information. +Sometimes messages are incorrectly logged as errors, when they are information. +When this happens, you can ignore the errors, and treat the messages as information. For example, you might see log messages that look similar to the following. ```bash @@ -185,4 +189,4 @@ ERROR 2025-04-24 22:49:44.434 nimutils.py:68] } - [Pre-Requisites & Support Matrix](prerequisites-support-matrix.md) - [Deployment options](deployment-options.md) - [Deploy with Helm](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/helm/README.md) -- [NeMo Retriever Library — prerequisites / deployment](https://docs.nvidia.com/nemo/retriever/latest/extraction/overview/) (supported **Helm** charts) +- [About getting started](getting-started-about.md) (prerequisites and deployment) diff --git a/nemo_retriever/README.md b/nemo_retriever/README.md index 7c2747cd42..3dcc626696 100644 --- a/nemo_retriever/README.md +++ b/nemo_retriever/README.md @@ -21,7 +21,7 @@ Before starting, make sure your system meets the following requirements: - The host is running CUDA 13.x so that `libcudart.so.13` is available. - Your GPUs are visible to the system and compatible with CUDA 13.x. ​ -If optical character recognition (OCR) fails with a `libcudart.so.13` error, install the CUDA 13 runtime for your platform and update `LD_LIBRARY_PATH` to include the CUDA lib64 directory, then rerun the pipeline. +If optical character recognition (OCR) fails with a `libcudart.so.13` error, install the CUDA 13 runtime for your platform and update `LD_LIBRARY_PATH` to include the CUDA lib64 directory, then rerun the pipeline. For example, the following command can be used to update the `LD_LIBRARY_PATH` value. @@ -44,7 +44,15 @@ For **local GPU inference** (Nemotron models running on your GPU), install with ```bash uv venv retriever --python 3.12 source retriever/bin/activate -uv pip install "nemo-retriever[local]==26.5.0" +uv pip install "nemo-retriever[local]" +``` + +The `[local]` extra resolves stable Nemotron extraction packages by default. To +try prerelease/nightly Nemotron packages from PyPI within the same supported +major-version windows, opt in with `--pre`: + +```bash +uv pip install --pre "nemo-retriever[local]==26.05-RC1" ``` Install matching **ingestion client** and **ingestion runtime** wheels at the same version when your workflow expects them (refer to the [NeMo Retriever Library prerequisites](https://docs.nvidia.com/nemo/retriever/latest/extraction/overview/) for the exact PyPI coordinates for your release). @@ -55,7 +63,7 @@ For **remote NIM inference only** (no local GPU required), the base package is s uv python install 3.12 uv venv retriever --python 3.12 source retriever/bin/activate -uv pip install nemo-retriever==26.5.0 +uv pip install nemo-retriever ``` Install matching **ingestion client** and **ingestion runtime** wheels at the same version when your workflow expects them (refer to the [NeMo Retriever Library prerequisites](https://docs.nvidia.com/nemo/retriever/latest/extraction/overview/) for the exact PyPI coordinates for your release). @@ -65,7 +73,7 @@ This creates a dedicated Python environment and installs the `nemo-retriever` Py If your PDF pipeline uses `extract_method="nemotron_parse"`, install the Nemotron Parse client dependencies with the `nemotron-parse` extra: ```bash -uv pip install "nemo-retriever[nemotron-parse]==26.5.0" +uv pip install "nemo-retriever[nemotron-parse]" ``` For local GPU inference with Nemotron Parse, combine the extras as `nemo-retriever[local,nemotron-parse]`. @@ -87,9 +95,10 @@ Skip this step if you are using remote NIM inference only. The [test PDF](../data/multimodal_test.pdf) contains text, tables, charts, and images. Additional test data resides [here](../data/). -> **Note:** `batch` is the primary intended run_mode of operation for this library. Other modes are experimental and subject to change or removal. +> **Note:** `retriever ingest` defaults to local, in-process execution. Use `retriever ingest batch ...` for Ray Data scale-out on larger workloads. +> `retriever pipeline run` keeps its legacy `--run-mode` flag for compatibility and development workflows. -The examples below use default local GPU inference (no `invoke_url` specified) and require the `[local]` extra and the CUDA 13 torch override from the setup steps above. For remote NIM inference without a local GPU, see [Run with remote inference](#run-with-remote-inference-no-local-gpu-required). +The examples below use default local GPU inference (no `invoke_url` specified) and require the `[local]` extra and the CUDA 13 torch override from the setup steps above. For remote NIM inference without a local GPU, refer to [Run with remote inference](#run-with-remote-inference-no-local-gpu-required). ### Ingest a test pdf ```python @@ -159,7 +168,7 @@ used in [Run a recall query](#run-a-recall-query) below. With the and embedding. For a realistic retrieval corpus, see [QA evaluation -- Step 1](./src/nemo_retriever/evaluation/README.md#step-1-ingest-and-embed-pdfs-nemo-retriever). -**No local GPU?** Set [`NVIDIA_API_KEY`](https://nvidia.github.io/NeMo-Retriever/extraction/api-keys/#nvidia-api-key) (see [Authentication and API keys](https://nvidia.github.io/NeMo-Retriever/extraction/api-keys/)) and route extraction and embedding +**No local GPU?** Set [`NVIDIA_API_KEY`](https://nvidia.github.io/NeMo-Retriever/extraction/api-keys/#nvidia-api-key) (refer to [Authentication and API keys](https://nvidia.github.io/NeMo-Retriever/extraction/api-keys/)) and route extraction and embedding through [build.nvidia.com](https://build.nvidia.com/) NIMs instead: ```bash @@ -426,7 +435,7 @@ ingestor = ( ### Render results as markdown If you want a readable markdown view of extracted results, pass a single document's extraction -records to `nemo_retriever.io.to_markdown`. The helper returns one markdown string (or `None` +records to `nemo_retriever.common.io.to_markdown`. The helper returns one markdown string (or `None` if there is no content), with per-page sections joined under a single document heading. For multi-document runs, pass one document at a time—for example, `to_markdown(results[0])`. @@ -508,7 +517,7 @@ ingestor = ( .embed( model_name="nvidia/llama-nemotron-embed-vl-1b-v2", #works with plain "text"s, "image"s, and "text_image" pairs - embed_modality="text_image" + embed_modality="text_image" ) ) ``` @@ -520,7 +529,7 @@ ingestor = ingestor.files(documents).extract(method="nemotron_parse") ## Run with remote inference, no local GPU required: -For build.nvidia.com hosted inference, set [`NVIDIA_API_KEY`](https://nvidia.github.io/NeMo-Retriever/extraction/api-keys/#nvidia-api-key) as an environment variable (see [Authentication and API keys](https://nvidia.github.io/NeMo-Retriever/extraction/api-keys/)). +For build.nvidia.com hosted inference, set [`NVIDIA_API_KEY`](https://nvidia.github.io/NeMo-Retriever/extraction/api-keys/#nvidia-api-key) as an environment variable (refer to [Authentication and API keys](https://nvidia.github.io/NeMo-Retriever/extraction/api-keys/)). ```python ingestor = ( @@ -625,38 +634,22 @@ sudo apt install python3.12-dev After installing the headers, restart the pipeline. -## ViDoRe Harness Sweep - -The harness includes BEIR-style ViDoRe dataset presets in `nemo_retriever/harness/test_configs.yaml` and a ready-made sweep definition in `nemo_retriever/harness/vidore_sweep.yaml`. - -The ViDoRe harness datasets are configured to: - -- read PDFs from `/datasets/retrieval-eval/vidore_v3_corpus_pdf/...` -- ingest with `embed_modality: text_image` -- embed at `embed_granularity: page` -- enable `extract_page_as_image: true` and `extract_infographics: true` -- evaluate with BEIR-style `ndcg` and `recall` metrics - -To run the full ViDoRe sweep: - -```bash -cd ~/NeMo-Retriever/nemo_retriever -retriever-harness sweep --runs-config harness/vidore_sweep.yaml -``` +## Retriever Harness -The same commands also work under the main CLI as `retriever harness ...` if you prefer a single top-level command namespace. +The developer harness runs code-owned benchmarks through `retriever harness`. +Use `retriever harness list --runsets` to see available benchmark names and +runsets, then run one benchmark with `retriever harness run `. -### Pipeline image storage +### Ingest image storage -Use the pipeline CLI to persist extracted image assets to local storage or any +Use root ingest to persist extracted image assets to local storage or any fsspec-compatible URI: ```bash -retriever pipeline run ./data \ +retriever ingest ./data \ --store-images-uri ./processed_docs/images ``` -The store stage writes the image payloads produced by the configured pipeline. -With `--embed-granularity page`, stored assets are page images. With -`--embed-granularity element`, stored assets are element images. Store is not -currently configured through the harness. +The store stage writes the image payloads produced by ingest. With +`--embed-granularity page`, stored assets are page images. With +`--embed-granularity element`, stored assets are element images. diff --git a/nemo_retriever/docs/cli/README.md b/nemo_retriever/docs/cli/README.md index f4b3d5ab54..a84c4cf9bb 100644 --- a/nemo_retriever/docs/cli/README.md +++ b/nemo_retriever/docs/cli/README.md @@ -1,81 +1,56 @@ -# Retriever CLI — replacement examples for the legacy ingestion-service CLI - -This folder contains `retriever` command-line examples that deliver the same -end-user outcomes as the legacy **ingestion-service** CLI examples that used to -live under `docs/`, `api/`, `client/`, and `deploy/` in older repository layouts. - -The historical CLI documentation is **not removed** from the ecosystem — these files sit -alongside it as a new-CLI counterpart you can link to or migrate to. - -## Supported vs development / experimental subcommands - -For product use and published examples, treat only these top-level subcommands as -the **supported public path**: - -- **`retriever ingest`** — ingest documents into LanceDB -- **`retriever query`** — query an existing LanceDB table - -`retriever pipeline` remains available as a **development / compatibility** -wrapper, including `retriever pipeline run`, while ingestion behavior migrates -onto the same implementation used by `retriever ingest`. Prefer `retriever -ingest` and `retriever query` for user-facing workflows. - -Any other top-level `retriever` subcommand — including but not limited to -`pipeline`, `pdf`, `html`, `txt`, `audio`, `chart`, `benchmark`, `harness`, -`eval`, `recall`, `service`, `local`, `compare`, `image`, and `skill-eval` — -is **development and experimental**. These commands may change without public -compatibility guarantees. - -## Key shape difference - -The legacy **ingestion-service** CLI was a **single command that talks to a running REST service on -`localhost:7670`** and composes work via repeated `--task extract|split|caption|embed|dedup|filter|udf`. - -`retriever` is a **multi-subcommand Typer app**. Public ingest/query examples -should map to `retriever ingest INPUT_PATH` followed by `retriever query ...`. -`retriever pipeline run INPUT_PATH` is still present for development workflows -that need pipeline-only evaluation, runtime summaries, Parquet export, or -service-mode compatibility. - -| Old intent | New subcommand | -|------------|----------------| -| Extract + embed + store a batch of documents | `retriever ingest` | -| Run an ad-hoc PDF extraction stage | `retriever pdf stage` | -| Run an HTML / text / audio / chart stage | `retriever html run`, `retriever txt run`, `retriever audio extract`, `retriever chart run` | -| Upload stage output to LanceDB | `retriever ingest` | -| Query LanceDB + compute recall@k | `retriever recall vdb-recall` | -| Run a QA evaluation sweep | `retriever eval run` | -| Serve / submit to the online REST API | `retriever online serve` / `retriever online stream-pdf` | -| Benchmark stage throughput | `retriever benchmark {split,extract,audio-extract,page-elements,ocr,all}` | -| Benchmark orchestration | `retriever harness {run,sweep,nightly,summary,compare}` | - -Rows that use subcommands other than `ingest` or `query` are -[development and experimental](#supported-vs-development--experimental-subcommands). - -## Contents - -| Topic | Location | Replaces example(s) in | -|-------|----------|------------------------| -| Quick start | [below](#quick-start) | Legacy service quickstart; **Helm** + [NeMo Retriever Library](https://docs.nvidia.com/nemo/retriever/latest/extraction/overview/) | -| CLI reference | [below](#cli-reference) | Prior `cli-reference` pages under `docs/docs/extraction/` | -| Client usage walk-through | [below](#client-usage-walk-through) | `client/client_examples/examples/cli_client_usage.ipynb` | -| PDF pre-splitting | [API guide](../../../docs/docs/extraction/nemo-retriever-api-reference.md#pdf-pre-splitting-for-parallel-ingest); [Large PDF page batches](#large-pdf-page-batches) below | Prior extraction docs | -| Benchmarking | [`benchmarking.md`](benchmarking.md) | `docs/docs/extraction/benchmarking.md` and `nemo_retriever/harness/HANDOFF.md` | +# Retriever CLI + +This page describes the public `retriever` command-line workflow for document +ingest and retrieval. + +For product-facing examples, prefer these commands: + +- `retriever ingest` - ingest supported documents and media into a Retriever index. +- `retriever query` - query a local LanceDB table written by local or batch ingest. +- `retriever query service` - query a Retriever service deployment. + +`retriever pipeline run` remains available as a development and compatibility +command for legacy pipeline workflows, evaluation, intermediate artifacts, and +pipeline-specific debugging. It is not the preferred public ingest interface. + +## Public ingest shape + +`retriever ingest` defaults to local, in-process ingest: + +```bash +retriever ingest DOCUMENTS... +``` + +Explicit modes are also available: + +```bash +retriever ingest local DOCUMENTS... +retriever ingest batch DOCUMENTS... +retriever ingest service DOCUMENTS... +``` + +The root ingest CLI uses subcommands instead of a `--run-mode` flag. Choose +the command that matches where ingest runs and where results are stored. + +| Command | What It Does | Writes To | Use When | +|---|---|---|---| +| `retriever ingest ...` | Local in-process ingest | local LanceDB | Default local ingest and CI/small corpus runs. | +| `retriever ingest local ...` | Local in-process ingest | local LanceDB | Same as the default, but explicit. | +| `retriever ingest batch ...` | Ray-backed batch ingest | local LanceDB | Larger or batch-tuned runs. | +| `retriever ingest service ...` | Sends documents to a Retriever service | service-configured storage | Remote service ingest. | + +This separation keeps invalid flag combinations out of the parser. For example, +service ingest does not expose LanceDB target flags, Ray tuning, local endpoint +configuration, local embed backend selection, or local media controls. > Use `retriever ingest` and `retriever query` for product-facing workflows. -> `retriever pipeline` is development / compatibility only; see -> [Supported vs development / experimental subcommands](#supported-vs-development--experimental-subcommands). +> `retriever pipeline run` is development / compatibility only. ## Quick start -For deployment of NeMo Retriever / **NIM** containers, use -[nemo_retriever/helm](https://github.com/NVIDIA/NeMo-Retriever/tree/main/nemo_retriever/helm) -and the [NeMo Retriever Library](https://docs.nvidia.com/nemo/retriever/latest/extraction/overview/) -Helm install guides. - -### Ingest a PDF +### Ingest a PDF locally ```bash retriever ingest ./data/multimodal_test.pdf \ @@ -85,19 +60,49 @@ retriever ingest ./data/multimodal_test.pdf \ --embed-model-name nvidia/llama-nemotron-embed-1b-v2 ``` -Then query the LanceDB table: +Then query the default LanceDB table: ```bash retriever query "What is in this document?" \ --embed-model-name nvidia/llama-nemotron-embed-1b-v2 ``` -Development-only pipeline features such as `--save-intermediate`, runtime -summaries, and post-ingest evaluation remain on `retriever pipeline run` while -the public path is restricted to ingest/query. +By default, local ingest writes to `lancedb/nemo-retriever` and `retriever query` +reads from the same table. + +The plain `retriever query` examples below apply to local and batch ingest output +written to LanceDB. Use `retriever query service` to query a Retriever service. + +### Ingest a larger corpus with batch mode + +```bash +retriever ingest batch ./data/pdf_corpus \ + --profile fast-text \ + --pdf-extract-workers 4 \ + --embed-workers 2 +``` + +Batch mode exposes Ray runtime and batch tuning flags such as `--ray-address`, +`--pdf-extract-workers`, `--ocr-workers`, and `--embed-workers`. -Route stages to self-hosted or hosted NIM endpoints by passing only the URLs you -want to override: +### Ingest through a Retriever service + +```bash +retriever ingest service ./data/pdf_corpus \ + --service-url http://localhost:7670 \ + --service-concurrency 8 +``` + +Use `--service-api-token` or `NEMO_RETRIEVER_API_TOKEN` when the service requires +a bearer token. Service ingest does not expose `--lancedb-uri`; the service +configures its vector database. Query the service with: + +```bash +retriever query service "What is in this corpus?" \ + --service-url http://localhost:7670 +``` + +### Route ingest to hosted or self-hosted NIM endpoints ```bash export NVIDIA_API_KEY=nvapi-... @@ -108,365 +113,247 @@ retriever ingest ./data/multimodal_test.pdf \ --table-structure-invoke-url https://ai.api.nvidia.com/v1/cv/nvidia/nemotron-table-structure-v1 \ --embed-invoke-url https://integrate.api.nvidia.com/v1/embeddings \ --embed-model-name nvidia/llama-nemotron-embed-1b-v2 +``` + +`NVIDIA_API_KEY` is required only when those URLs point at hosted +build.nvidia.com endpoints. `NGC_API_KEY` is used separately when pulling or +running self-hosted NIM containers. + +For NVIDIA inference hub rerank models that expose the Cohere-style rerank +route, pass the full `/v1/rerank` URL and the model name shown in the hub +snippet: + +```bash +export NGC_INFERENCE_API_KEY=... retriever query "What is in this document?" \ --embed-invoke-url https://integrate.api.nvidia.com/v1/embeddings \ --embed-model-name nvidia/llama-nemotron-embed-1b-v2 \ - --reranker-invoke-url https://ai.api.nvidia.com/v1/retrieval/nvidia/llama-nemotron-rerank-vl-1b-v2/reranking + --reranker-invoke-url https://inference-api.nvidia.com/v1/rerank \ + --reranker-model-name nvidia/nvidia/llama-3.2-nv-rerankqa-1b-v2 \ + --reranker-api-key-env NGC_INFERENCE_API_KEY ``` ### Query result controls -`retriever query` returns compact JSON hits with `source`, `page_number`, and `text`. -By default it retrieves and returns `--top-k` rows. Use these controls when you -need a wider candidate pool or a narrower result shape: +Both `retriever query` and `retriever query service` return compact JSON hits +with `source`, `page_number`, and `text`. Use `--candidate-k`, `--page-dedup`, +and `--content-types` to control how results are selected after vector +retrieval: ```bash -# Retrieve 30 candidates, then return the best 10. -retriever query "where is the warranty limitation discussed?" \ - --candidate-k 30 - -# Keep only the first hit from each document page. -retriever query "which pages discuss operating costs?" \ - --top-k 5 \ - --candidate-k 30 \ - --page-dedup - -# Search a wider pool, then keep only table rows. retriever query "annual revenue by region" \ --top-k 5 \ --candidate-k 40 \ --content-types table ``` -`--top-k` is the final number of hits returned. `--candidate-k` is the wider -candidate pool retrieved before page deduplication, content-type filtering, and -final truncation. It must be greater than or equal to `--top-k`, and should -usually be larger when page deduplication or content-type filtering might -otherwise remove too many of the top retrieved rows. Page deduplication and -content-type filtering are applied after vector retrieval, preserving the -retriever's ranking order and truncating the final output to `--top-k`. -When querying a table ingested with an explicit embedding model, pass the same -`--embed-model-name` to `retriever query`. -`--content-types` accepts comma-separated content types such as `text`, `table`, -`chart`, `image`, and `infographic`. `images` is accepted as an alias for -captioned image rows emitted by ingest. Hits with missing or unknown content -types are excluded while `--content-types` is active. +`--top-k` is the final number of results to return after filtering and +deduplication. `--candidate-k` is the number of raw results to retrieve from +LanceDB or the Retriever service before filtering, page deduplication, and +final truncation. If omitted, the candidate pool is the same size as +`--top-k`. Set `--candidate-k` larger than `--top-k` when page deduplication +or content-type filtering might remove too many of the nearest retrieved rows. +It must always be greater than or equal to `--top-k`. -`NVIDIA_API_KEY` is required only when those URLs point at hosted -build.nvidia.com endpoints. `NGC_API_KEY` is used separately when pulling or -running self-hosted NIM containers. +Page deduplication and content-type filtering are applied after vector +retrieval, preserving retriever ranking order and truncating the final output to +`--top-k`. When querying a local table ingested with an explicit embedding +model, pass the same `--embed-model-name` to `retriever query`. -### What you get +`--content-types` accepts comma-separated content types such as `text`, `table`, +`chart`, `image`, and `infographic`. `images` is accepted as an alias for +captioned image rows emitted by ingest. This option filters by content-type +metadata only; it does not filter by source, page, or other metadata +predicates. Hits with missing or unknown content-type metadata are excluded +while `--content-types` is active. In service mode, results must include +content-type metadata to match this filter. Default display values in the JSON +output are not used for content-type matching. -- Extracted text, tables, and charts as rows in LanceDB at `./lancedb` (default - table name `nemo-retriever`). -- Compact JSON retrieval hits from `retriever query`, including source, page, - and text fields. -- Extracted image assets when `retriever ingest` is run with - `--store-images-uri`. -- Pipeline-only development artifacts such as extraction Parquet, runtime - summaries, and evaluation reports remain available through - `retriever pipeline run`. -- Progress and stage logs on stderr. +### Agentic retrieval -### Inspect the results +`--agentic` swaps the single dense pass for an LLM-driven ReAct loop: the agent +issues several retrieval sub-queries, fuses the candidates, and selects a final +ranking. It searches the same LanceDB table built by `retriever ingest`, so it is +a drop-in alternative to standard retrieval — add `--agentic` and name the chat +model the agent drives with `--agentic-llm-model` (required): ```bash -ls ./lancedb -``` - -```python -import lancedb - -db = lancedb.connect("./lancedb") -tbl = db.open_table("nemo-retriever") -print(tbl.to_pandas().head()) -``` - -Or query via the Retriever Python client (`nemo_retriever/README.md`): - -```python -from nemo_retriever.retriever import Retriever - -retriever = Retriever( - vdb_kwargs={"uri": "lancedb", "table_name": "nemo-retriever"}, - embed_kwargs={ - "model_name": "nvidia/llama-nemotron-embed-1b-v2", - "embed_model_name": "nvidia/llama-nemotron-embed-1b-v2", - }, - top_k=5, -) -hits = retriever.query( - "Given their activities, which animal is responsible for the typos?" -) +retriever query "how does the ingestion pipeline handle tables?" \ + --agentic \ + --agentic-llm-model nvidia/llama-3.3-nemotron-super-49b-v1.5 + +# remote agent + embedding endpoints, fewer reasoning rounds +retriever query "summarize the deployment options" \ + --agentic \ + --agentic-llm-model nvidia/llama-3.3-nemotron-super-49b-v1.5 \ + --agentic-invoke-url http://localhost:9000/v1/chat/completions \ + --embed-invoke-url http://localhost:8000/v1 \ + --agentic-react-max-steps 5 ``` -### Larger datasets - -- Batch ingest: `retriever ingest ./data/pdf_corpus --run-mode batch`. -- Tune throughput with `--pdf-extract-workers`, `--pdf-extract-batch-size`, - `--page-elements-workers`, `--page-elements-batch-size`, `--ocr-workers`, - `--ocr-batch-size`, `--embed-workers`, and `--embed-batch-size`. -- For CI or debugging: `--run-mode inprocess` skips Ray startup. +Unlike the dense path (which returns text-enriched hits), agentic mode returns +the agent's ranked document IDs as JSON, each annotated with the source that +produced it (`final_results`, `rrf`, or `selection_agent`). It reuses the same +`--top-k`, `--lancedb-uri`, `--table-name`, `--embed-invoke-url`, and +`--embed-model-name` options as standard retrieval. + +**How it works.** Each agentic query runs `Query → ReActAgentOperator → (RRF +fusion) → SelectionAgentOperator → ranked results`: + +- `ReActAgentOperator` runs the per-query ReAct loop; every `retrieve` tool call + delegates to the standard `Retriever`, so the agent searches the same vector + DB and embedding config as dense retrieval. +- `RRFAggregatorOperator` fuses candidates from the loop's multiple searches with + reciprocal rank fusion. +- `SelectionAgentOperator` runs a final LLM selection pass over the fused set and + emits the ranked document IDs. + +Agentic-only knobs (apply only with `--agentic`): + +- `--agentic-invoke-url` — OpenAI-compatible chat-completions endpoint for the + agent LLM; defaults to the operators' built-in endpoint when omitted. +- `--agentic-reasoning-effort` (default `high`) — `reasoning_effort` forwarded on + agentic LLM calls. +- `--agentic-backend-top-k` (default `20`) — candidates pulled from the vector DB + per retrieval call. +- `--agentic-react-max-steps` (default `50`) — maximum ReAct loop iterations. +- `--agentic-text-truncation` (default `0`) — max characters of each candidate + shown to the agent; `0` disables truncation. +- `--agentic-temperature` (default `0.0`) — sampling temperature for agentic LLM + calls (`0.0` = greedy). -## CLI reference - -`retriever` is the Typer app installed with the `nemo-retriever` package. Subcommand -support policy: [Supported vs development / experimental subcommands](#supported-vs-development--experimental-subcommands). - -Document ingestion for users is `retriever ingest INPUT_PATH`, followed by -`retriever query` for retrieval. `retriever pipeline run INPUT_PATH` is retained -as a development / compatibility wrapper for pipeline-only behavior. +## Common ingest options + +### Local and batch ingest + +These options apply to `retriever ingest`, `retriever ingest local`, and +`retriever ingest batch` unless otherwise noted. + +| Option | Default | Notes | +|---|---|---| +| `DOCUMENTS...` | required | Files, directories, or shell globs. Supported file families are detected automatically. | +| `--profile` | `auto` | `auto` is normal manifest-routed ingest. `fast-text` is a PDF/document text-only profile for faster fallback ingest. | +| `--lancedb-uri` | `lancedb` | LanceDB database URI. | +| `--table-name` | `nemo-retriever` | LanceDB table name. Must match query-time storage flags. | +| `--overwrite/--append` | overwrite | Overwrite the table by default; use `--append` to add rows. | +| `--method` | planner default | PDF extraction method such as `pdfium` or `nemotron_parse`. | +| `--extract-text`, `--extract-tables`, `--extract-charts` | planner default | Enable or disable extraction families. | +| `--ocr-version` | planner default | OCR engine version for local extraction. | +| `--ocr-lang` | planner default | OCR v2 language selector for local extraction. | +| `--caption` | off | Add a captioning stage. | +| `--dedup` | off | Add image deduplication before captioning and embedding. | +| `--text-chunk` | off | Enable token chunking during extraction. | +| `--store-images-uri` | unset | Store extracted images at a local path or fsspec-compatible URI. | +| `--dry-run` | off | Print the resolved ingest plan without creating an ingestor. | +| `--quiet/--no-quiet` | quiet | Suppress verbose progress output by default. | + +Batch-only options include `--ray-address`, `--ray-log-to-driver`, +`--pdf-split-batch-size`, `--pdf-extract-workers`, `--ocr-workers`, +`--table-structure-workers`, `--nemotron-parse-workers`, `--embed-workers`, and +related batch-size / CPU / GPU tuning flags. + +### Service ingest + +`retriever ingest service` exposes only service-supported request controls. +It does not expose LanceDB target flags, Ray tuning, local endpoint URLs/API +keys, local embed backend selection, `--ocr-lang`, or local audio/video media +controls. + +| Option | Default | Notes | +|---|---|---| +| `DOCUMENTS...` | required | Files, directories, or shell globs sent to the service client. | +| `--service-url` | `http://localhost:7670` | Retriever service base URL. | +| `--service-concurrency` | `8` | Maximum concurrent document uploads. | +| `--service-api-token` | env fallback | Bearer token; also reads `NEMO_RETRIEVER_API_TOKEN`. | +| `--profile` | `auto` | Same profile names as local and batch ingest where supported. | +| `--caption`, `--dedup`, `--text-chunk` | off | Service-supported ingest controls. | +| `--store-images-uri` | unset | Service-accessible image storage URI. | +| `--dry-run` | off | Print the resolved service ingest request. Tokens are redacted. | + +## Examples + +### Custom LanceDB location ```bash -retriever --version -retriever --help -retriever ingest --help -retriever query --help +retriever ingest ./data/multimodal_test.pdf \ + --lancedb-uri ./my-lancedb \ + --table-name my-corpus ``` -### Extract a PDF with defaults - ```bash -retriever ingest ./data/test.pdf \ - --run-mode inprocess +retriever query "What is in this document?" \ + --lancedb-uri ./my-lancedb \ + --table-name my-corpus ``` -Results go to LanceDB (`./lancedb`, table `nemo-retriever` by default). Use -`retriever pipeline run --save-intermediate` only when you need development -Parquet artifacts. - -### Text chunking and PDF page batches - -Splitting is intrinsic to the pipeline. Control text chunks with `--text-chunk`. For -PDF pre-splitting and `--pdf-split-batch-size`, see -[PDF pre-splitting](../../../docs/docs/extraction/nemo-retriever-api-reference.md#pdf-pre-splitting-for-parallel-ingest) -and [Large PDF page batches](#large-pdf-page-batches): +### Fast text-only PDF fallback ```bash -retriever pipeline run ./data/test.pdf \ - --input-type pdf \ - --no-extract-tables --no-extract-charts \ - --text-chunk --text-chunk-max-tokens 512 --text-chunk-overlap-tokens 64 \ - --save-intermediate ./processed_docs +retriever ingest ./data/pdf_corpus \ + --profile fast-text \ + --embed-model-name nvidia/llama-nemotron-embed-1b-v2 ``` -There is no split-only mode without extraction; narrow flags to text extraction if you -only need chunk boundaries. - -### Nemotron OCR v2 language mode { #nemotron-ocr-v2-language-mode } - -The default OCR engine for **local** extraction (Hugging Face weights, no remote -`--ocr-invoke-url`) is **Nemotron OCR v2**, which runs in **multilingual** mode -by default (`multi`). - -| Flag | Values | Notes | -|------|--------|-------| -| `--ocr-lang` | `multi` (default), `english` | v2 only — English-only selector | -| `--ocr-version` | `v2` (default), `v1` | `v1` is the legacy English-only engine | +### OCR language mode ```bash -retriever pipeline run ./data/scanned.pdf \ - --input-type pdf \ - --method pdfium_hybrid \ +retriever ingest ./data/scanned.pdf \ + --ocr-version v2 \ --ocr-lang english - -retriever ingest ./data/scanned.pdf --ocr-version v1 ``` -Set the equivalent `ocr_lang` and `ocr_version` fields on `ExtractParams` (or the -ingest API) in Python. - -Remote OCR NIM endpoints choose their own model and language behavior. Local -`--ocr-lang` and `--ocr-version` are not sent on remote requests. For hosted -examples until OCR v2 is published on build.nvidia.com, keep -`--ocr-invoke-url` pointed at `nemotron-ocr-v1` (see [Quick start](#quick-start)). +For mixed-script documents, use `--ocr-lang multi` where supported by the local +OCR engine. -### PDF and Office documents - -Run once per input type (`--input-type doc` matches `*.docx` and `*.pptx`): +### Text chunking ```bash -retriever pipeline run ./data/test.pdf \ - --input-type pdf \ - --method pdfium \ - --text-chunk --text-chunk-max-tokens 512 \ - --save-intermediate ./processed_docs - -retriever pipeline run ./data/test.docx \ - --input-type doc \ - --text-chunk --text-chunk-max-tokens 512 \ - --save-intermediate ./processed_docs -``` - -Mixed PDF and docx in one invocation is not supported. - -### Large PDF page batches - -```bash -retriever pipeline run ./data/test.pdf \ - --input-type pdf \ - --method pdfium \ - --extract-text --no-extract-tables --no-extract-charts \ - --pdf-split-batch-size 64 \ - --save-intermediate ./processed_docs +retriever ingest ./data/test.pdf \ + --text-chunk \ + --text-chunk-max-tokens 512 \ + --text-chunk-overlap-tokens 64 ``` -### Caption images +### Captioning and image storage ```bash -retriever pipeline run ./data/test.pdf \ - --input-type pdf \ - --method pdfium \ +retriever ingest ./data/test.pdf \ --caption \ - --caption-model-name nvidia/nemotron-3-nano-omni-30b-a3b-reasoning \ --caption-invoke-url https://integrate.api.nvidia.com/v1/chat/completions \ --api-key "${NVIDIA_API_KEY}" \ - --store-images-uri ./processed_docs/images \ - --save-intermediate ./processed_docs + --store-images-uri ./processed_docs/images ``` -Custom caption prompts and `reasoning` flags are not exposed on the CLI — use -`nemo_retriever.ingestor.Ingestor.caption(...)` in Python. +## Results and diagnostics -### Directory of documents +Local and batch ingest report the number of input files and LanceDB rows written: -```bash -retriever pipeline run ./data/pdf_corpus \ - --input-type pdf \ - --method pdfium \ - --save-intermediate ./processed_docs +```text +Ingested 20 file(s) -> 1884 row(s) in LanceDB lancedb/nemo-retriever. ``` -There is no `dataset.json` loader; pass a directory or glob of files. +Service ingest reports the row count returned by the service result when +available: -### Store images to object storage - -```bash -retriever pipeline run ./data/test.pdf \ - --input-type pdf \ - --method pdfium \ - --store-images-uri s3://my-bucket/images \ - --save-intermediate ./processed_docs +```text +Ingested 20 file(s) -> 1940 row(s) through retriever service http://localhost:7670. ``` -Image URIs are written to row metadata. Use `--store-actors` to tune object-storage -write concurrency. +Use `--dry-run` on any ingest mode to inspect the resolved request without +creating an ingestor or contacting the service. -### Where results live +## Development / compatibility command -- **LanceDB** — `--lancedb-uri lancedb` (default). Default table name depends on the - subcommand: `retriever ingest` and `retriever query` use `nemo-retriever`; `retriever - pipeline run` still uses `nv-ingest`. Query via `retriever recall vdb-recall …` or - `nemo_retriever.retriever.Retriever`. -- **Parquet** — `--save-intermediate ` writes `/extraction.parquet`. -- **Images** — `--store-images-uri ` (local path or fsspec URI). Storage follows - `--embed-granularity` (page vs element images). +Use `retriever pipeline run` only when you need pipeline-specific behavior that +is intentionally not part of the first-class ingest/query workflow, such as: -### Errors and exit codes - -`retriever pipeline run` exits **0** on success and **non-zero** on validation or -pipeline failures. Use `--debug` or `--log-file ` for diagnostics. - -## Client usage walk-through - -Counterpart to `client/client_examples/examples/cli_client_usage.ipynb`. Covers help, a -single-PDF run, a batch directory run, and inspecting results. Drop these cells into a -notebook (e.g. `retriever_client_usage.ipynb`) if you prefer. - -### Help - -```bash -retriever --help -retriever pipeline run --help -``` - -Top-level `--help` lists the subcommand tree; `pipeline run --help` shows the -ingest-specific flags used below. - -### Run a single PDF - -```bash -retriever pipeline run "${SAMPLE_PDF0}" \ - --input-type pdf \ - --method pdfium \ - --extract-text --extract-tables --extract-charts \ - --dedup --dedup-iou-threshold 0.45 \ - --store-images-uri "${OUTPUT_DIRECTORY_SINGLE}/images" \ - --save-intermediate "${OUTPUT_DIRECTORY_SINGLE}" -``` - -- Table/structure detectors are chosen automatically; there is no CLI flag to pick a - specific table-extraction backend. -- `--dedup` with `--dedup-iou-threshold` removes duplicate image elements. -- There is no image scale/aspect-ratio filter in the `retriever` CLI today. -- `--store-images-uri` persists image assets at the configured embed granularity. - -### Run a batch of PDFs - -```bash -# $PDF_DIR is a directory of PDFs. -retriever pipeline run "${PDF_DIR}" \ - --input-type pdf \ - --method pdfium \ - --extract-text --extract-tables --extract-charts \ - --dedup --dedup-iou-threshold 0.45 \ - --store-images-uri "${OUTPUT_DIRECTORY_BATCH}/images" \ - --save-intermediate "${OUTPUT_DIRECTORY_BATCH}" -``` - -- Pass a directory or glob; there is no built-in `dataset.json` loader. -- Tune throughput with `--pdf-split-batch-size`, `--pdf-extract-batch-size`, etc. - -### Inspect results - -The batch walk-through above uses `retriever pipeline run`, which writes LanceDB table -`nv-ingest` by default. After `retriever ingest`, use `nemo-retriever` instead (see -[Inspect the results](#inspect-the-results)). - -```python -import pyarrow.parquet as pq -import lancedb - -df = pq.read_table(f"{OUTPUT_DIRECTORY_BATCH}/extraction.parquet").to_pandas() -print(df[["source_id", "text", "content_type"]].head()) - -db = lancedb.connect("./lancedb") -tbl = db.open_table("nv-ingest") -print(tbl.to_pandas().head()) -``` +- `--save-intermediate` Parquet artifacts. +- runtime metrics and pipeline reports. +- eval, recall, harness, or BEIR/QA workflows. +- legacy compatibility while callers migrate to `retriever ingest` and + `retriever query`. -## Gaps with no retriever-CLI equivalent (kept out of this folder) - -The following legacy **ingestion-service** CLI examples are **not** migrated here because the -new CLI does not yet expose an equivalent — continue to use the **ingestion-service** CLI -for these cases: - -- `--task 'udf:{…}'` — user-defined functions ([NeMo Retriever Graph](../../src/nemo_retriever/graph/README.md#nemo-retriever-graph)). `retriever` does not expose UDFs. -- `--task 'filter:{content_type:"image", min_size:…, min_aspect_ratio:…, max_aspect_ratio:…}'`. - The image scale/aspect-ratio filter stage is not reproduced in the new CLI. -- Bare service submission (legacy CLI `--doc foo.pdf` with no extract tasks - and full content-type metadata returned by the service). `retriever online submit` - is currently a stub — only `retriever online stream-pdf` is implemented. -- `gen_dataset.py` dataset creation with enumeration and sampling. -- `--collect_profiling_traces --zipkin_host --zipkin_port`. Use - `--runtime-metrics-dir` / `--runtime-metrics-prefix` instead for a different - metrics flavor. - -## Conventions used in the examples - -- Input paths assume you invoke `retriever` from the `nemo_retriever/` - directory (or point at absolute paths). -- `--save-intermediate ` writes the extraction DataFrame as - `/extraction.parquet` for inspection. LanceDB output goes to `--lancedb-uri` - (defaults to `./lancedb`). -- `--store-images-uri ` stores extracted image assets to a local path or - an fsspec URI (e.g. `s3://bucket/prefix`). Page granularity stores page - images; element granularity stores element images. -- `--run-mode inprocess` skips Ray and is ideal for single-file demos and CI; - `--run-mode batch` (the default) uses Ray Data for throughput. - -Run `retriever pipeline run --help` for the authoritative flag list. +Run `retriever pipeline run --help` for the compatibility command flag list. diff --git a/nemo_retriever/docs/cli/benchmarking.md b/nemo_retriever/docs/cli/benchmarking.md index e2200dae36..61331aa447 100644 --- a/nemo_retriever/docs/cli/benchmarking.md +++ b/nemo_retriever/docs/cli/benchmarking.md @@ -13,46 +13,86 @@ per-stage micro-benchmarks. ## Harness (development / experimental) -Run from the repository root (or any directory; pass `--config` if needed). Uses -`--dataset` and `--preset` against `nemo_retriever/harness/test_configs.yaml`. +Run from the repository root or any directory. The harness uses code-owned +benchmark names from `nemo_retriever.harness.benchmark_registry`; use +`retriever harness list` to discover the available benchmarks and runsets. ```bash -# Named dataset from nemo_retriever/harness/test_configs.yaml -retriever harness run --dataset bo767 --preset PE_GE_OCR_TE_DENSE +# List benchmark registry entries, optionally including runsets +retriever harness list +retriever harness list --runsets -# Default active profile (jp20 + single_gpu in test_configs.yaml) -retriever harness run --dataset jp20 +# Inspect one concrete benchmark spec +retriever harness show jp20_beir -# Custom directory on disk -retriever harness run --dataset /path/to/your/data +# Run one benchmark and write stable artifacts +retriever harness run jp20_beir -# Override a single config key -retriever harness run --dataset bo767 --override run_mode=inprocess +# Run one benchmark in batch mode +retriever harness run bo767_beir --mode batch + +# Override a resolved config key for this run +retriever harness run bo767_beir --set query.top_k=5 + +# Expand and run a code-owned benchmark runset +retriever harness run-set jp20_core ``` Related commands: ```bash -retriever harness --help # run, sweep, nightly, summary, compare, portal +retriever harness --help +retriever harness list --help +retriever harness show --help retriever harness run --help -retriever harness sweep --help -retriever harness nightly --help -retriever harness summary --help -retriever harness compare --help +retriever harness run-set --help +retriever harness diff --help ``` -Sweep and nightly examples: +### Agentic BEIR evaluation + +Harness runs use the standard dense retrieval path unless agentic retrieval is +enabled in the resolved benchmark query config. Set `query.agentic: true` in a +code-owned benchmark or runfile, or use repeatable `--set` overrides on the CLI. +The agentic harness path runs the same ReAct retrieval graph used by root query, +but only after ingest and only for BEIR evaluation (`evaluation.mode: beir`). +`retriever pipeline run` does not expose agentic evaluation flags. + +Minimal BEIR override example: ```bash -retriever harness sweep --runs-config nemo_retriever/harness/nightly_config.yaml -retriever harness nightly --runs-config nemo_retriever/harness/nightly_config.yaml --dry-run +retriever harness run jp20_beir \ + --set query.agentic=true \ + --set query.agentic_llm_model=nvidia/llama-3.3-nemotron-super-49b-v1.5 ``` +Useful agentic query overrides: + +- `query.agentic_llm_model` — chat model used by the ReAct and selection agents; + required when `query.agentic=true`. +- `query.agentic_invoke_url` — OpenAI-compatible chat-completions endpoint. Omit + to use the built-in NVIDIA endpoint. +- `query.agentic_backend_top_k` — backend candidate pool per ReAct retrieval + call. Must be at least the final requested metric depth (`max(evaluation.ks)`). +- `query.agentic_react_max_steps` — maximum ReAct loop iterations per query + (defaults to `50`). +- `query.agentic_text_truncation` — max characters of each candidate shown to + the agent; `0` disables truncation. +- `query.agentic_num_concurrent` — number of queries the agent batch runs + concurrently (defaults to `1`). +- `query.agentic_temperature` — defaults to `0.0`; hosted/default NVIDIA + endpoints are validated as `0.0..1.0`, while other OpenAI-compatible endpoints + allow `0.0..2.0`. +- `query.agentic_reasoning_effort` — optional provider-specific field forwarded + only when configured. + ### Image storage -Image persistence is configured on `retriever pipeline run`, not on the harness. -Use `--store-images-uri ` (local path or fsspec URI). Stored assets follow -`--embed-granularity` (page vs element images). +For normal ingest, configure image persistence on `retriever ingest` with +`--store-images-uri ` (local path or fsspec URI). The harness does not +configure store directly; `retriever pipeline run --store-images-uri ` +remains available for pipeline-specific compatibility workflows. Stored assets +follow `--embed-granularity` (page vs element images). ## Per-stage micro-benchmarks @@ -80,10 +120,11 @@ Each benchmark reports rows/sec (or chunk rows/sec for audio) for its actor. ## Notes -- **Configuration:** `retriever harness` uses `--dataset` / `--preset` / - `--override KEY=VALUE` against - `nemo_retriever/harness/test_configs.yaml`. -- **Launcher:** for internal benchmarking, `retriever harness run …` is the - benchmark orchestration entry point (development / experimental; no guarantees). +- **Configuration:** `retriever harness` uses code-owned benchmarks/runsets from + `nemo_retriever.harness.benchmark_registry`; use `--set KEY=VALUE` for small + per-run config overrides. +- **Launcher:** for internal benchmarking, `retriever harness run BENCHMARK` and + `retriever harness run-set RUNSET` are the benchmark orchestration entry points + (development / experimental; no guarantees). - **Stage benchmarks:** `retriever benchmark …` is specific to the retriever CLI and covers per-stage throughput rather than full harness orchestration.