From c613e02251ec6837adbfe41a0154a6f05ecc8a50 Mon Sep 17 00:00:00 2001 From: Mahika Wason Date: Fri, 26 Jun 2026 16:42:03 +0000 Subject: [PATCH 1/3] docs: add agentic retrieval README examples Document CLI and Python examples for agentic retrieval with hosted NVIDIA endpoints so users can smoke test against an existing LanceDB index. --- nemo_retriever/README.md | 78 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 78 insertions(+) diff --git a/nemo_retriever/README.md b/nemo_retriever/README.md index 60bb1973d7..d5894d4208 100644 --- a/nemo_retriever/README.md +++ b/nemo_retriever/README.md @@ -309,6 +309,84 @@ Answer: Cat is the animal whose activity (jumping onto a laptop) matches the location of the typos, so the cat is responsible for the typos in the documents. ``` +### Run agentic retrieval + +Agentic retrieval runs an LLM-driven ReAct loop over an existing LanceDB index. +It does not ingest documents; first build the index with one of the ingestion +flows above, then query the same `lancedb_uri`, `table_name`, and embedding +model. + +For [build.nvidia.com](https://build.nvidia.com/) hosted inference, set +`NVIDIA_API_KEY`. The hosted API uses `integrate.api.nvidia.com` endpoints: + +```bash +export NVIDIA_API_KEY=nvapi-... + +retriever query "Given their activities, which animal is responsible for the typos in my documents?" \ + --agentic \ + --agentic-llm-model nvidia/llama-3.3-nemotron-super-49b-v1.5 \ + --agentic-invoke-url https://integrate.api.nvidia.com/v1/chat/completions \ + --lancedb-uri lancedb \ + --table-name nemo-retriever \ + --embed-invoke-url https://integrate.api.nvidia.com/v1/embeddings \ + --embed-model-name nvidia/llama-nemotron-embed-1b-v2 +``` + +The `--agentic-invoke-url` option may be omitted to use the built-in NVIDIA +hosted chat-completions endpoint. Keep `--embed-invoke-url` explicit when you +want hosted embedding behavior on any machine. On CPU-only machines, embedding +actors resolve to CPU/remote implementations and default to hosted endpoints; +on GPU-capable machines, embedding prefers the local GPU implementation unless +an endpoint URL is provided. + +For a quick smoke test, lower the amount of agent work: + +```bash +retriever query "What is RAG?" \ + --agentic \ + --agentic-llm-model nvidia/llama-3.3-nemotron-super-49b-v1.5 \ + --lancedb-uri lancedb \ + --table-name nemo-retriever \ + --embed-invoke-url https://integrate.api.nvidia.com/v1/embeddings \ + --embed-model-name nvidia/llama-nemotron-embed-1b-v2 \ + --top-k 1 \ + --agentic-react-max-steps 1 \ + --agentic-backend-top-k 1 +``` + +The same flow is available from Python: + +```python +from nemo_retriever.cli.query_workflow import agentic_query_documents +from nemo_retriever.query.options import ( + QueryAgenticOptions, + QueryEmbedOptions, + QueryRequest, + QueryRetrievalOptions, + QueryStorageOptions, +) + +results = agentic_query_documents( + QueryRequest( + query="What is RAG?", + retrieval=QueryRetrievalOptions(top_k=10), + storage=QueryStorageOptions( + lancedb_uri="lancedb", + table_name="nemo-retriever", + ), + embed=QueryEmbedOptions( + embed_invoke_url="https://integrate.api.nvidia.com/v1/embeddings", + embed_model_name="nvidia/llama-nemotron-embed-1b-v2", + ), + agentic=QueryAgenticOptions( + enabled=True, + llm_model="nvidia/llama-3.3-nemotron-super-49b-v1.5", + invoke_url="https://integrate.api.nvidia.com/v1/chat/completions", + ), + ) +) +``` + ### Live RAG SDK (retrieve + answer in one call) The pattern above -- retrieve hits, build a prompt, call an LLM -- is baked into the SDK as `Retriever.answer()` so live applications can skip the boilerplate. The same `Retriever` instance powers three entry points: From 5227e5a959a8ef04915e5f14b668963c84b5ea4d Mon Sep 17 00:00:00 2001 From: Mahika Wason Date: Fri, 26 Jun 2026 16:47:17 +0000 Subject: [PATCH 2/3] docs: note API key for agentic Python example --- nemo_retriever/README.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/nemo_retriever/README.md b/nemo_retriever/README.md index d5894d4208..0bc7a684d0 100644 --- a/nemo_retriever/README.md +++ b/nemo_retriever/README.md @@ -354,7 +354,9 @@ retriever query "What is RAG?" \ --agentic-backend-top-k 1 ``` -The same flow is available from Python: +The same flow is available from Python. It uses the same `NVIDIA_API_KEY` +environment variable shown above for hosted embedding and chat-completions +requests. ```python from nemo_retriever.cli.query_workflow import agentic_query_documents @@ -366,6 +368,7 @@ from nemo_retriever.query.options import ( QueryStorageOptions, ) +# Requires NVIDIA_API_KEY=nvapi-... in the environment. results = agentic_query_documents( QueryRequest( query="What is RAG?", From ba0d39c18ef9cddfdb823afaec203dcd8b9effef Mon Sep 17 00:00:00 2001 From: Mahika Wason Date: Fri, 26 Jun 2026 16:54:48 +0000 Subject: [PATCH 3/3] docs: clarify agentic endpoint defaults --- nemo_retriever/README.md | 20 ++++++++------------ 1 file changed, 8 insertions(+), 12 deletions(-) diff --git a/nemo_retriever/README.md b/nemo_retriever/README.md index 0bc7a684d0..b9dcb5db43 100644 --- a/nemo_retriever/README.md +++ b/nemo_retriever/README.md @@ -317,7 +317,8 @@ flows above, then query the same `lancedb_uri`, `table_name`, and embedding model. For [build.nvidia.com](https://build.nvidia.com/) hosted inference, set -`NVIDIA_API_KEY`. The hosted API uses `integrate.api.nvidia.com` endpoints: +`NVIDIA_API_KEY`. On CPU-only machines, the CPU embedding actor and agent LLM +use the hosted NVIDIA endpoints by default: ```bash export NVIDIA_API_KEY=nvapi-... @@ -325,19 +326,17 @@ export NVIDIA_API_KEY=nvapi-... retriever query "Given their activities, which animal is responsible for the typos in my documents?" \ --agentic \ --agentic-llm-model nvidia/llama-3.3-nemotron-super-49b-v1.5 \ - --agentic-invoke-url https://integrate.api.nvidia.com/v1/chat/completions \ --lancedb-uri lancedb \ --table-name nemo-retriever \ - --embed-invoke-url https://integrate.api.nvidia.com/v1/embeddings \ --embed-model-name nvidia/llama-nemotron-embed-1b-v2 ``` -The `--agentic-invoke-url` option may be omitted to use the built-in NVIDIA -hosted chat-completions endpoint. Keep `--embed-invoke-url` explicit when you -want hosted embedding behavior on any machine. On CPU-only machines, embedding -actors resolve to CPU/remote implementations and default to hosted endpoints; -on GPU-capable machines, embedding prefers the local GPU implementation unless -an endpoint URL is provided. +The agentic LLM uses the built-in NVIDIA hosted chat-completions endpoint when +`--agentic-invoke-url` is omitted. On CPU-only machines, embedding actors also +resolve to CPU/remote implementations and default to hosted endpoints. On +GPU-capable machines, embedding prefers the local GPU implementation unless an +endpoint URL, such as `--embed-invoke-url https://integrate.api.nvidia.com/v1/embeddings`, +is provided. For a quick smoke test, lower the amount of agent work: @@ -347,7 +346,6 @@ retriever query "What is RAG?" \ --agentic-llm-model nvidia/llama-3.3-nemotron-super-49b-v1.5 \ --lancedb-uri lancedb \ --table-name nemo-retriever \ - --embed-invoke-url https://integrate.api.nvidia.com/v1/embeddings \ --embed-model-name nvidia/llama-nemotron-embed-1b-v2 \ --top-k 1 \ --agentic-react-max-steps 1 \ @@ -378,13 +376,11 @@ results = agentic_query_documents( table_name="nemo-retriever", ), embed=QueryEmbedOptions( - embed_invoke_url="https://integrate.api.nvidia.com/v1/embeddings", embed_model_name="nvidia/llama-nemotron-embed-1b-v2", ), agentic=QueryAgenticOptions( enabled=True, llm_model="nvidia/llama-3.3-nemotron-super-49b-v1.5", - invoke_url="https://integrate.api.nvidia.com/v1/chat/completions", ), ) )