Skip to content

docs(26.05): split Julio Helm and VDB reconcile updates #2316

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
kheiss-uwzoo wants to merge 1 commit into from
Closed

docs(26.05): split Julio Helm and VDB reconcile updates #2316

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 12 additions & 14 deletions docs/docs/extraction/vdbs.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ Pass `vdb_op="lancedb"` to `vdb_upload`, or construct a `LanceDB` instance and p
For parameter details, refer to the [Python API guide](nemo-retriever-api-reference.md).

```python
from nemo_retriever.vdb.lancedb import LanceDB
from nemo_retriever.common.vdb.lancedb import LanceDB

@greptile-apps greptile-apps Bot Jul 7, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 LanceDB Import Path Missing

When readers copy this example on the 26.05 branch, nemo_retriever.common.vdb.lancedb is not importable because the package still exposes LanceDB from nemo_retriever.vdb.lancedb. The snippet fails before users can create the LanceDB instance.

Suggested change
from nemo_retriever.common.vdb.lancedb import LanceDB
from nemo_retriever.vdb.lancedb import LanceDB
Prompt To Fix With AI 
This is a comment left during a code review.
Path: docs/docs/extraction/vdbs.md
Line: 63

Comment:
**LanceDB Import Path Missing**

When readers copy this example on the 26.05 branch, `nemo_retriever.common.vdb.lancedb` is not importable because the package still exposes `LanceDB` from `nemo_retriever.vdb.lancedb`. The snippet fails before users can create the LanceDB instance.

```suggestion
from nemo_retriever.vdb.lancedb import LanceDB
```

How can I resolve this? If you propose a fix, please make it concise.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!


vdb = LanceDB(
uri="./lancedb_data", # Path to LanceDB database directory
Expand All @@ -83,18 +83,15 @@ When using the `Ingestor` with `vdb_upload`, pass `vdb_op="lancedb"` or a `Lance

Semantic retrieval uses dense embeddings to find content that is similar in meaning to a query. In NeMo Retriever Library, the default vector path is LanceDB. Use these resources together with the sections on this page:

- [Metadata and filtering](#metadata-and-filtering) for sidecar metadata at ingest and query-time filters
- [Metadata and filtering](#metadata-and-filtering) for custom metadata at ingest and filtered retrieval
- [Concepts](concepts.md) for broader pipeline and search patterns
- [Use the NeMo Retriever Library Python API](nemo-retriever-api-reference.md) for `Retriever.query` and `LanceDB.retrieval` parameters

**Evaluation** — For evaluation and metrics, refer to [Evaluate on your data](evaluate-on-your-data.md).

## Metadata and filtering { #metadata-and-filtering }

This page covers LanceDB upload and retrieval. **Metadata is not duplicated here.**

- **Published guide** — [Custom metadata and filtering](custom-metadata.md) (sidecar `meta_*` on `vdb_upload`, compact JSON in LanceDB, server-side `where` on `Retriever.query`, and client-side `filter_hits_by_content_metadata`).
- **Canonical reference** — [Vector DB operators and LanceDB — Metadata filtering](https://github.com/NVIDIA/NeMo-Retriever/tree/main/nemo_retriever/src/nemo_retriever/vdb#metadata-filtering) in `nemo_retriever/src/nemo_retriever/vdb/README.md` (operator behavior and examples).
Refer to the [metadata filtering notebook](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/nemo_retriever_retriever_query_metadata_filter.ipynb) for an end-to-end example of adding custom metadata fields to your documents and filtering retrieval results with that metadata.

## LanceDB deployment characteristics { #lancedb-deployment-characteristics }

Expand All @@ -120,19 +117,19 @@ NeMo Retriever Library integrates with vector databases used for RAG collections

### Backends with `VDB` implementations (retriever adapters) { #vdb-backends-implementations }

NeMo Retriever graph operators [`IngestVdbOperator`](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/src/nemo_retriever/vdb/operators.py) and [`RetrieveVdbOperator`](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/src/nemo_retriever/vdb/operators.py) wrap concrete classes that implement the [`VDB`](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/src/nemo_retriever/vdb/adt_vdb.py) interface (`run` for ingest, `retrieval` for search). The library ships one first-party backend:
NeMo Retriever graph operators [`IngestVdbOperator`](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/src/nemo_retriever/operators/vdb.py) and [`RetrieveVdbOperator`](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/src/nemo_retriever/operators/vdb.py) wrap concrete classes that implement the [`VDB`](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/src/nemo_retriever/common/vdb/adt_vdb.py) interface (`run` for ingest, `retrieval` for search). The library ships one first-party backend:

@greptile-apps greptile-apps Bot Jul 7, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Source Links Target Missing Paths

The new links on this line send readers to nemo_retriever/src/nemo_retriever/operators/vdb.py and nemo_retriever/src/nemo_retriever/common/vdb/adt_vdb.py, but those paths are not present on this branch. Users following the custom VDB guidance will hit 404s instead of the operator and interface sources under nemo_retriever/src/nemo_retriever/vdb/.

Prompt To Fix With AI 
This is a comment left during a code review.
Path: docs/docs/extraction/vdbs.md
Line: 120

Comment:
**Source Links Target Missing Paths**

The new links on this line send readers to `nemo_retriever/src/nemo_retriever/operators/vdb.py` and `nemo_retriever/src/nemo_retriever/common/vdb/adt_vdb.py`, but those paths are not present on this branch. Users following the custom VDB guidance will hit 404s instead of the operator and interface sources under `nemo_retriever/src/nemo_retriever/vdb/`.

How can I resolve this? If you propose a fix, please make it concise.


| Backend | Project | Implementation |
|---------|---------|----------------|
| **LanceDB** | [LanceDB](https://lancedb.com/) · [documentation](https://lancedb.github.io/lancedb/) | [`lancedb.py`](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/src/nemo_retriever/vdb/lancedb.py) — pass `vdb_op="lancedb"` (recommended). |
| **LanceDB** | [LanceDB](https://lancedb.com/) · [documentation](https://lancedb.github.io/lancedb/) | [`lancedb.py`](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/src/nemo_retriever/common/vdb/lancedb.py) — pass `vdb_op="lancedb"` (recommended). |

On `GraphIngestor.vdb_upload`, omitting `vdb_op` does not select LanceDB; refer to [Upload to LanceDB](#upload-to-lancedb).

Pass `vdb_op="lancedb"` or a `LanceDB` instance. To integrate another vector database, subclass [`VDB`](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/src/nemo_retriever/vdb/adt_vdb.py) and pass your operator instance as `vdb` (refer to [Build a Custom Vector Database Operator](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/building_vdb_operator.ipynb)).
Pass `vdb_op="lancedb"` or a `LanceDB` instance. To integrate another vector database, subclass [`VDB`](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/src/nemo_retriever/common/vdb/adt_vdb.py) and pass your operator instance as `vdb` (refer to [Build a Custom Vector Database Operator](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/building_vdb_operator.ipynb)).

### RAG Blueprint and partner vector stores { #rag-blueprint-and-partner-vector-stores }

Some deployments use a different vector store than the default LanceDB path on this page—for example the [NVIDIA RAG Blueprint](https://docs.nvidia.com/rag/latest/index.html) (Docker Compose or Helm) or a partner package that subclasses the same [`VDB`](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/src/nemo_retriever/vdb/adt_vdb.py) interface. Use the following public references when you wire those stacks to ingestion and retrieval:
Some deployments use a different vector store than the default LanceDB path on this page—for example the [NVIDIA RAG Blueprint](https://docs.nvidia.com/rag/latest/index.html) (Docker Compose or Helm) or a partner package that subclasses the same [`VDB`](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/src/nemo_retriever/common/vdb/adt_vdb.py) interface. Use the following public references when you wire those stacks to ingestion and retrieval:

| Vector store | Where to configure or implement |
|--------------|--------------------------------|
Expand All @@ -144,7 +141,7 @@ Testing and release cadence for these integrations follow the owning project (RA

### More information (embeddings & custom `VDB`) { #vector-database-partners-more-info }

- [Custom metadata and filtering](custom-metadata.md) and the package [VDB README (metadata filtering)](https://github.com/NVIDIA/NeMo-Retriever/tree/main/nemo_retriever/src/nemo_retriever/vdb#metadata-filtering)
- [Metadata filtering notebook](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/nemo_retriever_retriever_query_metadata_filter.ipynb) and the package [VDB README (metadata filtering)](https://github.com/NVIDIA/NeMo-Retriever/tree/main/nemo_retriever/src/nemo_retriever/common/vdb#metadata-filtering)
- [Multimodal embeddings (VLM)](embedding.md)
- [NeMo Retriever Text Embedding NIM](https://docs.nvidia.com/nim/nemo-retriever/text-embedding/latest/overview.html)
- [NVIDIA NIM catalog](https://build.nvidia.com/) for embedding and retrieval-related NIMs
Expand All @@ -153,12 +150,13 @@ Testing and release cadence for these integrations follow the owning project (RA

NVIDIA documents and validates the first-party LanceDB operator for this library. If you integrate a different vector store, you are responsible for testing and maintaining that integration.

To implement a custom operator, follow the `VDB` abstract interface described in [Build a Custom Vector Database Operator](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/building_vdb_operator.ipynb).
To implement a custom operator, follow the `VDB` abstract interface described in [Build a Custom Vector Database Operator](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/building_vdb_operator.ipynb). For an overview of all customization paths (UDFs, graph pipelines, and embeddings), refer to [Customize & extend](customize-extend.md).

@greptile-apps greptile-apps Bot Jul 7, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Related Topic Link Missing

This adds a relative link to customize-extend.md, but that page is not present under docs/docs/extraction/ on this branch. Readers using the Related Topics section will land on a broken docs page instead of customization guidance.

Prompt To Fix With AI 
This is a comment left during a code review.
Path: docs/docs/extraction/vdbs.md
Line: 153

Comment:
**Related Topic Link Missing**

This adds a relative link to `customize-extend.md`, but that page is not present under `docs/docs/extraction/` on this branch. Readers using the Related Topics section will land on a broken docs page instead of customization guidance.

How can I resolve this? If you propose a fix, please make it concise.


## Related Topics { #related-topics }

- [Custom metadata and filtering](custom-metadata.md)
- [Vector DB operators and LanceDB (source)](https://github.com/NVIDIA/NeMo-Retriever/tree/main/nemo_retriever/src/nemo_retriever/vdb)
- [Metadata and filtering](#metadata-and-filtering)
- [Customize & extend](customize-extend.md)
- [Vector DB operators and LanceDB (source)](https://github.com/NVIDIA/NeMo-Retriever/tree/main/nemo_retriever/src/nemo_retriever/common/vdb)
- [Use the NeMo Retriever Library Python API](nemo-retriever-api-reference.md)
- [Store Extracted Images](nemo-retriever-api-reference.md)
- [Environment Variables](environment-config.md)
Expand Down
Loading
Loading