From 62018a2fddf66c47292f5179b771b4dfd249aa1e Mon Sep 17 00:00:00 2001 From: Kurt Heiss Date: Wed, 1 Jul 2026 08:58:19 -0700 Subject: [PATCH 1/6] docs(extraction): add Customize & extend hub page Add a user-facing customization overview with xrefs to graph, UDF, VDB, and embedding extension guides per doc IA review. --- docs/docs/extraction/concepts.md | 4 +- docs/docs/extraction/customize-extend.md | 71 ++++++++++++++++++++++++ docs/docs/extraction/releasenotes.md | 2 +- docs/docs/extraction/vdbs.md | 3 +- docs/mkdocs.yml | 7 ++- 5 files changed, 81 insertions(+), 6 deletions(-) create mode 100644 docs/docs/extraction/customize-extend.md diff --git a/docs/docs/extraction/concepts.md b/docs/docs/extraction/concepts.md index 4f18ba6c26..8c0f6f96c8 100644 --- a/docs/docs/extraction/concepts.md +++ b/docs/docs/extraction/concepts.md @@ -4,11 +4,11 @@ These terms appear throughout NeMo Retriever Library documentation. ## Job { #job } -An **ingestion job** is a unit of work you run on input content (documents, audio, video, and other supported types). You submit jobs through the **ingestor Python API** (for example `Ingestor` task chains such as `.extract(...)`) or the **`retriever ingest` CLI**—not by posting a standalone JSON job document. Default tasks target strong recall; customize behavior with task keyword arguments (including chunking and splitting on `.extract()`) or custom UDF-style operations ([NeMo Retriever graph](https://github.com/NVIDIA/NeMo-Retriever/tree/main/nemo_retriever/src/nemo_retriever/graph#nemo-retriever-graph)). Results are structured metadata and annotations (Ray Dataset, pandas `DataFrame`, or similar). +An **ingestion job** is a unit of work you run on input content (documents, audio, video, and other supported types). You submit jobs through the **ingestor Python API** (for example `Ingestor` task chains such as `.extract(...)`) or the **`retriever ingest` CLI**—not by posting a standalone JSON job document. Default tasks target strong recall; customize behavior with task keyword arguments (including chunking and splitting on `.extract()`) or custom UDF-style operations (refer to [Customize & extend](customize-extend.md)). Results are structured metadata and annotations (Ray Dataset, pandas `DataFrame`, or similar). ## Pipeline and tasks { #pipeline-and-tasks } -NeMo Retriever Library does **not** run one static pipeline on every document. You configure **tasks** such as parsing, chunking, embedding, storage, and filtering per job. Related topics: [Extending/Customizing NeMo Retriever Library with custom code](https://github.com/NVIDIA/NeMo-Retriever/tree/main/nemo_retriever/src/nemo_retriever/graph#nemo-retriever-graph). +NeMo Retriever Library does **not** run one static pipeline on every document. You configure **tasks** such as parsing, chunking, embedding, storage, and filtering per job. For UDFs, custom graph stages, and other extension paths, refer to [Customize & extend](customize-extend.md). ## Extraction metadata { #extraction-metadata } diff --git a/docs/docs/extraction/customize-extend.md b/docs/docs/extraction/customize-extend.md new file mode 100644 index 0000000000..9a64ff8cc1 --- /dev/null +++ b/docs/docs/extraction/customize-extend.md @@ -0,0 +1,71 @@ +# Customize & extend + +NeMo Retriever Library ships with defaults tuned for strong recall on common document types. When those defaults are not enough, you can extend the library at several levels—from task keyword arguments on the fluent ingestor API through custom graph operators and vector-database adapters. + +Use this page to choose an extension path and find the detailed guides in the repository. + +## On this page { #on-this-page } + +- [Start with task configuration](#start-with-task-configuration) +- [User-defined functions (UDFs)](#user-defined-functions-udfs) +- [Custom graph pipelines](#custom-graph-pipelines) +- [Custom vector databases](#custom-vector-databases) +- [Custom embedding models](#custom-embedding-models) +- [Related topics](#related-topics) + +## Start with task configuration { #start-with-task-configuration } + +Most customization does not require new code. Chain tasks on `create_ingestor(...)` and pass keyword arguments to control extraction, chunking, embedding, and storage—for example `extract_method`, chunking and splitting options on `.extract()`, `embed_modality` on `.embed()`, and `vdb_op` / `vdb_kwargs` on `.vdb_upload()`. + +For parameter details, refer to the [Python API guide](nemo-retriever-api-reference.md). For chunking behavior and pipeline concepts, refer to [Concepts](concepts.md). + +## User-defined functions (UDFs) { #user-defined-functions-udfs } + +A **user-defined function (UDF)** wraps your Python logic as a first-class pipeline stage. In the graph model, `UDFOperator` turns a plain callable into an operator you can chain with built-in stages—for example to normalize HTML, apply a custom split, or call an external service between extract and embed steps. + +Use UDFs when you need a small, self-contained transformation that is not covered by task keyword arguments. + +**Repository guides and examples** + +- [NeMo Retriever graph README — `UDFOperator`](https://github.com/NVIDIA/NeMo-Retriever/tree/main/nemo_retriever/src/nemo_retriever/graph#using-udfoperator) — API, lifecycle, and when to use `UDFOperator` versus a custom operator class +- [UDF example scripts](https://github.com/NVIDIA/NeMo-Retriever/tree/main/examples/udfs) — sample implementations such as HTML-to-Markdown conversion and structural splitting +- [NimClient and custom NIM endpoints](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/developer_docs/nimclient.md#nimclient-and-custom-nim-endpoints) — call custom or self-hosted NIM microservices from UDF stages + +## Custom graph pipelines { #custom-graph-pipelines } + +When you need to compose pipelines stage-by-stage, reuse operators across workflows, or run the same graph in-process or with Ray Data, use the **graph execution model** instead of (or alongside) the fluent `GraphIngestor` API. + +The graph package provides `AbstractOperator`, executors (`InprocessExecutor`, `RayDataExecutor`), and operator chaining with `>>`. Built-in ingestion operators live under `nemo_retriever.operators`; you can add your own operators or UDF stages anywhere in the chain. + +For the full guide—including custom operator classes, executors, and graph shape constraints—refer to the [NeMo Retriever graph README](https://github.com/NVIDIA/NeMo-Retriever/tree/main/nemo_retriever/src/nemo_retriever/graph#nemo-retriever-graph). + +## Custom vector databases { #custom-vector-databases } + +The supported user path for vector storage is **[LanceDB](vdbs.md)** (`vdb_op="lancedb"`). That page covers upload, semantic retrieval, metadata filtering, and LanceDB deployment characteristics. + +To integrate a different vector store, implement the [`VDB`](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/src/nemo_retriever/common/vdb/adt_vdb.py) interface and wire it through graph [`IngestVdbOperator`](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/src/nemo_retriever/operators/vdb.py) / [`RetrieveVdbOperator`](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/src/nemo_retriever/operators/vdb.py). NVIDIA validates the first-party LanceDB operator; you are responsible for testing and maintaining other backends. + +**Repository guides** + +- [Build a custom vector database operator (notebook)](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/building_vdb_operator.ipynb) — step-by-step walkthrough +- [Vector DB package (source)](https://github.com/NVIDIA/NeMo-Retriever/tree/main/nemo_retriever/src/nemo_retriever/common/vdb) — `VDB` abstract base and LanceDB reference implementation + +Partner and blueprint integrations (Elasticsearch, Pinecone, Teradata, and others) are summarized on [Vector databases — Vector database partners](vdbs.md#vector-database-partners). + +## Custom embedding models { #custom-embedding-models } + +!!! note "Coming soon" + + Dedicated documentation for wiring custom embedding models into ingestion graphs is in progress. Today, configure embedding through task parameters and environment variables: + + - [Multimodal embeddings (VLM)](embedding.md) — default and multimodal embed flows + - [Environment variables](environment-config.md) — `*_ENDPOINT` variables for self-hosted or hosted NIM embed services + - [NeMo Retriever Text Embedding NIM](https://docs.nvidia.com/nim/nemo-retriever/text-embedding/latest/overview.html) — OpenAI-compatible text embedding NIM + +## Related topics { #related-topics } + +- [Concepts — Pipeline and tasks](concepts.md#pipeline-and-tasks) +- [Vector databases](vdbs.md) +- [Multimodal embeddings (VLM)](embedding.md) +- [Python API guide](nemo-retriever-api-reference.md) +- [Starter kits and notebooks](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/README.md) diff --git a/docs/docs/extraction/releasenotes.md b/docs/docs/extraction/releasenotes.md index dd1a2b483a..a4b91d6974 100644 --- a/docs/docs/extraction/releasenotes.md +++ b/docs/docs/extraction/releasenotes.md @@ -73,7 +73,7 @@ Highlights for the 26.05 release include: ### Documentation - Documentation aligned to a Helm-first supported path for NIM and service deployment -- Documentation consolidates extraction concepts, ingest workflow, embeddings, audio/video guides, prerequisites and support matrix, and UDF/custom stages in the [graph README](https://github.com/NVIDIA/NeMo-Retriever/tree/26.05/nemo_retriever/src/nemo_retriever/graph#nemo-retriever-graph) +- Documentation consolidates extraction concepts, ingest workflow, embeddings, audio/video guides, prerequisites and support matrix, and customization paths in [Customize & extend](customize-extend.md) ## Release Notes for Previous Versions diff --git a/docs/docs/extraction/vdbs.md b/docs/docs/extraction/vdbs.md index ccc9fb24f3..d1b6e4de1e 100644 --- a/docs/docs/extraction/vdbs.md +++ b/docs/docs/extraction/vdbs.md @@ -150,11 +150,12 @@ Testing and release cadence for these integrations follow the owning project (RA NVIDIA documents and validates the first-party LanceDB operator for this library. If you integrate a different vector store, you are responsible for testing and maintaining that integration. -To implement a custom operator, follow the `VDB` abstract interface described in [Build a Custom Vector Database Operator](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/building_vdb_operator.ipynb). +To implement a custom operator, follow the `VDB` abstract interface described in [Build a Custom Vector Database Operator](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/building_vdb_operator.ipynb). For an overview of all customization paths (UDFs, graph pipelines, and embeddings), refer to [Customize & extend](customize-extend.md). ## Related Topics { #related-topics } - [Metadata and filtering](#metadata-and-filtering) +- [Customize & extend](customize-extend.md) - [Vector DB operators and LanceDB (source)](https://github.com/NVIDIA/NeMo-Retriever/tree/main/nemo_retriever/src/nemo_retriever/common/vdb) - [Use the NeMo Retriever Library Python API](nemo-retriever-api-reference.md) - [Store Extracted Images](nemo-retriever-api-reference.md) diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml index 4e23c7d0a6..57a3ea6d78 100644 --- a/docs/mkdocs.yml +++ b/docs/mkdocs.yml @@ -99,7 +99,7 @@ nav: - "7. Deployment & operations": - "Ray and distributed ingest": extraction/ray-logging.md - "8. Customize & extend": - - Extending/Customizing NeMo Retriever Library with custom code: https://github.com/NVIDIA/NeMo-Retriever/tree/main/nemo_retriever/src/nemo_retriever/graph#nemo-retriever-graph + - "Customize & extend": extraction/customize-extend.md - "9. Integrations & ecosystem": - "Starter kits": extraction/starter-kits.md - "10. Evaluation & benchmarks": @@ -185,7 +185,10 @@ plugins: extraction/chunking.md: extraction/concepts.md#chunking extraction/quickstart-library-mode.md: extraction/deployment-options.md extraction/workflow-video-ocr.md: extraction/audio-video.md - extraction/user-defined-stages.md: https://github.com/NVIDIA/NeMo-Retriever/tree/main/nemo_retriever/src/nemo_retriever/graph#nemo-retriever-graph + extraction/user-defined-stages.md: extraction/customize-extend.md + extraction/user-defined-functions/index.md: extraction/customize-extend.md#user-defined-functions-udfs + extraction/user-defined-functions.md: extraction/customize-extend.md#user-defined-functions-udfs + extraction/customize-and-extend.md: extraction/customize-extend.md extraction/nimclient.md: https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/developer_docs/nimclient.md#nimclient-and-custom-nim-endpoints - site-urls From 011f1d1a5330ebdef29db4166db3eb8621b00dcf Mon Sep 17 00:00:00 2001 From: Kurt Heiss Date: Wed, 1 Jul 2026 09:03:09 -0700 Subject: [PATCH 2/6] docs(extraction): polish Customize & extend after ::a/::p/::r Apply audit fixes: decision table, heading consistency, in-tree starter-kits link, and revert inaccurate 26.05 release-notes claim. --- docs/docs/extraction/concepts.md | 2 +- docs/docs/extraction/customize-extend.md | 20 +++++++++++++++----- docs/docs/extraction/releasenotes.md | 2 +- 3 files changed, 17 insertions(+), 7 deletions(-) diff --git a/docs/docs/extraction/concepts.md b/docs/docs/extraction/concepts.md index 8c0f6f96c8..bb6ce80998 100644 --- a/docs/docs/extraction/concepts.md +++ b/docs/docs/extraction/concepts.md @@ -4,7 +4,7 @@ These terms appear throughout NeMo Retriever Library documentation. ## Job { #job } -An **ingestion job** is a unit of work you run on input content (documents, audio, video, and other supported types). You submit jobs through the **ingestor Python API** (for example `Ingestor` task chains such as `.extract(...)`) or the **`retriever ingest` CLI**—not by posting a standalone JSON job document. Default tasks target strong recall; customize behavior with task keyword arguments (including chunking and splitting on `.extract()`) or custom UDF-style operations (refer to [Customize & extend](customize-extend.md)). Results are structured metadata and annotations (Ray Dataset, pandas `DataFrame`, or similar). +An **ingestion job** is a unit of work you run on input content (documents, audio, video, and other supported types). You submit jobs through the **ingestor Python API** (for example `Ingestor` task chains such as `.extract(...)`) or the **`retriever ingest` CLI**—not by posting a standalone JSON job document. Default tasks target strong recall; customize behavior with task keyword arguments (including chunking and splitting on `.extract()`) or custom UDF-style operations. For UDFs and other extension paths, refer to [Customize & extend](customize-extend.md). Results are structured metadata and annotations (Ray Dataset, pandas `DataFrame`, or similar). ## Pipeline and tasks { #pipeline-and-tasks } diff --git a/docs/docs/extraction/customize-extend.md b/docs/docs/extraction/customize-extend.md index 9a64ff8cc1..48b1ae866d 100644 --- a/docs/docs/extraction/customize-extend.md +++ b/docs/docs/extraction/customize-extend.md @@ -4,6 +4,16 @@ NeMo Retriever Library ships with defaults tuned for strong recall on common doc Use this page to choose an extension path and find the detailed guides in the repository. +The following table maps common needs to the right section: + +| If you need to… | Start here | +|-----------------|------------| +| Tune extraction, chunking, embedding, or upload without new code | [Start with task configuration](#start-with-task-configuration) | +| Add a small Python transformation between pipeline stages | [User-defined functions (UDFs)](#user-defined-functions-udfs) | +| Build or reuse operators stage-by-stage | [Custom graph pipelines](#custom-graph-pipelines) | +| Store vectors in a backend other than LanceDB | [Custom vector databases](#custom-vector-databases) | +| Wire a non-default embedding model | [Custom embedding models](#custom-embedding-models) | + ## On this page { #on-this-page } - [Start with task configuration](#start-with-task-configuration) @@ -11,7 +21,7 @@ Use this page to choose an extension path and find the detailed guides in the re - [Custom graph pipelines](#custom-graph-pipelines) - [Custom vector databases](#custom-vector-databases) - [Custom embedding models](#custom-embedding-models) -- [Related topics](#related-topics) +- [Related Topics](#related-topics) ## Start with task configuration { #start-with-task-configuration } @@ -25,7 +35,7 @@ A **user-defined function (UDF)** wraps your Python logic as a first-class pipel Use UDFs when you need a small, self-contained transformation that is not covered by task keyword arguments. -**Repository guides and examples** +### Repository guides and examples - [NeMo Retriever graph README — `UDFOperator`](https://github.com/NVIDIA/NeMo-Retriever/tree/main/nemo_retriever/src/nemo_retriever/graph#using-udfoperator) — API, lifecycle, and when to use `UDFOperator` versus a custom operator class - [UDF example scripts](https://github.com/NVIDIA/NeMo-Retriever/tree/main/examples/udfs) — sample implementations such as HTML-to-Markdown conversion and structural splitting @@ -45,7 +55,7 @@ The supported user path for vector storage is **[LanceDB](vdbs.md)** (`vdb_op="l To integrate a different vector store, implement the [`VDB`](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/src/nemo_retriever/common/vdb/adt_vdb.py) interface and wire it through graph [`IngestVdbOperator`](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/src/nemo_retriever/operators/vdb.py) / [`RetrieveVdbOperator`](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/src/nemo_retriever/operators/vdb.py). NVIDIA validates the first-party LanceDB operator; you are responsible for testing and maintaining other backends. -**Repository guides** +### Repository guides - [Build a custom vector database operator (notebook)](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/building_vdb_operator.ipynb) — step-by-step walkthrough - [Vector DB package (source)](https://github.com/NVIDIA/NeMo-Retriever/tree/main/nemo_retriever/src/nemo_retriever/common/vdb) — `VDB` abstract base and LanceDB reference implementation @@ -62,10 +72,10 @@ Partner and blueprint integrations (Elasticsearch, Pinecone, Teradata, and other - [Environment variables](environment-config.md) — `*_ENDPOINT` variables for self-hosted or hosted NIM embed services - [NeMo Retriever Text Embedding NIM](https://docs.nvidia.com/nim/nemo-retriever/text-embedding/latest/overview.html) — OpenAI-compatible text embedding NIM -## Related topics { #related-topics } +## Related Topics { #related-topics } - [Concepts — Pipeline and tasks](concepts.md#pipeline-and-tasks) - [Vector databases](vdbs.md) - [Multimodal embeddings (VLM)](embedding.md) - [Python API guide](nemo-retriever-api-reference.md) -- [Starter kits and notebooks](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/README.md) +- [Starter kits and notebooks](starter-kits.md) diff --git a/docs/docs/extraction/releasenotes.md b/docs/docs/extraction/releasenotes.md index a4b91d6974..15c7814664 100644 --- a/docs/docs/extraction/releasenotes.md +++ b/docs/docs/extraction/releasenotes.md @@ -73,7 +73,7 @@ Highlights for the 26.05 release include: ### Documentation - Documentation aligned to a Helm-first supported path for NIM and service deployment -- Documentation consolidates extraction concepts, ingest workflow, embeddings, audio/video guides, prerequisites and support matrix, and customization paths in [Customize & extend](customize-extend.md) +- Documentation consolidates extraction concepts, ingest workflow, embeddings, audio/video guides, prerequisites and support matrix, and UDF/custom stages in the [graph README](https://github.com/NVIDIA/NeMo-Retriever/tree/26.05/nemo_retriever/src/nemo_retriever/graph#nemo-retriever-graph) ## Release Notes for Previous Versions From c2025dd1712318199280e4ffb38b821251e0115b Mon Sep 17 00:00:00 2001 From: Kurt Heiss Date: Wed, 1 Jul 2026 09:03:32 -0700 Subject: [PATCH 3/6] docs(extraction): drop incidental releasenotes whitespace from PR scope --- docs/docs/extraction/releasenotes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/extraction/releasenotes.md b/docs/docs/extraction/releasenotes.md index 15c7814664..dd1a2b483a 100644 --- a/docs/docs/extraction/releasenotes.md +++ b/docs/docs/extraction/releasenotes.md @@ -73,7 +73,7 @@ Highlights for the 26.05 release include: ### Documentation - Documentation aligned to a Helm-first supported path for NIM and service deployment -- Documentation consolidates extraction concepts, ingest workflow, embeddings, audio/video guides, prerequisites and support matrix, and UDF/custom stages in the [graph README](https://github.com/NVIDIA/NeMo-Retriever/tree/26.05/nemo_retriever/src/nemo_retriever/graph#nemo-retriever-graph) +- Documentation consolidates extraction concepts, ingest workflow, embeddings, audio/video guides, prerequisites and support matrix, and UDF/custom stages in the [graph README](https://github.com/NVIDIA/NeMo-Retriever/tree/26.05/nemo_retriever/src/nemo_retriever/graph#nemo-retriever-graph) ## Release Notes for Previous Versions From 62dad0348414d2aa64f3b70ca1e399c36ce64b28 Mon Sep 17 00:00:00 2001 From: Kurt Heiss Date: Wed, 1 Jul 2026 13:27:03 -0700 Subject: [PATCH 4/6] Apply suggestion from @randerzander Co-authored-by: Randy Gelhausen From bf1ec76357a7d34d1ca582465de5548b66fc2eb8 Mon Sep 17 00:00:00 2001 From: Kurt Heiss Date: Wed, 1 Jul 2026 13:36:08 -0700 Subject: [PATCH 5/6] docs(extraction): remove placeholder Custom embedding models section --- docs/docs/extraction/customize-extend.md | 12 ------------ 1 file changed, 12 deletions(-) diff --git a/docs/docs/extraction/customize-extend.md b/docs/docs/extraction/customize-extend.md index 48b1ae866d..e7f3ab6981 100644 --- a/docs/docs/extraction/customize-extend.md +++ b/docs/docs/extraction/customize-extend.md @@ -12,7 +12,6 @@ The following table maps common needs to the right section: | Add a small Python transformation between pipeline stages | [User-defined functions (UDFs)](#user-defined-functions-udfs) | | Build or reuse operators stage-by-stage | [Custom graph pipelines](#custom-graph-pipelines) | | Store vectors in a backend other than LanceDB | [Custom vector databases](#custom-vector-databases) | -| Wire a non-default embedding model | [Custom embedding models](#custom-embedding-models) | ## On this page { #on-this-page } @@ -20,7 +19,6 @@ The following table maps common needs to the right section: - [User-defined functions (UDFs)](#user-defined-functions-udfs) - [Custom graph pipelines](#custom-graph-pipelines) - [Custom vector databases](#custom-vector-databases) -- [Custom embedding models](#custom-embedding-models) - [Related Topics](#related-topics) ## Start with task configuration { #start-with-task-configuration } @@ -62,16 +60,6 @@ To integrate a different vector store, implement the [`VDB`](https://github.com/ Partner and blueprint integrations (Elasticsearch, Pinecone, Teradata, and others) are summarized on [Vector databases — Vector database partners](vdbs.md#vector-database-partners). -## Custom embedding models { #custom-embedding-models } - -!!! note "Coming soon" - - Dedicated documentation for wiring custom embedding models into ingestion graphs is in progress. Today, configure embedding through task parameters and environment variables: - - - [Multimodal embeddings (VLM)](embedding.md) — default and multimodal embed flows - - [Environment variables](environment-config.md) — `*_ENDPOINT` variables for self-hosted or hosted NIM embed services - - [NeMo Retriever Text Embedding NIM](https://docs.nvidia.com/nim/nemo-retriever/text-embedding/latest/overview.html) — OpenAI-compatible text embedding NIM - ## Related Topics { #related-topics } - [Concepts — Pipeline and tasks](concepts.md#pipeline-and-tasks) From 7ce4d6bebf07b8cc04a7cb6e1d087e1159ef0757 Mon Sep 17 00:00:00 2001 From: Kurt Heiss Date: Thu, 2 Jul 2026 08:01:55 -0700 Subject: [PATCH 6/6] docs(extraction): drop outdated notebook links from customize-extend Per review: remove example notebook and starter-kit links until examples are refreshed; keep graph README and source-doc references. --- docs/docs/extraction/customize-extend.md | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/docs/docs/extraction/customize-extend.md b/docs/docs/extraction/customize-extend.md index e7f3ab6981..31fb87e7bc 100644 --- a/docs/docs/extraction/customize-extend.md +++ b/docs/docs/extraction/customize-extend.md @@ -33,10 +33,9 @@ A **user-defined function (UDF)** wraps your Python logic as a first-class pipel Use UDFs when you need a small, self-contained transformation that is not covered by task keyword arguments. -### Repository guides and examples +### Repository guides - [NeMo Retriever graph README — `UDFOperator`](https://github.com/NVIDIA/NeMo-Retriever/tree/main/nemo_retriever/src/nemo_retriever/graph#using-udfoperator) — API, lifecycle, and when to use `UDFOperator` versus a custom operator class -- [UDF example scripts](https://github.com/NVIDIA/NeMo-Retriever/tree/main/examples/udfs) — sample implementations such as HTML-to-Markdown conversion and structural splitting - [NimClient and custom NIM endpoints](https://github.com/NVIDIA/NeMo-Retriever/blob/main/nemo_retriever/developer_docs/nimclient.md#nimclient-and-custom-nim-endpoints) — call custom or self-hosted NIM microservices from UDF stages ## Custom graph pipelines { #custom-graph-pipelines } @@ -55,7 +54,6 @@ To integrate a different vector store, implement the [`VDB`](https://github.com/ ### Repository guides -- [Build a custom vector database operator (notebook)](https://github.com/NVIDIA/NeMo-Retriever/blob/main/examples/building_vdb_operator.ipynb) — step-by-step walkthrough - [Vector DB package (source)](https://github.com/NVIDIA/NeMo-Retriever/tree/main/nemo_retriever/src/nemo_retriever/common/vdb) — `VDB` abstract base and LanceDB reference implementation Partner and blueprint integrations (Elasticsearch, Pinecone, Teradata, and others) are summarized on [Vector databases — Vector database partners](vdbs.md#vector-database-partners). @@ -66,4 +64,3 @@ Partner and blueprint integrations (Elasticsearch, Pinecone, Teradata, and other - [Vector databases](vdbs.md) - [Multimodal embeddings (VLM)](embedding.md) - [Python API guide](nemo-retriever-api-reference.md) -- [Starter kits and notebooks](starter-kits.md)