From 9316caa14d5be695f457999d5ff11fa76e180a1a Mon Sep 17 00:00:00 2001 From: tgasser-nv <200644301+tgasser-nv@users.noreply.github.com> Date: Wed, 28 Oct 2026 21:14:29 -0500 Subject: [PATCH 1/7] Initial pass of Relay client docs --- crates/nemo-guardrails-relay-plugin/README.md | 2 + docs/index.yml | 5 +- docs/run-rails/using-hooks-server/index.mdx | 9 +- .../using-hooks-server/nemo-relay-plugin.mdx | 331 ++++++++++++++++++ .../using-hooks-server/nemo-relay.mdx | 3 +- .../run-rails/using-hooks-server/overview.mdx | 2 +- .../using-hooks-server/run-hooks-server.mdx | 2 +- 7 files changed, 349 insertions(+), 5 deletions(-) create mode 100644 docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx diff --git a/crates/nemo-guardrails-relay-plugin/README.md b/crates/nemo-guardrails-relay-plugin/README.md index b469e92f44..e87af968f1 100644 --- a/crates/nemo-guardrails-relay-plugin/README.md +++ b/crates/nemo-guardrails-relay-plugin/README.md @@ -8,6 +8,8 @@ Before each tool call that Relay manages, the plugin posts the tool's name and a A deny from the service becomes a guardrail rejection, and a check that fails (an error status, a timeout, a service that can't be reached) becomes an error. Relay blocks the call on either, from 0.10.1 on, with one exception: on an error, pi's extension lets the call run unless `NEMO_RELAY_PI_FAIL=closed` is set. +To install and configure the plugin, refer to [Check NeMo Relay Tool Calls with the NVIDIA NeMo Guardrails Plugin](./../../docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx). + ## Checks The plugin registers one tool conditional-execution guardrail, `nemo_guardrails_tool_call_check`, at the configured `priority`. diff --git a/docs/index.yml b/docs/index.yml index 5f2a1c7298..b92e1ad945 100644 --- a/docs/index.yml +++ b/docs/index.yml @@ -545,7 +545,10 @@ navigation: - page: Codex path: run-rails/using-hooks-server/codex.mdx slug: codex - - page: NeMo Relay + - page: NeMo Relay Plugin + path: run-rails/using-hooks-server/nemo-relay-plugin.mdx + slug: nemo-relay-plugin + - page: NeMo Relay Endpoint path: run-rails/using-hooks-server/nemo-relay.mdx slug: nemo-relay - page: Results and Errors diff --git a/docs/run-rails/using-hooks-server/index.mdx b/docs/run-rails/using-hooks-server/index.mdx index 11d2dcbdba..3c50f46cd8 100644 --- a/docs/run-rails/using-hooks-server/index.mdx +++ b/docs/run-rails/using-hooks-server/index.mdx @@ -43,7 +43,14 @@ Send every Codex tool call to the hooks server with a command hook. How To - + + +Check every tool call NeMo Relay manages with the NVIDIA NeMo Guardrails Plugin. + +How To + + + The endpoint a NeMo Relay plugin's tool guardrail calls. diff --git a/docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx b/docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx new file mode 100644 index 0000000000..107701dfb3 --- /dev/null +++ b/docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx @@ -0,0 +1,331 @@ +--- +title: "Check NeMo Relay Tool Calls with the NVIDIA NeMo Guardrails Plugin" +sidebar-title: "NeMo Relay Plugin" +description: "Install the NVIDIA NeMo Guardrails Plugin for NeMo Relay, which checks each tool call Relay manages with the Guardrails hooks server before the tool runs." +keywords: ["NeMo Relay", "NeMo Relay plugin", "nvidia.nemo_guardrails", "nemo-relay plugins add", "plugins.toml", "nemoguardrails hooks-server", "/v1/hooks/nemo-relay", "NemoRelayMiddleware"] +content: + type: "how_to" +--- + +The NVIDIA NeMo Guardrails Plugin for NeMo Relay, `nvidia.nemo_guardrails`, is a native [NeMo Relay](https://github.com/NVIDIA/NeMo-Relay) plugin that checks each tool call Relay manages before the tool runs. +It sends the tool's name and arguments to the Guardrails hooks server (`nemoguardrails hooks-server`) at `/v1/hooks/nemo-relay`, and the hooks server's tool policy and per-tool rails decide whether the call runs. +The plugin holds no rails of its own, so you decide what to block in the hooks server's configuration. + +## Where the Plugin Checks Calls + +The plugin registers a tool conditional-execution guardrail, which Relay runs before each tool call it manages: + +| Where Relay Runs | Calls the Plugin Checks | +|---|---| +| The Relay CLI with Claude Code or Codex, in every mode | Each `PreToolUse` hook. Through Relay 0.10, also each `PermissionRequest` hook. Refer to [Check Each Call Once](#check-each-call-once). | +| The Relay CLI with pi | Each tool call that Relay's pi extension sends to Relay | +| A Python application | Each tool call that runs through Relay: `nemo_relay.tools.execute`, Relay's LangChain middleware `NemoRelayMiddleware`, which also covers Deep Agents, and Relay's LangGraph tool node | +| OpenClaw | Not supported | + +The plugin checks tool calls only. +A tool's result reaches the model unchecked. + +## Prerequisites + +Before you install the plugin, meet the following prerequisites: + +- NeMo Relay 0.10.1 or a later 0.10 release. + Relay 0.10.0 refuses to load the plugin, because in that release an error from a guardrail let a Claude Code tool call run. +- A running hooks server, with a configuration for the tool names of the agent you check. + To check Claude Code through the Relay CLI, start one with the [Claude Code example configuration](https://github.com/NVIDIA-NeMo/Guardrails/tree/develop/examples/configs/agent_hooks/claude_code) from a clone of the repository: + + ```bash + nemoguardrails hooks-server --config examples/configs/agent_hooks/claude_code + ``` + + Relay forwards each tool's name as the agent or application gives it, and the hooks server matches it exactly. + For Claude Code or Codex, base the configuration on the [Claude Code](/run-guardrailed-inference/using-hooks-server/claude-code#claude-code-tool-names) or [Codex](/run-guardrailed-inference/using-hooks-server/codex#codex-tool-names) tool names. + To write your own configuration, refer to [Run the Hooks Server: Write a Hooks Configuration](/run-guardrailed-inference/using-hooks-server/run-hooks-server#write-a-hooks-configuration). +- To build the plugin: a clone of the [NeMo Guardrails repository](https://github.com/NVIDIA-NeMo/Guardrails), [rustup](https://rustup.rs/), and Python 3. + +## Build the Plugin Bundle + +A Relay plugin bundle is a directory that holds the plugin library, its manifest `relay-plugin.toml`, its configuration schema, and its license files. +Build the library and package the bundle from the repository root: + +```bash +cargo build --release --locked -p nemo-guardrails-relay-plugin +python3 crates/nemo-guardrails-relay-plugin/scripts/package_bundle.py \ + --library target/release/libnemo_guardrails_relay_plugin.so \ + --output ~/nemo-guardrails-relay-plugin +``` + +The build uses the Rust version that the repository's `rust-toolchain.toml` pins. +The library is `libnemo_guardrails_relay_plugin.so` on Linux, `libnemo_guardrails_relay_plugin.dylib` on macOS, and `nemo_guardrails_relay_plugin.dll` on Windows. + +Relay loads the plugin from the bundle directory, so keep the directory in place after you register it. +The bundle is not signed, so Relay loads it only after you allow integrity-only attestation for the plugin, as the following sections show. + +## Choose a Setup + +The plugin works the same way in every Relay mode. +How you run Relay decides who controls the plugin and its configuration: + +| | Try It Out | Production | +|---|---|---| +| Relay mode | `nemo-relay claude`, `nemo-relay codex`, `nemo-relay pi`, or a persistent integration from `nemo-relay install` | The managed Relay daemon, `nemo-relay daemon` | +| Registered with | `nemo-relay plugins add --user` | `nemo-relay plugins add --global` | +| Configuration file | Your user `plugins.toml` | The system `plugins.toml` | +| Hooks server | Runs on your machine, started by you | Runs where an administrator deploys it | +| Controlled by | The user, who can remove or reconfigure the plugin | An administrator | + +A Python application passes its own `plugins.toml` to Relay. +Refer to [Use the Plugin in a Python Application](#use-the-plugin-in-a-python-application). + +## Try the Plugin Out + +Register the plugin for your user, then run an agent through the Relay CLI: + +1. Start the hooks server, as described in [Prerequisites](#prerequisites). +1. Validate the bundle, and register it for your user: + + ```bash + nemo-relay plugins validate ~/nemo-guardrails-relay-plugin/relay-plugin.toml + nemo-relay plugins add --user ~/nemo-guardrails-relay-plugin/relay-plugin.toml + ``` + + `plugins add` adds a `[[plugins.dynamic]]` entry for the plugin to your user `plugins.toml`, at `$XDG_CONFIG_HOME/nemo-relay/plugins.toml`, or at `~/.config/nemo-relay/plugins.toml` when `XDG_CONFIG_HOME` is not set. +1. Open that file, allow the unsigned bundle, and add the plugin's configuration after its entry: + + ```toml + version = 1 + + [plugins.policy.overrides."nvidia.nemo_guardrails"] + attestation = "integrity_only" + + [[plugins.dynamic]] + manifest = "/home/user/nemo-guardrails-relay-plugin/relay-plugin.toml" + + [plugins.dynamic.config] + endpoint = "http://127.0.0.1:8100" + ``` + + Keep `[plugins.dynamic.config]` directly after the plugin's `[[plugins.dynamic]]` entry, because TOML attaches it to the entry above it. + Without the `attestation` override, Relay refuses to start, with an error that the plugin requires `integrity.signature` under host policy. +1. Enable the plugin, and validate its configuration: + + ```bash + nemo-relay plugins enable nvidia.nemo_guardrails + nemo-relay plugins validate nvidia.nemo_guardrails + ``` + + `plugins validate` checks the configuration against the plugin's schema, for example that `endpoint` is set. + It does not check the `attestation` override. +1. Run the agent through Relay: + + ```bash + nemo-relay claude + ``` + + For Codex, run `nemo-relay codex` against a hooks server with a configuration for the Codex tool names. + +For pi, run `nemo-relay pi`, and also set the following: + +- `NEMO_RELAY_PI_FAIL=closed` in pi's environment. + By default, Relay's pi extension lets a call run when the check fails. +- `timeout_millis` below pi's own limit, which `NEMO_RELAY_PI_TIMEOUT_MS` sets and which is 5000 milliseconds by default. + Otherwise pi stops waiting before the plugin does. + +Anyone who can edit your user `plugins.toml` can remove the plugin or point it at another hooks server. +To enforce the checks for every user, refer to [Run the Plugin in Production](#run-the-plugin-in-production). + +### Verify the Plugin + +With the hooks server running the Claude Code example configuration, check the plugin from Claude Code: + +1. Ask Claude to run `echo 'rm -rf /tmp/guardrails-hook-check'`. + The example configuration blocks the command because its text matches the `rm -rf` pattern. + Claude Code blocks the call, and Claude receives the reason, `guardrail rejected: NeMo Guardrails blocked tool call 'Bash': regex check tool call`. + The hooks server logs the check: + + ```text + INFO:nemoguardrails.server.hooks.app:Hook check on /v1/hooks/nemo-relay: event=NemoRelayToolConditionalExecutionGuardrail tool_name='Bash' tool_use_id=None outcome=deny code=None rail='regex check tool call' + ``` + +1. Stop the hooks server and ask Claude to run any command. + Claude Code blocks the call because the check fails. +1. Start the hooks server again. + +## Run the Plugin in Production + +In production, an administrator runs the managed Relay daemon, which loads the plugin for every user from the system `plugins.toml`. +Set up the daemon as described in the NeMo Relay [daemon documentation](https://github.com/NVIDIA/NeMo-Relay/blob/0.10.4/docs/daemon/about.mdx), then add the plugin: + +1. Put the bundle in a directory that administrators own and every user can read, such as `/opt/nemo-guardrails-relay-plugin`. +1. From an account that can write `/etc/nemo-relay`, register and enable the plugin: + + ```bash + sudo nemo-relay plugins add --global /opt/nemo-guardrails-relay-plugin/relay-plugin.toml + sudo nemo-relay plugins enable nvidia.nemo_guardrails + ``` + +1. Add the `attestation` override and the plugin's configuration to `/etc/nemo-relay/plugins.toml`, as in [Try the Plugin Out](#try-the-plugin-out), with the endpoint of the hooks server that serves your users. + Every local user can read this file, so do not put a credential in `headers`. +1. Run `nemo-relay plugins validate nvidia.nemo_guardrails`. + +The daemon enforces the checks only when the following conditions hold: + +- **Start the daemon with `--require-worker`.** + Without it, the daemon answers hooks itself, and lets the call run, while no worker is ready: during worker startup and recovery, and after a daemon restart. +- **Deliver the agent's hook settings through its managed settings.** + Otherwise a user can remove the hooks that send tool calls to the daemon. + Claude Code has managed settings; Relay documents no such mechanism for pi. +- **Do not load a plugin that intercepts model execution, such as Switchyard.** + The daemon's worker refuses to start with one, and with `--require-worker`, the agents then stop working. + +To stage a deployment, first run a daemon for your own user on one machine, as a macOS LaunchAgent or a Linux user service. + +## Use the Plugin in a Python Application + +A Python application loads the plugin from a `plugins.toml` it passes to Relay. +Write the file with the same entries as in [Try the Plugin Out](#try-the-plugin-out). +A relative `manifest` path resolves from the directory of the `plugins.toml` file: + +```toml +version = 1 + +[plugins.policy.overrides."nvidia.nemo_guardrails"] +attestation = "integrity_only" + +[[plugins.dynamic]] +manifest = "nemo-guardrails-relay-plugin/relay-plugin.toml" + +[plugins.dynamic.config] +endpoint = "http://127.0.0.1:8100" +``` + +Activate Relay's plugins with the file, and run the agent while they are active: + +```python +import asyncio + +from nemo_relay import plugin + + +async def main() -> None: + async with plugin.activate({}, "plugins.toml"): + ... # Run your agent here. + + +asyncio.run(main()) +``` + +Relay also reads the system `plugins.toml`, `/etc/nemo-relay/plugins.toml` on Linux and macOS, and its settings take precedence over the file you pass. + +For a LangChain or Deep Agents agent, add Relay's middleware, which runs each tool call through Relay: + +```python +from langchain.agents import create_agent +from nemo_relay.integrations.langchain.middleware import NemoRelayMiddleware + +agent = create_agent(model=model, tools=tools, middleware=[NemoRelayMiddleware()]) +``` + +When the hooks server denies a call, `invoke` and `ainvoke` raise a `RuntimeError`, and the tool does not run: + +```text +RuntimeError: guardrail rejected: NeMo Guardrails blocked tool call 'search': regex check tool call +``` + +A failed check raises a `RuntimeError` whose message starts with `internal error: NeMo Guardrails check failed for tool call`. +Neither error reaches the model, so handle them where your application handles the agent's errors. + +A Python application has no hook time limit, so the default `timeout_millis` leaves room for a rail that calls a model. + +## Configure the Plugin + +The plugin reads its configuration from the `[plugins.dynamic.config]` table of its entry: + +| Field | Default | Description | +|---|---|---| +| `endpoint` | None | Required. The base URL of the hooks server, using `http` or `https`, such as `http://127.0.0.1:8100`. The plugin sends each check to `/v1/hooks/nemo-relay` under it, keeping any path prefix. It must not contain a username or password; send credentials in `headers`. | +| `headers` | `{}` | Headers to send with every check, as header names and string values. The hooks server has no authentication, so use them for a proxy or gateway in front of it, such as an `authorization` header. The plugin never quotes a header value in an error. It rejects headers that its HTTP client sets, such as `content-type`, `content-length`, and `host`. | +| `timeout_millis` | `25000` | The time limit for one check, in milliseconds. Keep it below the agent's hook time limit: 30 seconds for Claude Code and Codex through the Relay CLI, and 5 seconds by default for pi. | +| `priority` | `0` | The plugin's place among Relay's tool guardrails. Lower values run earlier. | +| `executor` | None | Reserved for the Relay SDK's executor settings, such as `worker_threads`. | + +The following example sends an `authorization` header to a proxy in front of the hooks server: + +```toml +[plugins.dynamic.config] +endpoint = "https://guardrails.example.com" +timeout_millis = 10000 + +[plugins.dynamic.config.headers] +authorization = "Bearer " +``` + +Relay refuses to start when the configuration is not valid. + + + +The plugin uses the proxy settings of the Relay process that loads it, from `HTTP_PROXY`, `HTTPS_PROXY`, `ALL_PROXY`, and `NO_PROXY`, even for a hooks server on `127.0.0.1`. +If you set a proxy, add the hooks server's host to `NO_PROXY`, such as `NO_PROXY=127.0.0.1,localhost`. +Otherwise every check goes through the proxy, which then receives the tool's arguments and the configured `headers`, in plain text for an `http` endpoint. + + + +## What Happens to a Call + +The plugin turns the hooks server's answer into the guardrail's result, and Relay acts on it: + +| Hooks Server Answer | Plugin Result | Relay CLI with Claude Code or Codex | Relay CLI with pi | Python Application | +|---|---|---|---|---| +| Allow | Lets the call run | The tool runs, subject to the agent's own permission rules | The tool runs | The tool runs | +| Deny | A rejection: `NeMo Guardrails blocked tool call '': ` | Relay's hook exits `2`, and the agent blocks the call | pi blocks the call | `RuntimeError: guardrail rejected: ...` | +| An error status, no answer within `timeout_millis`, or no connection | An error: `NeMo Guardrails check failed for tool call '': ` | Relay's hook exits `2`, and the agent blocks the call | The tool runs, unless `NEMO_RELAY_PI_FAIL=closed` | `RuntimeError: internal error: ...` | + +The reason for a deny comes from the hooks server: the tool policy's reason, or the name of the per-tool rail that blocked the call. +For every deny reason and error, refer to [Hooks Server Results and Errors](/run-guardrailed-inference/using-hooks-server/results-and-errors). + +The detail of a failed check gives the HTTP status, the hooks server's error code and message, or says that the request timed out or could not be sent. +It never includes a header value or the tool's arguments. +The plugin does not retry a failed check, not even after a `503` with `Retry-After`. + +While the hooks server is stopped, restarting, or slower than `timeout_millis`, every check fails, so every tool call the plugin checks is blocked, except under pi without `NEMO_RELAY_PI_FAIL=closed`. +The same holds for the calls a per-tool rail checks while the model it calls cannot be reached. + +### Find the Reason for a Failed Check + +Through the Relay CLI's gateway, the agent receives the plugin's error inside Relay's message: + +```text +nemo-relay hook forward failed: hook gateway returned HTTP 500 Internal Server Error: NeMo Relay runtime error: internal error: NeMo Guardrails check failed for tool call 'Bash': error sending request for url (http://127.0.0.1:8100/v1/hooks/nemo-relay): client error (Connect): tcp connect error: Connection refused (os error 61) +``` + +Through the managed daemon, the agent can receive only Relay's own message, such as `daemon hook forward failed with HTTP 500`. + +The plugin also logs each failed check as a warning in Relay's log, with the target `nemo_relay.plugin.native.nvidia.nemo_guardrails`, the event `nemo_guardrails_check_failed`, and the fields `tool_name`, `status`, and `error_code`: + +```text +2026-10-28T18:09:36.742Z WARN root=01a183b3 target=nemo_relay.plugin.native.nvidia.nemo_guardrails event=nemo_guardrails_check_failed error_code=None status=None tool_name=Bash NeMo Guardrails check failed for tool call 'Bash': error sending request for url (http://127.0.0.1:8100/v1/hooks/nemo-relay): client error (Connect): tcp connect error: Connection refused (os error 61) +``` + +Relay logs only errors by default, so set its log level to `warn` or lower to keep these warnings, for example with `nemo-relay --log-level warn`. +The hooks server [logs each check that ends in an error](/run-guardrailed-inference/using-hooks-server/run-hooks-server#logs) at the `WARNING` level, with its error code. + +## Limits + +Plan your deployment around the following limits: + +- **No result checks.** + The hooks server checks tool calls only, so a tool's output reaches the model unchecked. +- **One configuration for each Relay process.** + Relay loads the plugin once in each process, with one `endpoint`, and a hooks server serves one configuration. + To apply another policy, run another hooks server, and point another Relay setup at it. +- **No authentication on the hooks server.** + Anyone who can reach the hooks server can send it calls, and learn what the policy allows. + Refer to [Run the Hooks Server: Expose the Server](/run-guardrailed-inference/using-hooks-server/run-hooks-server#expose-the-server). + +## Related Topics + +- [Check NeMo Relay Tool Calls with the Hooks Server](/run-guardrailed-inference/using-hooks-server/nemo-relay) +- [Run the Hooks Server](/run-guardrailed-inference/using-hooks-server/run-hooks-server) +- [Hooks Server Results and Errors](/run-guardrailed-inference/using-hooks-server/results-and-errors) +- [Configure Discoverable Plugins](https://github.com/NVIDIA/NeMo-Relay/blob/0.10.4/docs/configure-plugins/discoverable-plugins.mdx) in the NeMo Relay documentation +- [The plugin's README](https://github.com/NVIDIA-NeMo/Guardrails/tree/develop/crates/nemo-guardrails-relay-plugin), for building and testing it diff --git a/docs/run-rails/using-hooks-server/nemo-relay.mdx b/docs/run-rails/using-hooks-server/nemo-relay.mdx index 04d6f5dc88..99904bac14 100644 --- a/docs/run-rails/using-hooks-server/nemo-relay.mdx +++ b/docs/run-rails/using-hooks-server/nemo-relay.mdx @@ -1,6 +1,6 @@ --- title: "Check NeMo Relay Tool Calls with the Hooks Server" -sidebar-title: "NeMo Relay" +sidebar-title: "NeMo Relay Endpoint" description: "The /v1/hooks/nemo-relay endpoint of the Guardrails hooks server, for a NeMo Relay plugin's tool conditional-execution guardrail." keywords: ["NeMo Relay", "tool conditional-execution guardrail", "nemoguardrails hooks-server", "/v1/hooks/nemo-relay", "NemoRelayToolConditionalExecutionGuardrail"] content: @@ -10,6 +10,7 @@ content: [NeMo Relay](https://github.com/NVIDIA/NeMo-Relay) plugins can register a tool conditional-execution guardrail, which Relay runs before each tool call it manages, with the tool's name and arguments. A plugin can forward that check to the Guardrails hooks server (`nemoguardrails hooks-server`) at `/v1/hooks/nemo-relay`, so the hooks server's tool policy and per-tool rails decide whether the call runs. This page describes the endpoint such a plugin calls. +To install the NVIDIA NeMo Guardrails Plugin for NeMo Relay, which calls it, refer to [Check NeMo Relay Tool Calls with the NVIDIA NeMo Guardrails Plugin](/run-guardrailed-inference/using-hooks-server/nemo-relay-plugin). For how Relay runs these guardrails, refer to [conditional execution](https://github.com/NVIDIA/NeMo-Relay/blob/494018ac4d6abb0085853a02f4b6d75a75ff7049/docs/about-nemo-relay/concepts/middleware.mdx?plain=1#L235) in the NeMo Relay documentation. diff --git a/docs/run-rails/using-hooks-server/overview.mdx b/docs/run-rails/using-hooks-server/overview.mdx index 8c216c11e5..972a69fbb8 100644 --- a/docs/run-rails/using-hooks-server/overview.mdx +++ b/docs/run-rails/using-hooks-server/overview.mdx @@ -184,7 +184,7 @@ The server listens on `http://127.0.0.1:8100`. } ``` -1. To check your agent's calls for real, connect it to the hooks server: [Claude Code](/run-guardrailed-inference/using-hooks-server/claude-code), [Codex](/run-guardrailed-inference/using-hooks-server/codex), or [NeMo Relay](/run-guardrailed-inference/using-hooks-server/nemo-relay). +1. To check your agent's calls for real, connect it to the hooks server: [Claude Code](/run-guardrailed-inference/using-hooks-server/claude-code), [Codex](/run-guardrailed-inference/using-hooks-server/codex), or [NeMo Relay](/run-guardrailed-inference/using-hooks-server/nemo-relay-plugin). ## Known Limits diff --git a/docs/run-rails/using-hooks-server/run-hooks-server.mdx b/docs/run-rails/using-hooks-server/run-hooks-server.mdx index 1c9e727a3b..c1154f3023 100644 --- a/docs/run-rails/using-hooks-server/run-hooks-server.mdx +++ b/docs/run-rails/using-hooks-server/run-hooks-server.mdx @@ -10,7 +10,7 @@ content: The Guardrails hooks server (`nemoguardrails hooks-server`) loads one guardrails configuration at startup. It checks each tool call against that configuration's tool policy and per-tool rails. Use this guide to configure, start, and monitor the server. -To connect an agent, refer to the [Claude Code](/run-guardrailed-inference/using-hooks-server/claude-code), [Codex](/run-guardrailed-inference/using-hooks-server/codex), or [NeMo Relay](/run-guardrailed-inference/using-hooks-server/nemo-relay) page. +To connect an agent, refer to the [Claude Code](/run-guardrailed-inference/using-hooks-server/claude-code), [Codex](/run-guardrailed-inference/using-hooks-server/codex), or [NeMo Relay](/run-guardrailed-inference/using-hooks-server/nemo-relay-plugin) page. ## Start the Server From 3c436eec4040e32cf2540253da2668caa0eebe9e Mon Sep 17 00:00:00 2001 From: tgasser-nv <200644301+tgasser-nv@users.noreply.github.com> Date: Thu, 29 Oct 2026 09:23:19 -0500 Subject: [PATCH 2/7] Fix broken link --- docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx b/docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx index 107701dfb3..18cb76b225 100644 --- a/docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx +++ b/docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx @@ -17,7 +17,7 @@ The plugin registers a tool conditional-execution guardrail, which Relay runs be | Where Relay Runs | Calls the Plugin Checks | |---|---| -| The Relay CLI with Claude Code or Codex, in every mode | Each `PreToolUse` hook. Through Relay 0.10, also each `PermissionRequest` hook. Refer to [Check Each Call Once](#check-each-call-once). | +| The Relay CLI with Claude Code or Codex, in every mode | Each `PreToolUse` hook. Through Relay 0.10, also each `PermissionRequest` hook. Refer to [Check NeMo Relay Tool Calls with the Hooks Server: Check Each Call Once](/run-guardrailed-inference/using-hooks-server/nemo-relay#check-each-call-once). | | The Relay CLI with pi | Each tool call that Relay's pi extension sends to Relay | | A Python application | Each tool call that runs through Relay: `nemo_relay.tools.execute`, Relay's LangChain middleware `NemoRelayMiddleware`, which also covers Deep Agents, and Relay's LangGraph tool node | | OpenClaw | Not supported | From 8c274f3187aced89ef107ea288dc5557bfdc1d57 Mon Sep 17 00:00:00 2001 From: tgasser-nv <200644301+tgasser-nv@users.noreply.github.com> Date: Thu, 29 Oct 2026 09:40:29 -0500 Subject: [PATCH 3/7] Fix relay plugin docs permission prompts, Python load check and setup steps --- .../reference/hooks-server-api/nemo-relay.mdx | 2 +- .../using-hooks-server/nemo-relay-plugin.mdx | 64 ++++++++++++++----- .../using-hooks-server/nemo-relay.mdx | 4 +- .../run-rails/using-hooks-server/overview.mdx | 2 +- .../using-hooks-server/results-and-errors.mdx | 2 +- examples/configs/agent_hooks/README.md | 2 +- 6 files changed, 56 insertions(+), 20 deletions(-) diff --git a/docs/reference/hooks-server-api/nemo-relay.mdx b/docs/reference/hooks-server-api/nemo-relay.mdx index 12e95dbbf8..305d6080a9 100644 --- a/docs/reference/hooks-server-api/nemo-relay.mdx +++ b/docs/reference/hooks-server-api/nemo-relay.mdx @@ -49,7 +49,7 @@ The hooks server applies its tool policy, then the called tool's per-tool rails. The plugin returns the decision as its guardrail callback's result: `Ok(None)` for an allow, `Ok(Some(reason))` for a deny, and `Err` for any error status, timeout, or failed connection. The request examples come from the [NeMo Relay example configuration](https://github.com/NVIDIA-NeMo/Guardrails/tree/develop/examples/configs/agent_hooks/nemo_relay), which allows a `search` tool with only a `query` argument. -For the plugin side, refer to [Check NeMo Relay Tool Calls with the Hooks Server](/run-guardrailed-inference/using-hooks-server/nemo-relay). +For the plugin that calls this endpoint, refer to [Check NeMo Relay Tool Calls with the NVIDIA NeMo Guardrails Plugin](/run-guardrailed-inference/using-hooks-server/nemo-relay-plugin). ## Request diff --git a/docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx b/docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx index 18cb76b225..fa936f9c2b 100644 --- a/docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx +++ b/docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx @@ -31,8 +31,15 @@ Before you install the plugin, meet the following prerequisites: - NeMo Relay 0.10.1 or a later 0.10 release. Relay 0.10.0 refuses to load the plugin, because in that release an error from a guardrail let a Claude Code tool call run. -- A running hooks server, with a configuration for the tool names of the agent you check. - To check Claude Code through the Relay CLI, start one with the [Claude Code example configuration](https://github.com/NVIDIA-NeMo/Guardrails/tree/develop/examples/configs/agent_hooks/claude_code) from a clone of the repository: +- A running hooks server, with a configuration for the tool names of the agent or application you check. + Start one from a clone of the repository, with the example configuration for what you check: + + - Claude Code, through `nemo-relay claude`: the [Claude Code example configuration](https://github.com/NVIDIA-NeMo/Guardrails/tree/develop/examples/configs/agent_hooks/claude_code). + - Codex, through `nemo-relay codex`: the [Codex example configuration](https://github.com/NVIDIA-NeMo/Guardrails/tree/develop/examples/configs/agent_hooks/codex). + - A Python application: the [NeMo Relay example configuration](https://github.com/NVIDIA-NeMo/Guardrails/tree/develop/examples/configs/agent_hooks/nemo_relay), which allows only a `search` tool. The Python examples on this page use it. + - pi: the repository has no example, so write a configuration for pi's tool names. + + For example, to check Claude Code, run the following command: ```bash nemoguardrails hooks-server --config examples/configs/agent_hooks/claude_code @@ -41,12 +48,14 @@ Before you install the plugin, meet the following prerequisites: Relay forwards each tool's name as the agent or application gives it, and the hooks server matches it exactly. For Claude Code or Codex, base the configuration on the [Claude Code](/run-guardrailed-inference/using-hooks-server/claude-code#claude-code-tool-names) or [Codex](/run-guardrailed-inference/using-hooks-server/codex#codex-tool-names) tool names. To write your own configuration, refer to [Run the Hooks Server: Write a Hooks Configuration](/run-guardrailed-inference/using-hooks-server/run-hooks-server#write-a-hooks-configuration). -- To build the plugin: a clone of the [NeMo Guardrails repository](https://github.com/NVIDIA-NeMo/Guardrails), [rustup](https://rustup.rs/), and Python 3. +- To build the plugin: a clone of the [NeMo Guardrails repository](https://github.com/NVIDIA-NeMo/Guardrails), [rustup](https://rustup.rs/), a C compiler, such as the Xcode Command Line Tools on macOS or `build-essential` on Debian and Ubuntu, and Python 3. ## Build the Plugin Bundle A Relay plugin bundle is a directory that holds the plugin library, its manifest `relay-plugin.toml`, its configuration schema, and its license files. -Build the library and package the bundle from the repository root: +The library is `libnemo_guardrails_relay_plugin.so` on Linux, `libnemo_guardrails_relay_plugin.dylib` on macOS, and `nemo_guardrails_relay_plugin.dll` on Windows. +Build the library and package the bundle from the repository root, passing your platform's library to `--library`. +On Linux, run the following commands: ```bash cargo build --release --locked -p nemo-guardrails-relay-plugin @@ -55,12 +64,15 @@ python3 crates/nemo-guardrails-relay-plugin/scripts/package_bundle.py \ --output ~/nemo-guardrails-relay-plugin ``` +On macOS, pass `--library target/release/libnemo_guardrails_relay_plugin.dylib` instead. The build uses the Rust version that the repository's `rust-toolchain.toml` pins. -The library is `libnemo_guardrails_relay_plugin.so` on Linux, `libnemo_guardrails_relay_plugin.dylib` on macOS, and `nemo_guardrails_relay_plugin.dll` on Windows. Relay loads the plugin from the bundle directory, so keep the directory in place after you register it. The bundle is not signed, so Relay loads it only after you allow integrity-only attestation for the plugin, as the following sections show. +To update the plugin, for example to pick up a fix, package the new build into a new, empty directory, because the packaging script refuses an output directory that has files in it. +Then change the plugin's `manifest` path in `plugins.toml` to the new directory, restart any running Relay process, and remove the old directory. + ## Choose a Setup The plugin works the same way in every Relay mode. @@ -114,8 +126,9 @@ Register the plugin for your user, then run an agent through the Relay CLI: nemo-relay plugins validate nvidia.nemo_guardrails ``` - `plugins validate` checks the configuration against the plugin's schema, for example that `endpoint` is set. - It does not check the `attestation` override. + `plugins validate` checks the configuration against the plugin's schema: the field types, and that `endpoint` is set. + It does not check the `attestation` override, the endpoint's URL, or the header names. + The plugin checks the URL and the headers when Relay loads it, and Relay refuses to start when either is not valid. 1. Run the agent through Relay: ```bash @@ -157,16 +170,22 @@ In production, an administrator runs the managed Relay daemon, which loads the p Set up the daemon as described in the NeMo Relay [daemon documentation](https://github.com/NVIDIA/NeMo-Relay/blob/0.10.4/docs/daemon/about.mdx), then add the plugin: 1. Put the bundle in a directory that administrators own and every user can read, such as `/opt/nemo-guardrails-relay-plugin`. -1. From an account that can write `/etc/nemo-relay`, register and enable the plugin: +1. From an account that can write `/etc/nemo-relay`, register the plugin: ```bash sudo nemo-relay plugins add --global /opt/nemo-guardrails-relay-plugin/relay-plugin.toml - sudo nemo-relay plugins enable nvidia.nemo_guardrails ``` 1. Add the `attestation` override and the plugin's configuration to `/etc/nemo-relay/plugins.toml`, as in [Try the Plugin Out](#try-the-plugin-out), with the endpoint of the hooks server that serves your users. Every local user can read this file, so do not put a credential in `headers`. -1. Run `nemo-relay plugins validate nvidia.nemo_guardrails`. +1. Enable the plugin, and validate its configuration: + + ```bash + sudo nemo-relay plugins enable nvidia.nemo_guardrails + nemo-relay plugins validate nvidia.nemo_guardrails + ``` + + Enable the plugin only after you add its override and configuration, because Relay refuses to start with the plugin enabled and either one missing. The daemon enforces the checks only when the following conditions hold: @@ -174,7 +193,8 @@ The daemon enforces the checks only when the following conditions hold: Without it, the daemon answers hooks itself, and lets the call run, while no worker is ready: during worker startup and recovery, and after a daemon restart. - **Deliver the agent's hook settings through its managed settings.** Otherwise a user can remove the hooks that send tool calls to the daemon. - Claude Code has managed settings; Relay documents no such mechanism for pi. + Claude Code has managed settings, and Codex has managed hooks in `requirements.toml`, which users cannot disable, as described in [Codex: Deploy the Hook with Managed Settings](/run-guardrailed-inference/using-hooks-server/codex#deploy-the-hook-with-managed-settings). + Relay's managed pi extension takes effect only when pi starts with `--no-extensions` and the extension's path, and the user controls the command that starts pi. - **Do not load a plugin that intercepts model execution, such as Switchyard.** The daemon's worker refuses to start with one, and with `--require-worker`, the agents then stop working. @@ -183,7 +203,13 @@ To stage a deployment, first run a daemon for your own user on one machine, as a ## Use the Plugin in a Python Application A Python application loads the plugin from a `plugins.toml` it passes to Relay. -Write the file with the same entries as in [Try the Plugin Out](#try-the-plugin-out). +Install Relay's Python package from the same 0.10 release as the rest of your Relay setup, with the `langchain` extra if you use Relay's LangChain middleware: + +```bash +pip install "nemo-relay[langchain]>=0.10.1,<0.11" +``` + +Write the `plugins.toml` file with the same entries as in [Try the Plugin Out](#try-the-plugin-out). A relative `manifest` path resolves from the directory of the `plugins.toml` file: ```toml @@ -199,22 +225,30 @@ manifest = "nemo-guardrails-relay-plugin/relay-plugin.toml" endpoint = "http://127.0.0.1:8100" ``` -Activate Relay's plugins with the file, and run the agent while they are active: +Activate Relay's plugins with the file, check that the plugin loaded, and run the agent while the plugins are active: ```python import asyncio +from pathlib import Path from nemo_relay import plugin async def main() -> None: - async with plugin.activate({}, "plugins.toml"): + async with plugin.activate({}, Path(__file__).with_name("plugins.toml")) as activation: + report = activation.report + loaded = {entry["plugin_id"] for entry in report["dynamic_plugins"] if entry["selected"]} + if "nvidia.nemo_guardrails" not in loaded: + raise RuntimeError(f"NeMo Guardrails plugin not loaded: {report['config']['diagnostics']}") ... # Run your agent here. asyncio.run(main()) ``` +When the `plugins.toml` file you pass does not exist, Relay only adds a `plugin.configuration_file_missing` warning to the activation report, and activates without the plugin, so every tool runs unchecked. +The example builds the path from the script's location, so that it does not depend on the working directory, and stops when the plugin is not loaded. + Relay also reads the system `plugins.toml`, `/etc/nemo-relay/plugins.toml` on Linux and macOS, and its settings take precedence over the file you pass. For a LangChain or Deep Agents agent, add Relay's middleware, which runs each tool call through Relay: @@ -276,7 +310,7 @@ The plugin turns the hooks server's answer into the guardrail's result, and Rela | Hooks Server Answer | Plugin Result | Relay CLI with Claude Code or Codex | Relay CLI with pi | Python Application | |---|---|---|---|---| -| Allow | Lets the call run | The tool runs, subject to the agent's own permission rules | The tool runs | The tool runs | +| Allow | Lets the call run | The tool runs. Through Relay 0.10, Relay also approves Claude Code's permission prompt for the call, so Claude Code does not ask the user. Codex applies its own approval settings. | The tool runs | The tool runs | | Deny | A rejection: `NeMo Guardrails blocked tool call '': ` | Relay's hook exits `2`, and the agent blocks the call | pi blocks the call | `RuntimeError: guardrail rejected: ...` | | An error status, no answer within `timeout_millis`, or no connection | An error: `NeMo Guardrails check failed for tool call '': ` | Relay's hook exits `2`, and the agent blocks the call | The tool runs, unless `NEMO_RELAY_PI_FAIL=closed` | `RuntimeError: internal error: ...` | diff --git a/docs/run-rails/using-hooks-server/nemo-relay.mdx b/docs/run-rails/using-hooks-server/nemo-relay.mdx index 99904bac14..a9499906b1 100644 --- a/docs/run-rails/using-hooks-server/nemo-relay.mdx +++ b/docs/run-rails/using-hooks-server/nemo-relay.mdx @@ -81,7 +81,8 @@ Map every failure to `Err`, never to `Ok(None)`, so the plugin never reports a c What Relay then does with the call depends on the integration that runs the agent: - Relay's Claude Code and Codex integrations block the call on `Err` from Relay 0.10.1. - The agent sees Relay's own error text, not the plugin's. + Through the Relay CLI's gateway, the agent sees the plugin's error inside Relay's message; through the managed Relay daemon, it sees only Relay's own message. + Refer to [Check NeMo Relay Tool Calls with the NVIDIA NeMo Guardrails Plugin: Find the Reason for a Failed Check](/run-guardrailed-inference/using-hooks-server/nemo-relay-plugin#find-the-reason-for-a-failed-check). - Relay's pi extension treats `Err` as a fault, and by default, with `NEMO_RELAY_PI_FAIL=open`, lets the call run. To block it, change that setting as described in the NeMo Relay documentation. @@ -99,6 +100,7 @@ Account for the following sources of duplicate checks: - **Relay 0.10 checks some calls twice.** Through the 0.10 releases, Relay also runs the guardrail on Claude Code's and Codex's `PermissionRequest` event. A call that reaches a permission prompt then costs two requests to the hooks server, and two runs of any rail that calls a model. + When the second check allows the call, Relay also approves Claude Code's permission prompt, so Claude Code does not ask the user. ## Related Topics diff --git a/docs/run-rails/using-hooks-server/overview.mdx b/docs/run-rails/using-hooks-server/overview.mdx index 972a69fbb8..dc10061432 100644 --- a/docs/run-rails/using-hooks-server/overview.mdx +++ b/docs/run-rails/using-hooks-server/overview.mdx @@ -98,7 +98,7 @@ The hooks server supports the following agent integrations: |---|---|---|---| | [Claude Code](/run-guardrailed-inference/using-hooks-server/claude-code) | `/v1/hooks/claude-code` | An `http` hook on `PreToolUse` | 2.1.295, for `onFailure: "block"` | | [Codex](/run-guardrailed-inference/using-hooks-server/codex) | `/v1/hooks/codex` | A `command` hook on `PreToolUse` that runs `curl` | 0.124.0. From 0.129.0, Codex runs a hook only after you trust it | -| [NeMo Relay](/run-guardrailed-inference/using-hooks-server/nemo-relay) | `/v1/hooks/nemo-relay` | A plugin's tool conditional-execution guardrail | 0.10.1 | +| [NeMo Relay](/run-guardrailed-inference/using-hooks-server/nemo-relay-plugin) | `/v1/hooks/nemo-relay` | A plugin's tool conditional-execution guardrail | 0.10.1 | The hooks server handles only the `PreToolUse` event, and the NeMo Relay guardrail that stands in for it. Any other hook event returns HTTP `422`, so register the hooks server on `PreToolUse` only. diff --git a/docs/run-rails/using-hooks-server/results-and-errors.mdx b/docs/run-rails/using-hooks-server/results-and-errors.mdx index e58452938f..cbb305758e 100644 --- a/docs/run-rails/using-hooks-server/results-and-errors.mdx +++ b/docs/run-rails/using-hooks-server/results-and-errors.mdx @@ -118,7 +118,7 @@ The following table describes how to block tool calls when a check fails: |---|---|---| | Claude Code | Is a non-blocking error, and the tool runs | Set `onFailure: "block"` on the hook. Refer to [Claude Code: Add the Hook](/run-guardrailed-inference/using-hooks-server/claude-code#add-the-hook). | | Codex | Lets the tool run, unless the hook command exits `2`; a hook that times out lets it run too | Pass `curl` the `-f` option and a `--max-time` below the hook's `timeout`, and end the command with `\|\| exit 2`. Refer to [Codex: Add the Hook](/run-guardrailed-inference/using-hooks-server/codex#add-the-hook). | -| NeMo Relay | Depends on the plugin and the integration | The plugin returns `Err` from the guardrail callback for any failed request, and Relay's Claude Code and Codex integrations block the call on `Err` from Relay 0.10.1. Relay's pi extension lets the call run by default. Refer to [NeMo Relay: Map the Response to the Guardrail Result](/run-guardrailed-inference/using-hooks-server/nemo-relay#map-the-response-to-the-guardrail-result). | +| NeMo Relay | Depends on the plugin and the integration | The plugin returns `Err` from the guardrail callback for any failed request, and Relay's Claude Code and Codex integrations block the call on `Err` from Relay 0.10.1. Relay's pi extension lets the call run by default. Refer to [Check NeMo Relay Tool Calls with the NVIDIA NeMo Guardrails Plugin: What Happens to a Call](/run-guardrailed-inference/using-hooks-server/nemo-relay-plugin#what-happens-to-a-call). | These instructions apply to `PreToolUse` only. Other events have different failure behavior. For example, Claude Code ignores `onFailure` on its `Stop` event. diff --git a/examples/configs/agent_hooks/README.md b/examples/configs/agent_hooks/README.md index 2ab7277c31..80f5e738b9 100644 --- a/examples/configs/agent_hooks/README.md +++ b/examples/configs/agent_hooks/README.md @@ -41,7 +41,7 @@ The example's `Bash` rail blocks the force push: {"hookSpecificOutput":{"hookEventName":"PreToolUse","permissionDecision":"deny","permissionDecisionReason":"NeMo Guardrails: regex check tool call"}} ``` -To point the agent's hook at the server, follow the agent's page: [Claude Code](./../../../docs/run-rails/using-hooks-server/claude-code.mdx), [Codex](./../../../docs/run-rails/using-hooks-server/codex.mdx), or [NeMo Relay](./../../../docs/run-rails/using-hooks-server/nemo-relay.mdx). +To point the agent's hook at the server, follow the agent's page: [Claude Code](./../../../docs/run-rails/using-hooks-server/claude-code.mdx), [Codex](./../../../docs/run-rails/using-hooks-server/codex.mdx), or [NeMo Relay](./../../../docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx). ## One Server for Each Agent From 5810e6d95d70afa2d976a4013681a0325312eba9 Mon Sep 17 00:00:00 2001 From: tgasser-nv <200644301+tgasser-nv@users.noreply.github.com> Date: Thu, 29 Oct 2026 10:04:15 -0500 Subject: [PATCH 4/7] Add Python 3.11 floor and fix bundle update steps in Relay plugin docs --- .../using-hooks-server/nemo-relay-plugin.mdx | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx b/docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx index fa936f9c2b..3312b3e05c 100644 --- a/docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx +++ b/docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx @@ -70,8 +70,20 @@ The build uses the Rust version that the repository's `rust-toolchain.toml` pins Relay loads the plugin from the bundle directory, so keep the directory in place after you register it. The bundle is not signed, so Relay loads it only after you allow integrity-only attestation for the plugin, as the following sections show. -To update the plugin, for example to pick up a fix, package the new build into a new, empty directory, because the packaging script refuses an output directory that has files in it. -Then change the plugin's `manifest` path in `plugins.toml` to the new directory, restart any running Relay process, and remove the old directory. +To update the plugin, for example to pick up a fix, replace the bundle at the same path. +Relay records the manifest's path when you register the plugin, and refuses to start when `plugins.toml` names another one. +Replace the whole bundle, not only the library: the packaging script writes the library's digest into the manifest, and Relay refuses a library that does not match it. +Build the new library as shown previously, package it next to the old bundle, because the packaging script refuses an output directory that has files in it, and swap the directories while Relay is stopped: + +```bash +python3 crates/nemo-guardrails-relay-plugin/scripts/package_bundle.py \ + --library target/release/libnemo_guardrails_relay_plugin.so \ + --output ~/nemo-guardrails-relay-plugin.new +rm -rf ~/nemo-guardrails-relay-plugin +mv ~/nemo-guardrails-relay-plugin.new ~/nemo-guardrails-relay-plugin +``` + +Then start Relay again, and run `nemo-relay plugins validate nvidia.nemo_guardrails`. ## Choose a Setup @@ -203,7 +215,8 @@ To stage a deployment, first run a daemon for your own user on one machine, as a ## Use the Plugin in a Python Application A Python application loads the plugin from a `plugins.toml` it passes to Relay. -Install Relay's Python package from the same 0.10 release as the rest of your Relay setup, with the `langchain` extra if you use Relay's LangChain middleware: +Relay's Python package needs Python 3.11 or later. +Install it from the same 0.10 release as the rest of your Relay setup, with the `langchain` extra if you use Relay's LangChain middleware: ```bash pip install "nemo-relay[langchain]>=0.10.1,<0.11" From 3168f12d45c9220a38650b6b67ac515a6746d6e2 Mon Sep 17 00:00:00 2001 From: tgasser-nv <200644301+tgasser-nv@users.noreply.github.com> Date: Thu, 29 Oct 2026 13:24:08 -0500 Subject: [PATCH 5/7] Coderabbit feedback fixes --- .../using-hooks-server/nemo-relay-plugin.mdx | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx b/docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx index 3312b3e05c..fd506d52fd 100644 --- a/docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx +++ b/docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx @@ -178,7 +178,8 @@ With the hooks server running the Claude Code example configuration, check the p ## Run the Plugin in Production -In production, an administrator runs the managed Relay daemon, which loads the plugin for every user from the system `plugins.toml`. +In production, an administrator runs the managed Relay daemon. +Its workers load the plugin for every user from the system `plugins.toml` of the machine they run on. Set up the daemon as described in the NeMo Relay [daemon documentation](https://github.com/NVIDIA/NeMo-Relay/blob/0.10.4/docs/daemon/about.mdx), then add the plugin: 1. Put the bundle in a directory that administrators own and every user can read, such as `/opt/nemo-guardrails-relay-plugin`. @@ -199,6 +200,10 @@ Set up the daemon as described in the NeMo Relay [daemon documentation](https:// Enable the plugin only after you add its override and configuration, because Relay refuses to start with the plugin enabled and either one missing. +With a [remote daemon](https://github.com/NVIDIA/NeMo-Relay/blob/0.10.4/docs/daemon/remote-server.mdx), the workers run on the client machines, not on the daemon host. +Repeat these steps on every client machine, with an endpoint that the client can reach, and run `nemo-relay plugins validate nvidia.nemo_guardrails` there. +A plugin installed only on the daemon host checks no calls from remote workers. + The daemon enforces the checks only when the following conditions hold: - **Start the daemon with `--require-worker`.** @@ -223,7 +228,7 @@ pip install "nemo-relay[langchain]>=0.10.1,<0.11" ``` Write the `plugins.toml` file with the same entries as in [Try the Plugin Out](#try-the-plugin-out). -A relative `manifest` path resolves from the directory of the `plugins.toml` file: +Point `manifest` at the bundle you built: ```toml version = 1 @@ -232,12 +237,14 @@ version = 1 attestation = "integrity_only" [[plugins.dynamic]] -manifest = "nemo-guardrails-relay-plugin/relay-plugin.toml" +manifest = "/home/user/nemo-guardrails-relay-plugin/relay-plugin.toml" [plugins.dynamic.config] endpoint = "http://127.0.0.1:8100" ``` +A relative `manifest` path also works, and resolves from the directory of the `plugins.toml` file, for example when the bundle ships beside your application. + Activate Relay's plugins with the file, check that the plugin loaded, and run the agent while the plugins are active: ```python @@ -331,7 +338,8 @@ The reason for a deny comes from the hooks server: the tool policy's reason, or For every deny reason and error, refer to [Hooks Server Results and Errors](/run-guardrailed-inference/using-hooks-server/results-and-errors). The detail of a failed check gives the HTTP status, the hooks server's error code and message, or says that the request timed out or could not be sent. -It never includes a header value or the tool's arguments. +The plugin adds no header value or tool argument to it, but it repeats the server's error message, up to 512 characters. +A proxy or gateway in front of the hooks server could echo a header or an argument in that message. The plugin does not retry a failed check, not even after a `503` with `Retry-After`. While the hooks server is stopped, restarting, or slower than `timeout_millis`, every check fails, so every tool call the plugin checks is blocked, except under pi without `NEMO_RELAY_PI_FAIL=closed`. From b65d1c23f5e337eb8bc4a8dd9af57c8df2671d38 Mon Sep 17 00:00:00 2001 From: tgasser-nv <200644301+tgasser-nv@users.noreply.github.com> Date: Thu, 29 Oct 2026 13:31:00 -0500 Subject: [PATCH 6/7] Followup to Code rabbit partial-fix --- docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx b/docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx index fd506d52fd..93cef26ce3 100644 --- a/docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx +++ b/docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx @@ -298,7 +298,7 @@ The plugin reads its configuration from the `[plugins.dynamic.config]` table of | Field | Default | Description | |---|---|---| | `endpoint` | None | Required. The base URL of the hooks server, using `http` or `https`, such as `http://127.0.0.1:8100`. The plugin sends each check to `/v1/hooks/nemo-relay` under it, keeping any path prefix. It must not contain a username or password; send credentials in `headers`. | -| `headers` | `{}` | Headers to send with every check, as header names and string values. The hooks server has no authentication, so use them for a proxy or gateway in front of it, such as an `authorization` header. The plugin never quotes a header value in an error. It rejects headers that its HTTP client sets, such as `content-type`, `content-length`, and `host`. | +| `headers` | `{}` | Headers to send with every check, as header names and string values. The hooks server has no authentication, so use them for a proxy or gateway in front of it, such as an `authorization` header. The plugin adds no header value to its own errors, but a failed check repeats the server's error message, which a proxy or gateway could fill with one; refer to [What Happens to a Call](#what-happens-to-a-call). It rejects headers that its HTTP client sets, such as `content-type`, `content-length`, and `host`. | | `timeout_millis` | `25000` | The time limit for one check, in milliseconds. Keep it below the agent's hook time limit: 30 seconds for Claude Code and Codex through the Relay CLI, and 5 seconds by default for pi. | | `priority` | `0` | The plugin's place among Relay's tool guardrails. Lower values run earlier. | | `executor` | None | Reserved for the Relay SDK's executor settings, such as `worker_threads`. | From dc7f9248b6878f5270c1a762a2754bd109dabef8 Mon Sep 17 00:00:00 2001 From: Miyoung Choi Date: Fri, 30 Oct 2026 09:57:07 -0700 Subject: [PATCH 7/7] docs: correct Relay plugin setup and failure guidance Audit the client guide against the Guardrails plugin and NeMo Relay 0.10.4 source. Correct system-registration validation permissions, distinguish managed pi failure handling and timeouts from the standalone extension, and remove the unsupported daemon restriction on model execution intercepts. Cover malformed decision responses and preserve the previous plugin bundle during updates. Apply NVIDIA documentation style to the client guide and keep the endpoint and error reference consistent. Validation: pre-commit on the three edited pages; Fern check (0 errors, 567 warnings outside these pages); 18 documentation tooling tests; Python and shell syntax checks and TOML examples checked against the plugin schema. This change does not include a live-agent deployment test. AI assistance: Codex audited source contracts and edited the documentation. Signed-off-by: Miyoung Choi --- .../using-hooks-server/nemo-relay-plugin.mdx | 167 +++++++++++------- .../using-hooks-server/nemo-relay.mdx | 12 +- .../using-hooks-server/results-and-errors.mdx | 2 +- 3 files changed, 114 insertions(+), 67 deletions(-) diff --git a/docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx b/docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx index 93cef26ce3..c0e6fe340d 100644 --- a/docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx +++ b/docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx @@ -1,15 +1,16 @@ --- title: "Check NeMo Relay Tool Calls with the NVIDIA NeMo Guardrails Plugin" sidebar-title: "NeMo Relay Plugin" -description: "Install the NVIDIA NeMo Guardrails Plugin for NeMo Relay, which checks each tool call Relay manages with the Guardrails hooks server before the tool runs." +description: "Build, install, and configure the NVIDIA NeMo Guardrails Plugin for NeMo Relay to check tool calls before they run." keywords: ["NeMo Relay", "NeMo Relay plugin", "nvidia.nemo_guardrails", "nemo-relay plugins add", "plugins.toml", "nemoguardrails hooks-server", "/v1/hooks/nemo-relay", "NemoRelayMiddleware"] content: type: "how_to" --- -The NVIDIA NeMo Guardrails Plugin for NeMo Relay, `nvidia.nemo_guardrails`, is a native [NeMo Relay](https://github.com/NVIDIA/NeMo-Relay) plugin that checks each tool call Relay manages before the tool runs. -It sends the tool's name and arguments to the Guardrails hooks server (`nemoguardrails hooks-server`) at `/v1/hooks/nemo-relay`, and the hooks server's tool policy and per-tool rails decide whether the call runs. -The plugin holds no rails of its own, so you decide what to block in the hooks server's configuration. +The NVIDIA NeMo Guardrails Plugin for [NeMo Relay](https://github.com/NVIDIA/NeMo-Relay), `nvidia.nemo_guardrails`, checks each tool call Relay manages before the tool runs. +It sends the tool's name and arguments to the hooks server in the NVIDIA NeMo Guardrails library at `/v1/hooks/nemo-relay`. +The hooks server's tool policy and per-tool rails decide whether the call runs. +Configure the rails in the hooks server, which you start with `nemoguardrails hooks-server`. ## Where the Plugin Checks Calls @@ -19,7 +20,7 @@ The plugin registers a tool conditional-execution guardrail, which Relay runs be |---|---| | The Relay CLI with Claude Code or Codex, in every mode | Each `PreToolUse` hook. Through Relay 0.10, also each `PermissionRequest` hook. Refer to [Check NeMo Relay Tool Calls with the Hooks Server: Check Each Call Once](/run-guardrailed-inference/using-hooks-server/nemo-relay#check-each-call-once). | | The Relay CLI with pi | Each tool call that Relay's pi extension sends to Relay | -| A Python application | Each tool call that runs through Relay: `nemo_relay.tools.execute`, Relay's LangChain middleware `NemoRelayMiddleware`, which also covers Deep Agents, and Relay's LangGraph tool node | +| A Python application | Each tool call through `nemo_relay.tools.execute`, the `NemoRelayMiddleware` middleware for LangChain and Deep Agents, or Relay's LangGraph tool node | | OpenClaw | Not supported | The plugin checks tool calls only. @@ -30,14 +31,14 @@ A tool's result reaches the model unchecked. Before you install the plugin, meet the following prerequisites: - NeMo Relay 0.10.1 or a later 0.10 release. - Relay 0.10.0 refuses to load the plugin, because in that release an error from a guardrail let a Claude Code tool call run. + Relay 0.10.0 refuses to load the plugin. In that release, an error from a guardrail lets a Claude Code tool call run. - A running hooks server, with a configuration for the tool names of the agent or application you check. - Start one from a clone of the repository, with the example configuration for what you check: + Start the server from a clone of the repository with the configuration for your integration: - Claude Code, through `nemo-relay claude`: the [Claude Code example configuration](https://github.com/NVIDIA-NeMo/Guardrails/tree/develop/examples/configs/agent_hooks/claude_code). - Codex, through `nemo-relay codex`: the [Codex example configuration](https://github.com/NVIDIA-NeMo/Guardrails/tree/develop/examples/configs/agent_hooks/codex). - A Python application: the [NeMo Relay example configuration](https://github.com/NVIDIA-NeMo/Guardrails/tree/develop/examples/configs/agent_hooks/nemo_relay), which allows only a `search` tool. The Python examples on this page use it. - - pi: the repository has no example, so write a configuration for pi's tool names. + - pi: Write a configuration for pi's tool names. The repository has no pi example configuration. For example, to check Claude Code, run the following command: @@ -48,12 +49,20 @@ Before you install the plugin, meet the following prerequisites: Relay forwards each tool's name as the agent or application gives it, and the hooks server matches it exactly. For Claude Code or Codex, base the configuration on the [Claude Code](/run-guardrailed-inference/using-hooks-server/claude-code#claude-code-tool-names) or [Codex](/run-guardrailed-inference/using-hooks-server/codex#codex-tool-names) tool names. To write your own configuration, refer to [Run the Hooks Server: Write a Hooks Configuration](/run-guardrailed-inference/using-hooks-server/run-hooks-server#write-a-hooks-configuration). -- To build the plugin: a clone of the [NeMo Guardrails repository](https://github.com/NVIDIA-NeMo/Guardrails), [rustup](https://rustup.rs/), a C compiler, such as the Xcode Command Line Tools on macOS or `build-essential` on Debian and Ubuntu, and Python 3. +- A clone of the [NeMo Guardrails repository](https://github.com/NVIDIA-NeMo/Guardrails), [rustup](https://rustup.rs/), a C compiler, and Python 3 to build the plugin. + For the C compiler, install the Xcode Command Line Tools on macOS or `build-essential` on Debian and Ubuntu. ## Build the Plugin Bundle -A Relay plugin bundle is a directory that holds the plugin library, its manifest `relay-plugin.toml`, its configuration schema, and its license files. -The library is `libnemo_guardrails_relay_plugin.so` on Linux, `libnemo_guardrails_relay_plugin.dylib` on macOS, and `nemo_guardrails_relay_plugin.dll` on Windows. +A Relay plugin bundle contains the plugin library, its manifest `relay-plugin.toml`, its configuration schema, and its license files. +The library filename depends on your operating system: + +| Operating System | Library Filename | +|---|---| +| Linux | `libnemo_guardrails_relay_plugin.so` | +| macOS | `libnemo_guardrails_relay_plugin.dylib` | +| Windows | `nemo_guardrails_relay_plugin.dll` | + Build the library and package the bundle from the repository root, passing your platform's library to `--library`. On Linux, run the following commands: @@ -70,27 +79,38 @@ The build uses the Rust version that the repository's `rust-toolchain.toml` pins Relay loads the plugin from the bundle directory, so keep the directory in place after you register it. The bundle is not signed, so Relay loads it only after you allow integrity-only attestation for the plugin, as the following sections show. -To update the plugin, for example to pick up a fix, replace the bundle at the same path. -Relay records the manifest's path when you register the plugin, and refuses to start when `plugins.toml` names another one. -Replace the whole bundle, not only the library: the packaging script writes the library's digest into the manifest, and Relay refuses a library that does not match it. -Build the new library as shown previously, package it next to the old bundle, because the packaging script refuses an output directory that has files in it, and swap the directories while Relay is stopped: +### Update the Plugin Bundle + +To update the plugin, replace the bundle at the registered path. +Relay records the manifest's path during registration and refuses to start if the path in `plugins.toml` differs from the registered path. +Replace the entire bundle because the packaging script writes the library's digest into the manifest. +Relay refuses to load a library that does not match that digest. + +Build the new library as described previously, then package it in an empty directory next to the current bundle. +The packaging script rejects an output directory that contains files. +Stop Relay before replacing the bundle, and ensure that the `.previous` backup path does not exist. +On Linux, run the following commands: ```bash python3 crates/nemo-guardrails-relay-plugin/scripts/package_bundle.py \ --library target/release/libnemo_guardrails_relay_plugin.so \ - --output ~/nemo-guardrails-relay-plugin.new -rm -rf ~/nemo-guardrails-relay-plugin -mv ~/nemo-guardrails-relay-plugin.new ~/nemo-guardrails-relay-plugin + --output ~/nemo-guardrails-relay-plugin.new && + mv ~/nemo-guardrails-relay-plugin ~/nemo-guardrails-relay-plugin.previous && + mv ~/nemo-guardrails-relay-plugin.new ~/nemo-guardrails-relay-plugin ``` -Then start Relay again, and run `nemo-relay plugins validate nvidia.nemo_guardrails`. +On macOS, use the `.dylib` library path instead. +Validate the replacement with `nemo-relay plugins validate nvidia.nemo_guardrails`, then restart Relay. +For a globally registered plugin, run validation with `sudo`. +Keep the `.previous` bundle until you verify the replacement. +If replacement or validation fails, restore the previous bundle at the registered path before restarting Relay. ## Choose a Setup The plugin works the same way in every Relay mode. How you run Relay decides who controls the plugin and its configuration: -| | Try It Out | Production | +| Setting | Try It Out | Production | |---|---|---| | Relay mode | `nemo-relay claude`, `nemo-relay codex`, `nemo-relay pi`, or a persistent integration from `nemo-relay install` | The managed Relay daemon, `nemo-relay daemon` | | Registered with | `nemo-relay plugins add --user` | `nemo-relay plugins add --global` | @@ -113,7 +133,8 @@ Register the plugin for your user, then run an agent through the Relay CLI: nemo-relay plugins add --user ~/nemo-guardrails-relay-plugin/relay-plugin.toml ``` - `plugins add` adds a `[[plugins.dynamic]]` entry for the plugin to your user `plugins.toml`, at `$XDG_CONFIG_HOME/nemo-relay/plugins.toml`, or at `~/.config/nemo-relay/plugins.toml` when `XDG_CONFIG_HOME` is not set. + `plugins add` adds a `[[plugins.dynamic]]` entry to your user `plugins.toml` at `$XDG_CONFIG_HOME/nemo-relay/plugins.toml`. + If `XDG_CONFIG_HOME` is not set, the file is at `~/.config/nemo-relay/plugins.toml`. 1. Open that file, allow the unsigned bundle, and add the plugin's configuration after its entry: ```toml @@ -129,7 +150,8 @@ Register the plugin for your user, then run an agent through the Relay CLI: endpoint = "http://127.0.0.1:8100" ``` - Keep `[plugins.dynamic.config]` directly after the plugin's `[[plugins.dynamic]]` entry, because TOML attaches it to the entry above it. + Replace the example `manifest` value with the absolute path to your bundle's `relay-plugin.toml`. + Keep `[plugins.dynamic.config]` directly after the plugin's `[[plugins.dynamic]]` entry because TOML attaches it to that entry. Without the `attestation` override, Relay refuses to start, with an error that the plugin requires `integrity.signature` under host policy. 1. Enable the plugin, and validate its configuration: @@ -138,9 +160,10 @@ Register the plugin for your user, then run an agent through the Relay CLI: nemo-relay plugins validate nvidia.nemo_guardrails ``` - `plugins validate` checks the configuration against the plugin's schema: the field types, and that `endpoint` is set. + `plugins validate` checks the configuration against the plugin's schema, including field types and the required `endpoint` field. It does not check the `attestation` override, the endpoint's URL, or the header names. - The plugin checks the URL and the headers when Relay loads it, and Relay refuses to start when either is not valid. + The plugin checks the URL and headers when Relay loads it. + Relay refuses to start if either is invalid. 1. Run the agent through Relay: ```bash @@ -152,9 +175,9 @@ Register the plugin for your user, then run an agent through the Relay CLI: For pi, run `nemo-relay pi`, and also set the following: - `NEMO_RELAY_PI_FAIL=closed` in pi's environment. - By default, Relay's pi extension lets a call run when the check fails. -- `timeout_millis` below pi's own limit, which `NEMO_RELAY_PI_TIMEOUT_MS` sets and which is 5000 milliseconds by default. - Otherwise pi stops waiting before the plugin does. + By default, Relay's standalone pi extension lets a call run when the check fails. +- `timeout_millis` below the extension's `NEMO_RELAY_PI_TIMEOUT_MS` limit, which defaults to 5,000 milliseconds. + Otherwise, pi stops waiting before the plugin does. Anyone who can edit your user `plugins.toml` can remove the plugin or point it at another hooks server. To enforce the checks for every user, refer to [Run the Plugin in Production](#run-the-plugin-in-production). @@ -165,7 +188,12 @@ With the hooks server running the Claude Code example configuration, check the p 1. Ask Claude to run `echo 'rm -rf /tmp/guardrails-hook-check'`. The example configuration blocks the command because its text matches the `rm -rf` pattern. - Claude Code blocks the call, and Claude receives the reason, `guardrail rejected: NeMo Guardrails blocked tool call 'Bash': regex check tool call`. + Claude Code blocks the call and returns the following reason to Claude: + + ```text + guardrail rejected: NeMo Guardrails blocked tool call 'Bash': regex check tool call + ``` + The hooks server logs the check: ```text @@ -189,39 +217,46 @@ Set up the daemon as described in the NeMo Relay [daemon documentation](https:// sudo nemo-relay plugins add --global /opt/nemo-guardrails-relay-plugin/relay-plugin.toml ``` -1. Add the `attestation` override and the plugin's configuration to `/etc/nemo-relay/plugins.toml`, as in [Try the Plugin Out](#try-the-plugin-out), with the endpoint of the hooks server that serves your users. - Every local user can read this file, so do not put a credential in `headers`. +1. Add the `attestation` override and plugin configuration to `/etc/nemo-relay/plugins.toml`, as described in [Try the Plugin Out](#try-the-plugin-out). + Set `endpoint` to the hooks server that serves your users. + Every local user can read this file, so do not put credentials in `headers`. 1. Enable the plugin, and validate its configuration: ```bash sudo nemo-relay plugins enable nvidia.nemo_guardrails - nemo-relay plugins validate nvidia.nemo_guardrails + sudo nemo-relay plugins validate nvidia.nemo_guardrails ``` Enable the plugin only after you add its override and configuration, because Relay refuses to start with the plugin enabled and either one missing. + Validating a registered plugin writes its lifecycle state, so system registrations require administrator permissions. With a [remote daemon](https://github.com/NVIDIA/NeMo-Relay/blob/0.10.4/docs/daemon/remote-server.mdx), the workers run on the client machines, not on the daemon host. -Repeat these steps on every client machine, with an endpoint that the client can reach, and run `nemo-relay plugins validate nvidia.nemo_guardrails` there. +Repeat these steps on every client machine with an endpoint that the client can reach. +On each client, run `sudo nemo-relay plugins validate nvidia.nemo_guardrails`. A plugin installed only on the daemon host checks no calls from remote workers. The daemon enforces the checks only when the following conditions hold: -- **Start the daemon with `--require-worker`.** - Without it, the daemon answers hooks itself, and lets the call run, while no worker is ready: during worker startup and recovery, and after a daemon restart. -- **Deliver the agent's hook settings through its managed settings.** +- Start the daemon with `--require-worker`. + Without it, the daemon allows calls while no worker is ready, including during worker startup and recovery or after a daemon restart. +- Deliver the agent's hook settings through its managed settings. Otherwise a user can remove the hooks that send tool calls to the daemon. - Claude Code has managed settings, and Codex has managed hooks in `requirements.toml`, which users cannot disable, as described in [Codex: Deploy the Hook with Managed Settings](/run-guardrailed-inference/using-hooks-server/codex#deploy-the-hook-with-managed-settings). - Relay's managed pi extension takes effect only when pi starts with `--no-extensions` and the extension's path, and the user controls the command that starts pi. -- **Do not load a plugin that intercepts model execution, such as Switchyard.** - The daemon's worker refuses to start with one, and with `--require-worker`, the agents then stop working. + Claude Code has managed settings, and Codex has managed hooks in `requirements.toml` that users cannot disable. + Refer to [Codex: Deploy the Hook with Managed Settings](/run-guardrailed-inference/using-hooks-server/codex#deploy-the-hook-with-managed-settings). + Relay's managed pi extension takes effect only when pi starts with `--no-extensions` and the extension's path. + The user controls the command that starts pi. + +The daemon's managed pi extension blocks tool calls when a check fails and uses a fixed 30-second hook timeout. +The `NEMO_RELAY_PI_FAIL` and `NEMO_RELAY_PI_TIMEOUT_MS` settings apply to the standalone pi extension, not the managed extension. To stage a deployment, first run a daemon for your own user on one machine, as a macOS LaunchAgent or a Linux user service. ## Use the Plugin in a Python Application A Python application loads the plugin from a `plugins.toml` it passes to Relay. -Relay's Python package needs Python 3.11 or later. -Install it from the same 0.10 release as the rest of your Relay setup, with the `langchain` extra if you use Relay's LangChain middleware: +Relay's Python package requires Python 3.11 or later. +Install the same 0.10 release as the rest of your Relay setup. +Include the optional `langchain` extra if you use Relay's LangChain middleware: ```bash pip install "nemo-relay[langchain]>=0.10.1,<0.11" @@ -243,7 +278,8 @@ manifest = "/home/user/nemo-guardrails-relay-plugin/relay-plugin.toml" endpoint = "http://127.0.0.1:8100" ``` -A relative `manifest` path also works, and resolves from the directory of the `plugins.toml` file, for example when the bundle ships beside your application. +Replace `manifest` with the absolute path to your bundle's `relay-plugin.toml`. +A relative `manifest` path resolves from the directory that contains `plugins.toml`, for example when the bundle ships with your application. Activate Relay's plugins with the file, check that the plugin loaded, and run the agent while the plugins are active: @@ -266,10 +302,12 @@ async def main() -> None: asyncio.run(main()) ``` -When the `plugins.toml` file you pass does not exist, Relay only adds a `plugin.configuration_file_missing` warning to the activation report, and activates without the plugin, so every tool runs unchecked. -The example builds the path from the script's location, so that it does not depend on the working directory, and stops when the plugin is not loaded. +If the `plugins.toml` file does not exist, Relay adds a `plugin.configuration_file_missing` warning and continues activation. +Without another configuration that loads the plugin, tool calls run unchecked. +The example resolves the path from the script's location and stops if the plugin is not loaded. -Relay also reads the system `plugins.toml`, `/etc/nemo-relay/plugins.toml` on Linux and macOS, and its settings take precedence over the file you pass. +Relay also reads the system `plugins.toml`, which is `/etc/nemo-relay/plugins.toml` on Linux and macOS. +Its settings take precedence over the file you pass. For a LangChain or Deep Agents agent, add Relay's middleware, which runs each tool call through Relay: @@ -287,7 +325,8 @@ RuntimeError: guardrail rejected: NeMo Guardrails blocked tool call 'search': re ``` A failed check raises a `RuntimeError` whose message starts with `internal error: NeMo Guardrails check failed for tool call`. -Neither error reaches the model, so handle them where your application handles the agent's errors. +Neither error reaches the model. +Handle both in your application's error handler. A Python application has no hook time limit, so the default `timeout_millis` leaves room for a rail that calls a model. @@ -297,11 +336,11 @@ The plugin reads its configuration from the `[plugins.dynamic.config]` table of | Field | Default | Description | |---|---|---| -| `endpoint` | None | Required. The base URL of the hooks server, using `http` or `https`, such as `http://127.0.0.1:8100`. The plugin sends each check to `/v1/hooks/nemo-relay` under it, keeping any path prefix. It must not contain a username or password; send credentials in `headers`. | -| `headers` | `{}` | Headers to send with every check, as header names and string values. The hooks server has no authentication, so use them for a proxy or gateway in front of it, such as an `authorization` header. The plugin adds no header value to its own errors, but a failed check repeats the server's error message, which a proxy or gateway could fill with one; refer to [What Happens to a Call](#what-happens-to-a-call). It rejects headers that its HTTP client sets, such as `content-type`, `content-length`, and `host`. | -| `timeout_millis` | `25000` | The time limit for one check, in milliseconds. Keep it below the agent's hook time limit: 30 seconds for Claude Code and Codex through the Relay CLI, and 5 seconds by default for pi. | +| `endpoint` | None | Required. The base URL of the hooks server, using `http` or `https`, such as `http://127.0.0.1:8100`. The plugin sends each check to `/v1/hooks/nemo-relay` under it, keeping any path prefix. The URL must not contain a username or password. Send credentials in `headers`. | +| `headers` | `{}` | Header names and string values to send with every check. Use headers to authenticate with a proxy or gateway, such as through an `authorization` header. The hooks server itself has no authentication. The plugin rejects headers that its HTTP client controls, including `content-type`, `content-length`, and `host`. For error-message handling, refer to [What Happens to a Call](#what-happens-to-a-call). | +| `timeout_millis` | `25000` | A positive integer that sets the time limit for one check, in milliseconds. Keep it below the integration's hook timeout. Claude Code, Codex, and managed pi use 30 seconds. Standalone pi defaults to five seconds. | | `priority` | `0` | The plugin's place among Relay's tool guardrails. Lower values run earlier. | -| `executor` | None | Reserved for the Relay SDK's executor settings, such as `worker_threads`. | +| `executor` | None | Optional Relay software development kit (SDK) executor settings. `worker_threads` must be a positive integer. | The following example sends an `authorization` header to a proxy in front of the hooks server: @@ -320,7 +359,8 @@ Relay refuses to start when the configuration is not valid. The plugin uses the proxy settings of the Relay process that loads it, from `HTTP_PROXY`, `HTTPS_PROXY`, `ALL_PROXY`, and `NO_PROXY`, even for a hooks server on `127.0.0.1`. If you set a proxy, add the hooks server's host to `NO_PROXY`, such as `NO_PROXY=127.0.0.1,localhost`. -Otherwise every check goes through the proxy, which then receives the tool's arguments and the configured `headers`, in plain text for an `http` endpoint. +Otherwise, checks go through the proxy, which receives the tool's arguments and configured `headers`. +An `http` endpoint sends these values in plain text. @@ -328,11 +368,13 @@ Otherwise every check goes through the proxy, which then receives the tool's arg The plugin turns the hooks server's answer into the guardrail's result, and Relay acts on it: -| Hooks Server Answer | Plugin Result | Relay CLI with Claude Code or Codex | Relay CLI with pi | Python Application | +| Hooks Server Answer | Plugin Result | Relay CLI with Claude Code or Codex | Standalone pi Extension | Python Application | |---|---|---|---|---| | Allow | Lets the call run | The tool runs. Through Relay 0.10, Relay also approves Claude Code's permission prompt for the call, so Claude Code does not ask the user. Codex applies its own approval settings. | The tool runs | The tool runs | | Deny | A rejection: `NeMo Guardrails blocked tool call '': ` | Relay's hook exits `2`, and the agent blocks the call | pi blocks the call | `RuntimeError: guardrail rejected: ...` | -| An error status, no answer within `timeout_millis`, or no connection | An error: `NeMo Guardrails check failed for tool call '': ` | Relay's hook exits `2`, and the agent blocks the call | The tool runs, unless `NEMO_RELAY_PI_FAIL=closed` | `RuntimeError: internal error: ...` | +| A status other than `200`, an invalid decision body, a timeout, or a failed connection | An error: `NeMo Guardrails check failed for tool call '': ` | Relay's hook exits `2`, and the agent blocks the call | The tool runs, unless `NEMO_RELAY_PI_FAIL=closed` | `RuntimeError: internal error: ...` | + +The managed pi extension blocks both denied calls and failed checks. The reason for a deny comes from the hooks server: the tool policy's reason, or the name of the per-tool rail that blocked the call. For every deny reason and error, refer to [Hooks Server Results and Errors](/run-guardrailed-inference/using-hooks-server/results-and-errors). @@ -340,9 +382,10 @@ For every deny reason and error, refer to [Hooks Server Results and Errors](/run The detail of a failed check gives the HTTP status, the hooks server's error code and message, or says that the request timed out or could not be sent. The plugin adds no header value or tool argument to it, but it repeats the server's error message, up to 512 characters. A proxy or gateway in front of the hooks server could echo a header or an argument in that message. -The plugin does not retry a failed check, not even after a `503` with `Retry-After`. +The plugin does not retry failed checks, including a `503` response with `Retry-After`. -While the hooks server is stopped, restarting, or slower than `timeout_millis`, every check fails, so every tool call the plugin checks is blocked, except under pi without `NEMO_RELAY_PI_FAIL=closed`. +If the hooks server is stopped, restarting, or slower than `timeout_millis`, every check fails. +The integration blocks these tool calls unless you use the standalone pi extension without `NEMO_RELAY_PI_FAIL=closed`. The same holds for the calls a per-tool rail checks while the model it calls cannot be reached. ### Find the Reason for a Failed Check @@ -355,26 +398,28 @@ nemo-relay hook forward failed: hook gateway returned HTTP 500 Internal Server E Through the managed daemon, the agent can receive only Relay's own message, such as `daemon hook forward failed with HTTP 500`. -The plugin also logs each failed check as a warning in Relay's log, with the target `nemo_relay.plugin.native.nvidia.nemo_guardrails`, the event `nemo_guardrails_check_failed`, and the fields `tool_name`, `status`, and `error_code`: +The plugin also logs each failed check as a warning in Relay's log. +The warning uses the target `nemo_relay.plugin.native.nvidia.nemo_guardrails`, the event `nemo_guardrails_check_failed`, and the fields `tool_name`, `status`, and `error_code`: ```text 2026-10-28T18:09:36.742Z WARN root=01a183b3 target=nemo_relay.plugin.native.nvidia.nemo_guardrails event=nemo_guardrails_check_failed error_code=None status=None tool_name=Bash NeMo Guardrails check failed for tool call 'Bash': error sending request for url (http://127.0.0.1:8100/v1/hooks/nemo-relay): client error (Connect): tcp connect error: Connection refused (os error 61) ``` -Relay logs only errors by default, so set its log level to `warn` or lower to keep these warnings, for example with `nemo-relay --log-level warn`. +Relay logs only errors by default. +Set its log level to `warn` or a more detailed level to retain these warnings, for example with `nemo-relay --log-level warn`. The hooks server [logs each check that ends in an error](/run-guardrailed-inference/using-hooks-server/run-hooks-server#logs) at the `WARNING` level, with its error code. ## Limits Plan your deployment around the following limits: -- **No result checks.** +- No result checks. The hooks server checks tool calls only, so a tool's output reaches the model unchecked. -- **One configuration for each Relay process.** +- One configuration for each Relay process. Relay loads the plugin once in each process, with one `endpoint`, and a hooks server serves one configuration. To apply another policy, run another hooks server, and point another Relay setup at it. -- **No authentication on the hooks server.** - Anyone who can reach the hooks server can send it calls, and learn what the policy allows. +- No authentication on the hooks server. + Anyone who can reach the hooks server can send it calls and learn what the policy allows. Refer to [Run the Hooks Server: Expose the Server](/run-guardrailed-inference/using-hooks-server/run-hooks-server#expose-the-server). ## Related Topics diff --git a/docs/run-rails/using-hooks-server/nemo-relay.mdx b/docs/run-rails/using-hooks-server/nemo-relay.mdx index a9499906b1..6abaaeac72 100644 --- a/docs/run-rails/using-hooks-server/nemo-relay.mdx +++ b/docs/run-rails/using-hooks-server/nemo-relay.mdx @@ -10,7 +10,7 @@ content: [NeMo Relay](https://github.com/NVIDIA/NeMo-Relay) plugins can register a tool conditional-execution guardrail, which Relay runs before each tool call it manages, with the tool's name and arguments. A plugin can forward that check to the Guardrails hooks server (`nemoguardrails hooks-server`) at `/v1/hooks/nemo-relay`, so the hooks server's tool policy and per-tool rails decide whether the call runs. This page describes the endpoint such a plugin calls. -To install the NVIDIA NeMo Guardrails Plugin for NeMo Relay, which calls it, refer to [Check NeMo Relay Tool Calls with the NVIDIA NeMo Guardrails Plugin](/run-guardrailed-inference/using-hooks-server/nemo-relay-plugin). +To install the plugin that calls this endpoint, refer to [Check NeMo Relay Tool Calls with the NVIDIA NeMo Guardrails Plugin](/run-guardrailed-inference/using-hooks-server/nemo-relay-plugin). For how Relay runs these guardrails, refer to [conditional execution](https://github.com/NVIDIA/NeMo-Relay/blob/494018ac4d6abb0085853a02f4b6d75a75ff7049/docs/about-nemo-relay/concepts/middleware.mdx?plain=1#L235) in the NeMo Relay documentation. @@ -75,16 +75,18 @@ The plugin returns the hooks server's answer as the guardrail callback's result: |---|---|---| | `200` with `"decision": "allow"` | `Ok(None)` | Runs the tool | | `200` with `"decision": "deny"` | `Ok(Some(reason))` | Rejects the call with the reason | -| Any other status, a timeout, or a failed connection | `Err` | Treats the guardrail as failed; what happens to the call depends on the integration | +| Any other status, an invalid decision body, a timeout, or a failed connection | `Err` | Treats the guardrail as failed; what happens to the call depends on the integration | Map every failure to `Err`, never to `Ok(None)`, so the plugin never reports a check that could not finish as an allow. What Relay then does with the call depends on the integration that runs the agent: - Relay's Claude Code and Codex integrations block the call on `Err` from Relay 0.10.1. - Through the Relay CLI's gateway, the agent sees the plugin's error inside Relay's message; through the managed Relay daemon, it sees only Relay's own message. + Through the Relay CLI's gateway, the agent receives the plugin's error inside Relay's message. + Through the managed Relay daemon, the agent receives only Relay's own message. Refer to [Check NeMo Relay Tool Calls with the NVIDIA NeMo Guardrails Plugin: Find the Reason for a Failed Check](/run-guardrailed-inference/using-hooks-server/nemo-relay-plugin#find-the-reason-for-a-failed-check). -- Relay's pi extension treats `Err` as a fault, and by default, with `NEMO_RELAY_PI_FAIL=open`, lets the call run. - To block it, change that setting as described in the NeMo Relay documentation. +- Relay's standalone pi extension treats `Err` as a fault and lets the call run by default, with `NEMO_RELAY_PI_FAIL=open`. + Set `NEMO_RELAY_PI_FAIL=closed` to block the call. +- Relay's managed pi extension blocks the call on `Err` and does not use `NEMO_RELAY_PI_FAIL`. ## Tool Names diff --git a/docs/run-rails/using-hooks-server/results-and-errors.mdx b/docs/run-rails/using-hooks-server/results-and-errors.mdx index cbb305758e..7f1a53a583 100644 --- a/docs/run-rails/using-hooks-server/results-and-errors.mdx +++ b/docs/run-rails/using-hooks-server/results-and-errors.mdx @@ -118,7 +118,7 @@ The following table describes how to block tool calls when a check fails: |---|---|---| | Claude Code | Is a non-blocking error, and the tool runs | Set `onFailure: "block"` on the hook. Refer to [Claude Code: Add the Hook](/run-guardrailed-inference/using-hooks-server/claude-code#add-the-hook). | | Codex | Lets the tool run, unless the hook command exits `2`; a hook that times out lets it run too | Pass `curl` the `-f` option and a `--max-time` below the hook's `timeout`, and end the command with `\|\| exit 2`. Refer to [Codex: Add the Hook](/run-guardrailed-inference/using-hooks-server/codex#add-the-hook). | -| NeMo Relay | Depends on the plugin and the integration | The plugin returns `Err` from the guardrail callback for any failed request, and Relay's Claude Code and Codex integrations block the call on `Err` from Relay 0.10.1. Relay's pi extension lets the call run by default. Refer to [Check NeMo Relay Tool Calls with the NVIDIA NeMo Guardrails Plugin: What Happens to a Call](/run-guardrailed-inference/using-hooks-server/nemo-relay-plugin#what-happens-to-a-call). | +| NeMo Relay | Depends on the plugin and the integration | The plugin returns `Err` from the guardrail callback for a failed check. From Relay 0.10.1, the Claude Code and Codex integrations block the call on `Err`. The managed pi extension also blocks it. The standalone pi extension lets it run unless you set `NEMO_RELAY_PI_FAIL=closed`. Refer to [Check NeMo Relay Tool Calls with the NVIDIA NeMo Guardrails Plugin: What Happens to a Call](/run-guardrailed-inference/using-hooks-server/nemo-relay-plugin#what-happens-to-a-call). | These instructions apply to `PreToolUse` only. Other events have different failure behavior. For example, Claude Code ignores `onFailure` on its `Stop` event.