diff --git a/crates/nemo-guardrails-relay-plugin/README.md b/crates/nemo-guardrails-relay-plugin/README.md index b469e92f44..e87af968f1 100644 --- a/crates/nemo-guardrails-relay-plugin/README.md +++ b/crates/nemo-guardrails-relay-plugin/README.md @@ -8,6 +8,8 @@ Before each tool call that Relay manages, the plugin posts the tool's name and a A deny from the service becomes a guardrail rejection, and a check that fails (an error status, a timeout, a service that can't be reached) becomes an error. Relay blocks the call on either, from 0.10.1 on, with one exception: on an error, pi's extension lets the call run unless `NEMO_RELAY_PI_FAIL=closed` is set. +To install and configure the plugin, refer to [Check NeMo Relay Tool Calls with the NVIDIA NeMo Guardrails Plugin](./../../docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx). + ## Checks The plugin registers one tool conditional-execution guardrail, `nemo_guardrails_tool_call_check`, at the configured `priority`. diff --git a/docs/index.yml b/docs/index.yml index 5f2a1c7298..b92e1ad945 100644 --- a/docs/index.yml +++ b/docs/index.yml @@ -545,7 +545,10 @@ navigation: - page: Codex path: run-rails/using-hooks-server/codex.mdx slug: codex - - page: NeMo Relay + - page: NeMo Relay Plugin + path: run-rails/using-hooks-server/nemo-relay-plugin.mdx + slug: nemo-relay-plugin + - page: NeMo Relay Endpoint path: run-rails/using-hooks-server/nemo-relay.mdx slug: nemo-relay - page: Results and Errors diff --git a/docs/reference/hooks-server-api/nemo-relay.mdx b/docs/reference/hooks-server-api/nemo-relay.mdx index 12e95dbbf8..305d6080a9 100644 --- a/docs/reference/hooks-server-api/nemo-relay.mdx +++ b/docs/reference/hooks-server-api/nemo-relay.mdx @@ -49,7 +49,7 @@ The hooks server applies its tool policy, then the called tool's per-tool rails. The plugin returns the decision as its guardrail callback's result: `Ok(None)` for an allow, `Ok(Some(reason))` for a deny, and `Err` for any error status, timeout, or failed connection. The request examples come from the [NeMo Relay example configuration](https://github.com/NVIDIA-NeMo/Guardrails/tree/develop/examples/configs/agent_hooks/nemo_relay), which allows a `search` tool with only a `query` argument. -For the plugin side, refer to [Check NeMo Relay Tool Calls with the Hooks Server](/run-guardrailed-inference/using-hooks-server/nemo-relay). +For the plugin that calls this endpoint, refer to [Check NeMo Relay Tool Calls with the NVIDIA NeMo Guardrails Plugin](/run-guardrailed-inference/using-hooks-server/nemo-relay-plugin). ## Request diff --git a/docs/run-rails/using-hooks-server/index.mdx b/docs/run-rails/using-hooks-server/index.mdx index 11d2dcbdba..3c50f46cd8 100644 --- a/docs/run-rails/using-hooks-server/index.mdx +++ b/docs/run-rails/using-hooks-server/index.mdx @@ -43,7 +43,14 @@ Send every Codex tool call to the hooks server with a command hook. How To - + + +Check every tool call NeMo Relay manages with the NVIDIA NeMo Guardrails Plugin. + +How To + + + The endpoint a NeMo Relay plugin's tool guardrail calls. diff --git a/docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx b/docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx new file mode 100644 index 0000000000..c0e6fe340d --- /dev/null +++ b/docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx @@ -0,0 +1,431 @@ +--- +title: "Check NeMo Relay Tool Calls with the NVIDIA NeMo Guardrails Plugin" +sidebar-title: "NeMo Relay Plugin" +description: "Build, install, and configure the NVIDIA NeMo Guardrails Plugin for NeMo Relay to check tool calls before they run." +keywords: ["NeMo Relay", "NeMo Relay plugin", "nvidia.nemo_guardrails", "nemo-relay plugins add", "plugins.toml", "nemoguardrails hooks-server", "/v1/hooks/nemo-relay", "NemoRelayMiddleware"] +content: + type: "how_to" +--- + +The NVIDIA NeMo Guardrails Plugin for [NeMo Relay](https://github.com/NVIDIA/NeMo-Relay), `nvidia.nemo_guardrails`, checks each tool call Relay manages before the tool runs. +It sends the tool's name and arguments to the hooks server in the NVIDIA NeMo Guardrails library at `/v1/hooks/nemo-relay`. +The hooks server's tool policy and per-tool rails decide whether the call runs. +Configure the rails in the hooks server, which you start with `nemoguardrails hooks-server`. + +## Where the Plugin Checks Calls + +The plugin registers a tool conditional-execution guardrail, which Relay runs before each tool call it manages: + +| Where Relay Runs | Calls the Plugin Checks | +|---|---| +| The Relay CLI with Claude Code or Codex, in every mode | Each `PreToolUse` hook. Through Relay 0.10, also each `PermissionRequest` hook. Refer to [Check NeMo Relay Tool Calls with the Hooks Server: Check Each Call Once](/run-guardrailed-inference/using-hooks-server/nemo-relay#check-each-call-once). | +| The Relay CLI with pi | Each tool call that Relay's pi extension sends to Relay | +| A Python application | Each tool call through `nemo_relay.tools.execute`, the `NemoRelayMiddleware` middleware for LangChain and Deep Agents, or Relay's LangGraph tool node | +| OpenClaw | Not supported | + +The plugin checks tool calls only. +A tool's result reaches the model unchecked. + +## Prerequisites + +Before you install the plugin, meet the following prerequisites: + +- NeMo Relay 0.10.1 or a later 0.10 release. + Relay 0.10.0 refuses to load the plugin. In that release, an error from a guardrail lets a Claude Code tool call run. +- A running hooks server, with a configuration for the tool names of the agent or application you check. + Start the server from a clone of the repository with the configuration for your integration: + + - Claude Code, through `nemo-relay claude`: the [Claude Code example configuration](https://github.com/NVIDIA-NeMo/Guardrails/tree/develop/examples/configs/agent_hooks/claude_code). + - Codex, through `nemo-relay codex`: the [Codex example configuration](https://github.com/NVIDIA-NeMo/Guardrails/tree/develop/examples/configs/agent_hooks/codex). + - A Python application: the [NeMo Relay example configuration](https://github.com/NVIDIA-NeMo/Guardrails/tree/develop/examples/configs/agent_hooks/nemo_relay), which allows only a `search` tool. The Python examples on this page use it. + - pi: Write a configuration for pi's tool names. The repository has no pi example configuration. + + For example, to check Claude Code, run the following command: + + ```bash + nemoguardrails hooks-server --config examples/configs/agent_hooks/claude_code + ``` + + Relay forwards each tool's name as the agent or application gives it, and the hooks server matches it exactly. + For Claude Code or Codex, base the configuration on the [Claude Code](/run-guardrailed-inference/using-hooks-server/claude-code#claude-code-tool-names) or [Codex](/run-guardrailed-inference/using-hooks-server/codex#codex-tool-names) tool names. + To write your own configuration, refer to [Run the Hooks Server: Write a Hooks Configuration](/run-guardrailed-inference/using-hooks-server/run-hooks-server#write-a-hooks-configuration). +- A clone of the [NeMo Guardrails repository](https://github.com/NVIDIA-NeMo/Guardrails), [rustup](https://rustup.rs/), a C compiler, and Python 3 to build the plugin. + For the C compiler, install the Xcode Command Line Tools on macOS or `build-essential` on Debian and Ubuntu. + +## Build the Plugin Bundle + +A Relay plugin bundle contains the plugin library, its manifest `relay-plugin.toml`, its configuration schema, and its license files. +The library filename depends on your operating system: + +| Operating System | Library Filename | +|---|---| +| Linux | `libnemo_guardrails_relay_plugin.so` | +| macOS | `libnemo_guardrails_relay_plugin.dylib` | +| Windows | `nemo_guardrails_relay_plugin.dll` | + +Build the library and package the bundle from the repository root, passing your platform's library to `--library`. +On Linux, run the following commands: + +```bash +cargo build --release --locked -p nemo-guardrails-relay-plugin +python3 crates/nemo-guardrails-relay-plugin/scripts/package_bundle.py \ + --library target/release/libnemo_guardrails_relay_plugin.so \ + --output ~/nemo-guardrails-relay-plugin +``` + +On macOS, pass `--library target/release/libnemo_guardrails_relay_plugin.dylib` instead. +The build uses the Rust version that the repository's `rust-toolchain.toml` pins. + +Relay loads the plugin from the bundle directory, so keep the directory in place after you register it. +The bundle is not signed, so Relay loads it only after you allow integrity-only attestation for the plugin, as the following sections show. + +### Update the Plugin Bundle + +To update the plugin, replace the bundle at the registered path. +Relay records the manifest's path during registration and refuses to start if the path in `plugins.toml` differs from the registered path. +Replace the entire bundle because the packaging script writes the library's digest into the manifest. +Relay refuses to load a library that does not match that digest. + +Build the new library as described previously, then package it in an empty directory next to the current bundle. +The packaging script rejects an output directory that contains files. +Stop Relay before replacing the bundle, and ensure that the `.previous` backup path does not exist. +On Linux, run the following commands: + +```bash +python3 crates/nemo-guardrails-relay-plugin/scripts/package_bundle.py \ + --library target/release/libnemo_guardrails_relay_plugin.so \ + --output ~/nemo-guardrails-relay-plugin.new && + mv ~/nemo-guardrails-relay-plugin ~/nemo-guardrails-relay-plugin.previous && + mv ~/nemo-guardrails-relay-plugin.new ~/nemo-guardrails-relay-plugin +``` + +On macOS, use the `.dylib` library path instead. +Validate the replacement with `nemo-relay plugins validate nvidia.nemo_guardrails`, then restart Relay. +For a globally registered plugin, run validation with `sudo`. +Keep the `.previous` bundle until you verify the replacement. +If replacement or validation fails, restore the previous bundle at the registered path before restarting Relay. + +## Choose a Setup + +The plugin works the same way in every Relay mode. +How you run Relay decides who controls the plugin and its configuration: + +| Setting | Try It Out | Production | +|---|---|---| +| Relay mode | `nemo-relay claude`, `nemo-relay codex`, `nemo-relay pi`, or a persistent integration from `nemo-relay install` | The managed Relay daemon, `nemo-relay daemon` | +| Registered with | `nemo-relay plugins add --user` | `nemo-relay plugins add --global` | +| Configuration file | Your user `plugins.toml` | The system `plugins.toml` | +| Hooks server | Runs on your machine, started by you | Runs where an administrator deploys it | +| Controlled by | The user, who can remove or reconfigure the plugin | An administrator | + +A Python application passes its own `plugins.toml` to Relay. +Refer to [Use the Plugin in a Python Application](#use-the-plugin-in-a-python-application). + +## Try the Plugin Out + +Register the plugin for your user, then run an agent through the Relay CLI: + +1. Start the hooks server, as described in [Prerequisites](#prerequisites). +1. Validate the bundle, and register it for your user: + + ```bash + nemo-relay plugins validate ~/nemo-guardrails-relay-plugin/relay-plugin.toml + nemo-relay plugins add --user ~/nemo-guardrails-relay-plugin/relay-plugin.toml + ``` + + `plugins add` adds a `[[plugins.dynamic]]` entry to your user `plugins.toml` at `$XDG_CONFIG_HOME/nemo-relay/plugins.toml`. + If `XDG_CONFIG_HOME` is not set, the file is at `~/.config/nemo-relay/plugins.toml`. +1. Open that file, allow the unsigned bundle, and add the plugin's configuration after its entry: + + ```toml + version = 1 + + [plugins.policy.overrides."nvidia.nemo_guardrails"] + attestation = "integrity_only" + + [[plugins.dynamic]] + manifest = "/home/user/nemo-guardrails-relay-plugin/relay-plugin.toml" + + [plugins.dynamic.config] + endpoint = "http://127.0.0.1:8100" + ``` + + Replace the example `manifest` value with the absolute path to your bundle's `relay-plugin.toml`. + Keep `[plugins.dynamic.config]` directly after the plugin's `[[plugins.dynamic]]` entry because TOML attaches it to that entry. + Without the `attestation` override, Relay refuses to start, with an error that the plugin requires `integrity.signature` under host policy. +1. Enable the plugin, and validate its configuration: + + ```bash + nemo-relay plugins enable nvidia.nemo_guardrails + nemo-relay plugins validate nvidia.nemo_guardrails + ``` + + `plugins validate` checks the configuration against the plugin's schema, including field types and the required `endpoint` field. + It does not check the `attestation` override, the endpoint's URL, or the header names. + The plugin checks the URL and headers when Relay loads it. + Relay refuses to start if either is invalid. +1. Run the agent through Relay: + + ```bash + nemo-relay claude + ``` + + For Codex, run `nemo-relay codex` against a hooks server with a configuration for the Codex tool names. + +For pi, run `nemo-relay pi`, and also set the following: + +- `NEMO_RELAY_PI_FAIL=closed` in pi's environment. + By default, Relay's standalone pi extension lets a call run when the check fails. +- `timeout_millis` below the extension's `NEMO_RELAY_PI_TIMEOUT_MS` limit, which defaults to 5,000 milliseconds. + Otherwise, pi stops waiting before the plugin does. + +Anyone who can edit your user `plugins.toml` can remove the plugin or point it at another hooks server. +To enforce the checks for every user, refer to [Run the Plugin in Production](#run-the-plugin-in-production). + +### Verify the Plugin + +With the hooks server running the Claude Code example configuration, check the plugin from Claude Code: + +1. Ask Claude to run `echo 'rm -rf /tmp/guardrails-hook-check'`. + The example configuration blocks the command because its text matches the `rm -rf` pattern. + Claude Code blocks the call and returns the following reason to Claude: + + ```text + guardrail rejected: NeMo Guardrails blocked tool call 'Bash': regex check tool call + ``` + + The hooks server logs the check: + + ```text + INFO:nemoguardrails.server.hooks.app:Hook check on /v1/hooks/nemo-relay: event=NemoRelayToolConditionalExecutionGuardrail tool_name='Bash' tool_use_id=None outcome=deny code=None rail='regex check tool call' + ``` + +1. Stop the hooks server and ask Claude to run any command. + Claude Code blocks the call because the check fails. +1. Start the hooks server again. + +## Run the Plugin in Production + +In production, an administrator runs the managed Relay daemon. +Its workers load the plugin for every user from the system `plugins.toml` of the machine they run on. +Set up the daemon as described in the NeMo Relay [daemon documentation](https://github.com/NVIDIA/NeMo-Relay/blob/0.10.4/docs/daemon/about.mdx), then add the plugin: + +1. Put the bundle in a directory that administrators own and every user can read, such as `/opt/nemo-guardrails-relay-plugin`. +1. From an account that can write `/etc/nemo-relay`, register the plugin: + + ```bash + sudo nemo-relay plugins add --global /opt/nemo-guardrails-relay-plugin/relay-plugin.toml + ``` + +1. Add the `attestation` override and plugin configuration to `/etc/nemo-relay/plugins.toml`, as described in [Try the Plugin Out](#try-the-plugin-out). + Set `endpoint` to the hooks server that serves your users. + Every local user can read this file, so do not put credentials in `headers`. +1. Enable the plugin, and validate its configuration: + + ```bash + sudo nemo-relay plugins enable nvidia.nemo_guardrails + sudo nemo-relay plugins validate nvidia.nemo_guardrails + ``` + + Enable the plugin only after you add its override and configuration, because Relay refuses to start with the plugin enabled and either one missing. + Validating a registered plugin writes its lifecycle state, so system registrations require administrator permissions. + +With a [remote daemon](https://github.com/NVIDIA/NeMo-Relay/blob/0.10.4/docs/daemon/remote-server.mdx), the workers run on the client machines, not on the daemon host. +Repeat these steps on every client machine with an endpoint that the client can reach. +On each client, run `sudo nemo-relay plugins validate nvidia.nemo_guardrails`. +A plugin installed only on the daemon host checks no calls from remote workers. + +The daemon enforces the checks only when the following conditions hold: + +- Start the daemon with `--require-worker`. + Without it, the daemon allows calls while no worker is ready, including during worker startup and recovery or after a daemon restart. +- Deliver the agent's hook settings through its managed settings. + Otherwise a user can remove the hooks that send tool calls to the daemon. + Claude Code has managed settings, and Codex has managed hooks in `requirements.toml` that users cannot disable. + Refer to [Codex: Deploy the Hook with Managed Settings](/run-guardrailed-inference/using-hooks-server/codex#deploy-the-hook-with-managed-settings). + Relay's managed pi extension takes effect only when pi starts with `--no-extensions` and the extension's path. + The user controls the command that starts pi. + +The daemon's managed pi extension blocks tool calls when a check fails and uses a fixed 30-second hook timeout. +The `NEMO_RELAY_PI_FAIL` and `NEMO_RELAY_PI_TIMEOUT_MS` settings apply to the standalone pi extension, not the managed extension. + +To stage a deployment, first run a daemon for your own user on one machine, as a macOS LaunchAgent or a Linux user service. + +## Use the Plugin in a Python Application + +A Python application loads the plugin from a `plugins.toml` it passes to Relay. +Relay's Python package requires Python 3.11 or later. +Install the same 0.10 release as the rest of your Relay setup. +Include the optional `langchain` extra if you use Relay's LangChain middleware: + +```bash +pip install "nemo-relay[langchain]>=0.10.1,<0.11" +``` + +Write the `plugins.toml` file with the same entries as in [Try the Plugin Out](#try-the-plugin-out). +Point `manifest` at the bundle you built: + +```toml +version = 1 + +[plugins.policy.overrides."nvidia.nemo_guardrails"] +attestation = "integrity_only" + +[[plugins.dynamic]] +manifest = "/home/user/nemo-guardrails-relay-plugin/relay-plugin.toml" + +[plugins.dynamic.config] +endpoint = "http://127.0.0.1:8100" +``` + +Replace `manifest` with the absolute path to your bundle's `relay-plugin.toml`. +A relative `manifest` path resolves from the directory that contains `plugins.toml`, for example when the bundle ships with your application. + +Activate Relay's plugins with the file, check that the plugin loaded, and run the agent while the plugins are active: + +```python +import asyncio +from pathlib import Path + +from nemo_relay import plugin + + +async def main() -> None: + async with plugin.activate({}, Path(__file__).with_name("plugins.toml")) as activation: + report = activation.report + loaded = {entry["plugin_id"] for entry in report["dynamic_plugins"] if entry["selected"]} + if "nvidia.nemo_guardrails" not in loaded: + raise RuntimeError(f"NeMo Guardrails plugin not loaded: {report['config']['diagnostics']}") + ... # Run your agent here. + + +asyncio.run(main()) +``` + +If the `plugins.toml` file does not exist, Relay adds a `plugin.configuration_file_missing` warning and continues activation. +Without another configuration that loads the plugin, tool calls run unchecked. +The example resolves the path from the script's location and stops if the plugin is not loaded. + +Relay also reads the system `plugins.toml`, which is `/etc/nemo-relay/plugins.toml` on Linux and macOS. +Its settings take precedence over the file you pass. + +For a LangChain or Deep Agents agent, add Relay's middleware, which runs each tool call through Relay: + +```python +from langchain.agents import create_agent +from nemo_relay.integrations.langchain.middleware import NemoRelayMiddleware + +agent = create_agent(model=model, tools=tools, middleware=[NemoRelayMiddleware()]) +``` + +When the hooks server denies a call, `invoke` and `ainvoke` raise a `RuntimeError`, and the tool does not run: + +```text +RuntimeError: guardrail rejected: NeMo Guardrails blocked tool call 'search': regex check tool call +``` + +A failed check raises a `RuntimeError` whose message starts with `internal error: NeMo Guardrails check failed for tool call`. +Neither error reaches the model. +Handle both in your application's error handler. + +A Python application has no hook time limit, so the default `timeout_millis` leaves room for a rail that calls a model. + +## Configure the Plugin + +The plugin reads its configuration from the `[plugins.dynamic.config]` table of its entry: + +| Field | Default | Description | +|---|---|---| +| `endpoint` | None | Required. The base URL of the hooks server, using `http` or `https`, such as `http://127.0.0.1:8100`. The plugin sends each check to `/v1/hooks/nemo-relay` under it, keeping any path prefix. The URL must not contain a username or password. Send credentials in `headers`. | +| `headers` | `{}` | Header names and string values to send with every check. Use headers to authenticate with a proxy or gateway, such as through an `authorization` header. The hooks server itself has no authentication. The plugin rejects headers that its HTTP client controls, including `content-type`, `content-length`, and `host`. For error-message handling, refer to [What Happens to a Call](#what-happens-to-a-call). | +| `timeout_millis` | `25000` | A positive integer that sets the time limit for one check, in milliseconds. Keep it below the integration's hook timeout. Claude Code, Codex, and managed pi use 30 seconds. Standalone pi defaults to five seconds. | +| `priority` | `0` | The plugin's place among Relay's tool guardrails. Lower values run earlier. | +| `executor` | None | Optional Relay software development kit (SDK) executor settings. `worker_threads` must be a positive integer. | + +The following example sends an `authorization` header to a proxy in front of the hooks server: + +```toml +[plugins.dynamic.config] +endpoint = "https://guardrails.example.com" +timeout_millis = 10000 + +[plugins.dynamic.config.headers] +authorization = "Bearer " +``` + +Relay refuses to start when the configuration is not valid. + + + +The plugin uses the proxy settings of the Relay process that loads it, from `HTTP_PROXY`, `HTTPS_PROXY`, `ALL_PROXY`, and `NO_PROXY`, even for a hooks server on `127.0.0.1`. +If you set a proxy, add the hooks server's host to `NO_PROXY`, such as `NO_PROXY=127.0.0.1,localhost`. +Otherwise, checks go through the proxy, which receives the tool's arguments and configured `headers`. +An `http` endpoint sends these values in plain text. + + + +## What Happens to a Call + +The plugin turns the hooks server's answer into the guardrail's result, and Relay acts on it: + +| Hooks Server Answer | Plugin Result | Relay CLI with Claude Code or Codex | Standalone pi Extension | Python Application | +|---|---|---|---|---| +| Allow | Lets the call run | The tool runs. Through Relay 0.10, Relay also approves Claude Code's permission prompt for the call, so Claude Code does not ask the user. Codex applies its own approval settings. | The tool runs | The tool runs | +| Deny | A rejection: `NeMo Guardrails blocked tool call '': ` | Relay's hook exits `2`, and the agent blocks the call | pi blocks the call | `RuntimeError: guardrail rejected: ...` | +| A status other than `200`, an invalid decision body, a timeout, or a failed connection | An error: `NeMo Guardrails check failed for tool call '': ` | Relay's hook exits `2`, and the agent blocks the call | The tool runs, unless `NEMO_RELAY_PI_FAIL=closed` | `RuntimeError: internal error: ...` | + +The managed pi extension blocks both denied calls and failed checks. + +The reason for a deny comes from the hooks server: the tool policy's reason, or the name of the per-tool rail that blocked the call. +For every deny reason and error, refer to [Hooks Server Results and Errors](/run-guardrailed-inference/using-hooks-server/results-and-errors). + +The detail of a failed check gives the HTTP status, the hooks server's error code and message, or says that the request timed out or could not be sent. +The plugin adds no header value or tool argument to it, but it repeats the server's error message, up to 512 characters. +A proxy or gateway in front of the hooks server could echo a header or an argument in that message. +The plugin does not retry failed checks, including a `503` response with `Retry-After`. + +If the hooks server is stopped, restarting, or slower than `timeout_millis`, every check fails. +The integration blocks these tool calls unless you use the standalone pi extension without `NEMO_RELAY_PI_FAIL=closed`. +The same holds for the calls a per-tool rail checks while the model it calls cannot be reached. + +### Find the Reason for a Failed Check + +Through the Relay CLI's gateway, the agent receives the plugin's error inside Relay's message: + +```text +nemo-relay hook forward failed: hook gateway returned HTTP 500 Internal Server Error: NeMo Relay runtime error: internal error: NeMo Guardrails check failed for tool call 'Bash': error sending request for url (http://127.0.0.1:8100/v1/hooks/nemo-relay): client error (Connect): tcp connect error: Connection refused (os error 61) +``` + +Through the managed daemon, the agent can receive only Relay's own message, such as `daemon hook forward failed with HTTP 500`. + +The plugin also logs each failed check as a warning in Relay's log. +The warning uses the target `nemo_relay.plugin.native.nvidia.nemo_guardrails`, the event `nemo_guardrails_check_failed`, and the fields `tool_name`, `status`, and `error_code`: + +```text +2026-10-28T18:09:36.742Z WARN root=01a183b3 target=nemo_relay.plugin.native.nvidia.nemo_guardrails event=nemo_guardrails_check_failed error_code=None status=None tool_name=Bash NeMo Guardrails check failed for tool call 'Bash': error sending request for url (http://127.0.0.1:8100/v1/hooks/nemo-relay): client error (Connect): tcp connect error: Connection refused (os error 61) +``` + +Relay logs only errors by default. +Set its log level to `warn` or a more detailed level to retain these warnings, for example with `nemo-relay --log-level warn`. +The hooks server [logs each check that ends in an error](/run-guardrailed-inference/using-hooks-server/run-hooks-server#logs) at the `WARNING` level, with its error code. + +## Limits + +Plan your deployment around the following limits: + +- No result checks. + The hooks server checks tool calls only, so a tool's output reaches the model unchecked. +- One configuration for each Relay process. + Relay loads the plugin once in each process, with one `endpoint`, and a hooks server serves one configuration. + To apply another policy, run another hooks server, and point another Relay setup at it. +- No authentication on the hooks server. + Anyone who can reach the hooks server can send it calls and learn what the policy allows. + Refer to [Run the Hooks Server: Expose the Server](/run-guardrailed-inference/using-hooks-server/run-hooks-server#expose-the-server). + +## Related Topics + +- [Check NeMo Relay Tool Calls with the Hooks Server](/run-guardrailed-inference/using-hooks-server/nemo-relay) +- [Run the Hooks Server](/run-guardrailed-inference/using-hooks-server/run-hooks-server) +- [Hooks Server Results and Errors](/run-guardrailed-inference/using-hooks-server/results-and-errors) +- [Configure Discoverable Plugins](https://github.com/NVIDIA/NeMo-Relay/blob/0.10.4/docs/configure-plugins/discoverable-plugins.mdx) in the NeMo Relay documentation +- [The plugin's README](https://github.com/NVIDIA-NeMo/Guardrails/tree/develop/crates/nemo-guardrails-relay-plugin), for building and testing it diff --git a/docs/run-rails/using-hooks-server/nemo-relay.mdx b/docs/run-rails/using-hooks-server/nemo-relay.mdx index 04d6f5dc88..6abaaeac72 100644 --- a/docs/run-rails/using-hooks-server/nemo-relay.mdx +++ b/docs/run-rails/using-hooks-server/nemo-relay.mdx @@ -1,6 +1,6 @@ --- title: "Check NeMo Relay Tool Calls with the Hooks Server" -sidebar-title: "NeMo Relay" +sidebar-title: "NeMo Relay Endpoint" description: "The /v1/hooks/nemo-relay endpoint of the Guardrails hooks server, for a NeMo Relay plugin's tool conditional-execution guardrail." keywords: ["NeMo Relay", "tool conditional-execution guardrail", "nemoguardrails hooks-server", "/v1/hooks/nemo-relay", "NemoRelayToolConditionalExecutionGuardrail"] content: @@ -10,6 +10,7 @@ content: [NeMo Relay](https://github.com/NVIDIA/NeMo-Relay) plugins can register a tool conditional-execution guardrail, which Relay runs before each tool call it manages, with the tool's name and arguments. A plugin can forward that check to the Guardrails hooks server (`nemoguardrails hooks-server`) at `/v1/hooks/nemo-relay`, so the hooks server's tool policy and per-tool rails decide whether the call runs. This page describes the endpoint such a plugin calls. +To install the plugin that calls this endpoint, refer to [Check NeMo Relay Tool Calls with the NVIDIA NeMo Guardrails Plugin](/run-guardrailed-inference/using-hooks-server/nemo-relay-plugin). For how Relay runs these guardrails, refer to [conditional execution](https://github.com/NVIDIA/NeMo-Relay/blob/494018ac4d6abb0085853a02f4b6d75a75ff7049/docs/about-nemo-relay/concepts/middleware.mdx?plain=1#L235) in the NeMo Relay documentation. @@ -74,15 +75,18 @@ The plugin returns the hooks server's answer as the guardrail callback's result: |---|---|---| | `200` with `"decision": "allow"` | `Ok(None)` | Runs the tool | | `200` with `"decision": "deny"` | `Ok(Some(reason))` | Rejects the call with the reason | -| Any other status, a timeout, or a failed connection | `Err` | Treats the guardrail as failed; what happens to the call depends on the integration | +| Any other status, an invalid decision body, a timeout, or a failed connection | `Err` | Treats the guardrail as failed; what happens to the call depends on the integration | Map every failure to `Err`, never to `Ok(None)`, so the plugin never reports a check that could not finish as an allow. What Relay then does with the call depends on the integration that runs the agent: - Relay's Claude Code and Codex integrations block the call on `Err` from Relay 0.10.1. - The agent sees Relay's own error text, not the plugin's. -- Relay's pi extension treats `Err` as a fault, and by default, with `NEMO_RELAY_PI_FAIL=open`, lets the call run. - To block it, change that setting as described in the NeMo Relay documentation. + Through the Relay CLI's gateway, the agent receives the plugin's error inside Relay's message. + Through the managed Relay daemon, the agent receives only Relay's own message. + Refer to [Check NeMo Relay Tool Calls with the NVIDIA NeMo Guardrails Plugin: Find the Reason for a Failed Check](/run-guardrailed-inference/using-hooks-server/nemo-relay-plugin#find-the-reason-for-a-failed-check). +- Relay's standalone pi extension treats `Err` as a fault and lets the call run by default, with `NEMO_RELAY_PI_FAIL=open`. + Set `NEMO_RELAY_PI_FAIL=closed` to block the call. +- Relay's managed pi extension blocks the call on `Err` and does not use `NEMO_RELAY_PI_FAIL`. ## Tool Names @@ -98,6 +102,7 @@ Account for the following sources of duplicate checks: - **Relay 0.10 checks some calls twice.** Through the 0.10 releases, Relay also runs the guardrail on Claude Code's and Codex's `PermissionRequest` event. A call that reaches a permission prompt then costs two requests to the hooks server, and two runs of any rail that calls a model. + When the second check allows the call, Relay also approves Claude Code's permission prompt, so Claude Code does not ask the user. ## Related Topics diff --git a/docs/run-rails/using-hooks-server/overview.mdx b/docs/run-rails/using-hooks-server/overview.mdx index 8c216c11e5..dc10061432 100644 --- a/docs/run-rails/using-hooks-server/overview.mdx +++ b/docs/run-rails/using-hooks-server/overview.mdx @@ -98,7 +98,7 @@ The hooks server supports the following agent integrations: |---|---|---|---| | [Claude Code](/run-guardrailed-inference/using-hooks-server/claude-code) | `/v1/hooks/claude-code` | An `http` hook on `PreToolUse` | 2.1.295, for `onFailure: "block"` | | [Codex](/run-guardrailed-inference/using-hooks-server/codex) | `/v1/hooks/codex` | A `command` hook on `PreToolUse` that runs `curl` | 0.124.0. From 0.129.0, Codex runs a hook only after you trust it | -| [NeMo Relay](/run-guardrailed-inference/using-hooks-server/nemo-relay) | `/v1/hooks/nemo-relay` | A plugin's tool conditional-execution guardrail | 0.10.1 | +| [NeMo Relay](/run-guardrailed-inference/using-hooks-server/nemo-relay-plugin) | `/v1/hooks/nemo-relay` | A plugin's tool conditional-execution guardrail | 0.10.1 | The hooks server handles only the `PreToolUse` event, and the NeMo Relay guardrail that stands in for it. Any other hook event returns HTTP `422`, so register the hooks server on `PreToolUse` only. @@ -184,7 +184,7 @@ The server listens on `http://127.0.0.1:8100`. } ``` -1. To check your agent's calls for real, connect it to the hooks server: [Claude Code](/run-guardrailed-inference/using-hooks-server/claude-code), [Codex](/run-guardrailed-inference/using-hooks-server/codex), or [NeMo Relay](/run-guardrailed-inference/using-hooks-server/nemo-relay). +1. To check your agent's calls for real, connect it to the hooks server: [Claude Code](/run-guardrailed-inference/using-hooks-server/claude-code), [Codex](/run-guardrailed-inference/using-hooks-server/codex), or [NeMo Relay](/run-guardrailed-inference/using-hooks-server/nemo-relay-plugin). ## Known Limits diff --git a/docs/run-rails/using-hooks-server/results-and-errors.mdx b/docs/run-rails/using-hooks-server/results-and-errors.mdx index e58452938f..7f1a53a583 100644 --- a/docs/run-rails/using-hooks-server/results-and-errors.mdx +++ b/docs/run-rails/using-hooks-server/results-and-errors.mdx @@ -118,7 +118,7 @@ The following table describes how to block tool calls when a check fails: |---|---|---| | Claude Code | Is a non-blocking error, and the tool runs | Set `onFailure: "block"` on the hook. Refer to [Claude Code: Add the Hook](/run-guardrailed-inference/using-hooks-server/claude-code#add-the-hook). | | Codex | Lets the tool run, unless the hook command exits `2`; a hook that times out lets it run too | Pass `curl` the `-f` option and a `--max-time` below the hook's `timeout`, and end the command with `\|\| exit 2`. Refer to [Codex: Add the Hook](/run-guardrailed-inference/using-hooks-server/codex#add-the-hook). | -| NeMo Relay | Depends on the plugin and the integration | The plugin returns `Err` from the guardrail callback for any failed request, and Relay's Claude Code and Codex integrations block the call on `Err` from Relay 0.10.1. Relay's pi extension lets the call run by default. Refer to [NeMo Relay: Map the Response to the Guardrail Result](/run-guardrailed-inference/using-hooks-server/nemo-relay#map-the-response-to-the-guardrail-result). | +| NeMo Relay | Depends on the plugin and the integration | The plugin returns `Err` from the guardrail callback for a failed check. From Relay 0.10.1, the Claude Code and Codex integrations block the call on `Err`. The managed pi extension also blocks it. The standalone pi extension lets it run unless you set `NEMO_RELAY_PI_FAIL=closed`. Refer to [Check NeMo Relay Tool Calls with the NVIDIA NeMo Guardrails Plugin: What Happens to a Call](/run-guardrailed-inference/using-hooks-server/nemo-relay-plugin#what-happens-to-a-call). | These instructions apply to `PreToolUse` only. Other events have different failure behavior. For example, Claude Code ignores `onFailure` on its `Stop` event. diff --git a/docs/run-rails/using-hooks-server/run-hooks-server.mdx b/docs/run-rails/using-hooks-server/run-hooks-server.mdx index 1c9e727a3b..c1154f3023 100644 --- a/docs/run-rails/using-hooks-server/run-hooks-server.mdx +++ b/docs/run-rails/using-hooks-server/run-hooks-server.mdx @@ -10,7 +10,7 @@ content: The Guardrails hooks server (`nemoguardrails hooks-server`) loads one guardrails configuration at startup. It checks each tool call against that configuration's tool policy and per-tool rails. Use this guide to configure, start, and monitor the server. -To connect an agent, refer to the [Claude Code](/run-guardrailed-inference/using-hooks-server/claude-code), [Codex](/run-guardrailed-inference/using-hooks-server/codex), or [NeMo Relay](/run-guardrailed-inference/using-hooks-server/nemo-relay) page. +To connect an agent, refer to the [Claude Code](/run-guardrailed-inference/using-hooks-server/claude-code), [Codex](/run-guardrailed-inference/using-hooks-server/codex), or [NeMo Relay](/run-guardrailed-inference/using-hooks-server/nemo-relay-plugin) page. ## Start the Server diff --git a/examples/configs/agent_hooks/README.md b/examples/configs/agent_hooks/README.md index 2ab7277c31..80f5e738b9 100644 --- a/examples/configs/agent_hooks/README.md +++ b/examples/configs/agent_hooks/README.md @@ -41,7 +41,7 @@ The example's `Bash` rail blocks the force push: {"hookSpecificOutput":{"hookEventName":"PreToolUse","permissionDecision":"deny","permissionDecisionReason":"NeMo Guardrails: regex check tool call"}} ``` -To point the agent's hook at the server, follow the agent's page: [Claude Code](./../../../docs/run-rails/using-hooks-server/claude-code.mdx), [Codex](./../../../docs/run-rails/using-hooks-server/codex.mdx), or [NeMo Relay](./../../../docs/run-rails/using-hooks-server/nemo-relay.mdx). +To point the agent's hook at the server, follow the agent's page: [Claude Code](./../../../docs/run-rails/using-hooks-server/claude-code.mdx), [Codex](./../../../docs/run-rails/using-hooks-server/codex.mdx), or [NeMo Relay](./../../../docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx). ## One Server for Each Agent