Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions crates/nemo-guardrails-relay-plugin/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ Before each tool call that Relay manages, the plugin posts the tool's name and a
A deny from the service becomes a guardrail rejection, and a check that fails (an error status, a timeout, a service that can't be reached) becomes an error.
Relay blocks the call on either, from 0.10.1 on, with one exception: on an error, pi's extension lets the call run unless `NEMO_RELAY_PI_FAIL=closed` is set.

To install and configure the plugin, refer to [Check NeMo Relay Tool Calls with the NVIDIA NeMo Guardrails Plugin](./../../docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx).

## Checks

The plugin registers one tool conditional-execution guardrail, `nemo_guardrails_tool_call_check`, at the configured `priority`.
Expand Down
5 changes: 4 additions & 1 deletion docs/index.yml
Original file line number Diff line number Diff line change
Expand Up @@ -545,7 +545,10 @@ navigation:
- page: Codex
path: run-rails/using-hooks-server/codex.mdx
slug: codex
- page: NeMo Relay
- page: NeMo Relay Plugin
path: run-rails/using-hooks-server/nemo-relay-plugin.mdx
slug: nemo-relay-plugin
- page: NeMo Relay Endpoint
path: run-rails/using-hooks-server/nemo-relay.mdx
slug: nemo-relay
- page: Results and Errors
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/hooks-server-api/nemo-relay.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ The hooks server applies its tool policy, then the called tool's per-tool rails.
The plugin returns the decision as its guardrail callback's result: `Ok(None)` for an allow, `Ok(Some(reason))` for a deny, and `Err` for any error status, timeout, or failed connection.

The request examples come from the [NeMo Relay example configuration](https://github.com/NVIDIA-NeMo/Guardrails/tree/develop/examples/configs/agent_hooks/nemo_relay), which allows a `search` tool with only a `query` argument.
For the plugin side, refer to [Check NeMo Relay Tool Calls with the Hooks Server](/run-guardrailed-inference/using-hooks-server/nemo-relay).
For the plugin that calls this endpoint, refer to [Check NeMo Relay Tool Calls with the NVIDIA NeMo Guardrails Plugin](/run-guardrailed-inference/using-hooks-server/nemo-relay-plugin).

## Request

Expand Down
9 changes: 8 additions & 1 deletion docs/run-rails/using-hooks-server/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,14 @@ Send every Codex tool call to the hooks server with a command hook.
<Badge intent="tip" minimal outlined>How To</Badge>
</Card>

<Card title="NeMo Relay" href="/run-guardrailed-inference/using-hooks-server/nemo-relay">
<Card title="NeMo Relay Plugin" href="/run-guardrailed-inference/using-hooks-server/nemo-relay-plugin">

Check every tool call NeMo Relay manages with the NVIDIA NeMo Guardrails Plugin.

<Badge intent="tip" minimal outlined>How To</Badge>
</Card>

<Card title="NeMo Relay Endpoint" href="/run-guardrailed-inference/using-hooks-server/nemo-relay">

The endpoint a NeMo Relay plugin's tool guardrail calls.

Expand Down
431 changes: 431 additions & 0 deletions docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx

Large diffs are not rendered by default.

15 changes: 10 additions & 5 deletions docs/run-rails/using-hooks-server/nemo-relay.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: "Check NeMo Relay Tool Calls with the Hooks Server"
sidebar-title: "NeMo Relay"
sidebar-title: "NeMo Relay Endpoint"
description: "The /v1/hooks/nemo-relay endpoint of the Guardrails hooks server, for a NeMo Relay plugin's tool conditional-execution guardrail."
keywords: ["NeMo Relay", "tool conditional-execution guardrail", "nemoguardrails hooks-server", "/v1/hooks/nemo-relay", "NemoRelayToolConditionalExecutionGuardrail"]
content:
Expand All @@ -10,6 +10,7 @@ content:
[NeMo Relay](https://github.com/NVIDIA/NeMo-Relay) plugins can register a tool conditional-execution guardrail, which Relay runs before each tool call it manages, with the tool's name and arguments.
A plugin can forward that check to the Guardrails hooks server (`nemoguardrails hooks-server`) at `/v1/hooks/nemo-relay`, so the hooks server's tool policy and per-tool rails decide whether the call runs.
This page describes the endpoint such a plugin calls.
To install the plugin that calls this endpoint, refer to [Check NeMo Relay Tool Calls with the NVIDIA NeMo Guardrails Plugin](/run-guardrailed-inference/using-hooks-server/nemo-relay-plugin).

For how Relay runs these guardrails, refer to [conditional execution](https://github.com/NVIDIA/NeMo-Relay/blob/494018ac4d6abb0085853a02f4b6d75a75ff7049/docs/about-nemo-relay/concepts/middleware.mdx?plain=1#L235) in the NeMo Relay documentation.

Expand Down Expand Up @@ -74,15 +75,18 @@ The plugin returns the hooks server's answer as the guardrail callback's result:
|---|---|---|
| `200` with `"decision": "allow"` | `Ok(None)` | Runs the tool |
| `200` with `"decision": "deny"` | `Ok(Some(reason))` | Rejects the call with the reason |
| Any other status, a timeout, or a failed connection | `Err` | Treats the guardrail as failed; what happens to the call depends on the integration |
| Any other status, an invalid decision body, a timeout, or a failed connection | `Err` | Treats the guardrail as failed; what happens to the call depends on the integration |

Map every failure to `Err`, never to `Ok(None)`, so the plugin never reports a check that could not finish as an allow.
What Relay then does with the call depends on the integration that runs the agent:

- Relay's Claude Code and Codex integrations block the call on `Err` from Relay 0.10.1.
The agent sees Relay's own error text, not the plugin's.
- Relay's pi extension treats `Err` as a fault, and by default, with `NEMO_RELAY_PI_FAIL=open`, lets the call run.
To block it, change that setting as described in the NeMo Relay documentation.
Through the Relay CLI's gateway, the agent receives the plugin's error inside Relay's message.
Through the managed Relay daemon, the agent receives only Relay's own message.
Refer to [Check NeMo Relay Tool Calls with the NVIDIA NeMo Guardrails Plugin: Find the Reason for a Failed Check](/run-guardrailed-inference/using-hooks-server/nemo-relay-plugin#find-the-reason-for-a-failed-check).
- Relay's standalone pi extension treats `Err` as a fault and lets the call run by default, with `NEMO_RELAY_PI_FAIL=open`.
Set `NEMO_RELAY_PI_FAIL=closed` to block the call.
- Relay's managed pi extension blocks the call on `Err` and does not use `NEMO_RELAY_PI_FAIL`.

## Tool Names

Expand All @@ -98,6 +102,7 @@ Account for the following sources of duplicate checks:
- **Relay 0.10 checks some calls twice.**
Through the 0.10 releases, Relay also runs the guardrail on Claude Code's and Codex's `PermissionRequest` event.
A call that reaches a permission prompt then costs two requests to the hooks server, and two runs of any rail that calls a model.
When the second check allows the call, Relay also approves Claude Code's permission prompt, so Claude Code does not ask the user.

## Related Topics

Expand Down
4 changes: 2 additions & 2 deletions docs/run-rails/using-hooks-server/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ The hooks server supports the following agent integrations:
|---|---|---|---|
| [Claude Code](/run-guardrailed-inference/using-hooks-server/claude-code) | `/v1/hooks/claude-code` | An `http` hook on `PreToolUse` | 2.1.295, for `onFailure: "block"` |
| [Codex](/run-guardrailed-inference/using-hooks-server/codex) | `/v1/hooks/codex` | A `command` hook on `PreToolUse` that runs `curl` | 0.124.0. From 0.129.0, Codex runs a hook only after you trust it |
| [NeMo Relay](/run-guardrailed-inference/using-hooks-server/nemo-relay) | `/v1/hooks/nemo-relay` | A plugin's tool conditional-execution guardrail | 0.10.1 |
| [NeMo Relay](/run-guardrailed-inference/using-hooks-server/nemo-relay-plugin) | `/v1/hooks/nemo-relay` | A plugin's tool conditional-execution guardrail | 0.10.1 |

The hooks server handles only the `PreToolUse` event, and the NeMo Relay guardrail that stands in for it.
Any other hook event returns HTTP `422`, so register the hooks server on `PreToolUse` only.
Expand Down Expand Up @@ -184,7 +184,7 @@ The server listens on `http://127.0.0.1:8100`.
}
```

1. To check your agent's calls for real, connect it to the hooks server: [Claude Code](/run-guardrailed-inference/using-hooks-server/claude-code), [Codex](/run-guardrailed-inference/using-hooks-server/codex), or [NeMo Relay](/run-guardrailed-inference/using-hooks-server/nemo-relay).
1. To check your agent's calls for real, connect it to the hooks server: [Claude Code](/run-guardrailed-inference/using-hooks-server/claude-code), [Codex](/run-guardrailed-inference/using-hooks-server/codex), or [NeMo Relay](/run-guardrailed-inference/using-hooks-server/nemo-relay-plugin).

## Known Limits

Expand Down
2 changes: 1 addition & 1 deletion docs/run-rails/using-hooks-server/results-and-errors.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ The following table describes how to block tool calls when a check fails:
|---|---|---|
| Claude Code | Is a non-blocking error, and the tool runs | Set `onFailure: "block"` on the hook. Refer to [Claude Code: Add the Hook](/run-guardrailed-inference/using-hooks-server/claude-code#add-the-hook). |
| Codex | Lets the tool run, unless the hook command exits `2`; a hook that times out lets it run too | Pass `curl` the `-f` option and a `--max-time` below the hook's `timeout`, and end the command with `\|\| exit 2`. Refer to [Codex: Add the Hook](/run-guardrailed-inference/using-hooks-server/codex#add-the-hook). |
| NeMo Relay | Depends on the plugin and the integration | The plugin returns `Err` from the guardrail callback for any failed request, and Relay's Claude Code and Codex integrations block the call on `Err` from Relay 0.10.1. Relay's pi extension lets the call run by default. Refer to [NeMo Relay: Map the Response to the Guardrail Result](/run-guardrailed-inference/using-hooks-server/nemo-relay#map-the-response-to-the-guardrail-result). |
| NeMo Relay | Depends on the plugin and the integration | The plugin returns `Err` from the guardrail callback for a failed check. From Relay 0.10.1, the Claude Code and Codex integrations block the call on `Err`. The managed pi extension also blocks it. The standalone pi extension lets it run unless you set `NEMO_RELAY_PI_FAIL=closed`. Refer to [Check NeMo Relay Tool Calls with the NVIDIA NeMo Guardrails Plugin: What Happens to a Call](/run-guardrailed-inference/using-hooks-server/nemo-relay-plugin#what-happens-to-a-call). |

These instructions apply to `PreToolUse` only.
Other events have different failure behavior. For example, Claude Code ignores `onFailure` on its `Stop` event.
Expand Down
2 changes: 1 addition & 1 deletion docs/run-rails/using-hooks-server/run-hooks-server.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ content:
The Guardrails hooks server (`nemoguardrails hooks-server`) loads one guardrails configuration at startup.
It checks each tool call against that configuration's tool policy and per-tool rails.
Use this guide to configure, start, and monitor the server.
To connect an agent, refer to the [Claude Code](/run-guardrailed-inference/using-hooks-server/claude-code), [Codex](/run-guardrailed-inference/using-hooks-server/codex), or [NeMo Relay](/run-guardrailed-inference/using-hooks-server/nemo-relay) page.
To connect an agent, refer to the [Claude Code](/run-guardrailed-inference/using-hooks-server/claude-code), [Codex](/run-guardrailed-inference/using-hooks-server/codex), or [NeMo Relay](/run-guardrailed-inference/using-hooks-server/nemo-relay-plugin) page.

## Start the Server

Expand Down
2 changes: 1 addition & 1 deletion examples/configs/agent_hooks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ The example's `Bash` rail blocks the force push:
{"hookSpecificOutput":{"hookEventName":"PreToolUse","permissionDecision":"deny","permissionDecisionReason":"NeMo Guardrails: regex check tool call"}}
```

To point the agent's hook at the server, follow the agent's page: [Claude Code](./../../../docs/run-rails/using-hooks-server/claude-code.mdx), [Codex](./../../../docs/run-rails/using-hooks-server/codex.mdx), or [NeMo Relay](./../../../docs/run-rails/using-hooks-server/nemo-relay.mdx).
To point the agent's hook at the server, follow the agent's page: [Claude Code](./../../../docs/run-rails/using-hooks-server/claude-code.mdx), [Codex](./../../../docs/run-rails/using-hooks-server/codex.mdx), or [NeMo Relay](./../../../docs/run-rails/using-hooks-server/nemo-relay-plugin.mdx).

## One Server for Each Agent

Expand Down
Loading