fix(checks): Log transformed messages on OTEL request span

[tgasser-nv]

Description

Prior to this PR, incheck_async() and check() both wrote out the original un-transformed messages prior to any transformation. This could leak the sensitive data transform rails like Gliner-PII should protect.

Related Issue(s)

Verification

Pre-commit

$ uv run pre-commit run --all-files
check yaml...............................................................Passed
fix end of files.........................................................Passed
trim trailing whitespace.................................................Passed
ruff (legacy alias)......................................................Passed
ruff format..............................................................Passed
Insert license in comments...............................................Passed
zizmor...................................................................Passed
ty.......................................................................Passed

Unit-test

$ make test
env -u OPENAI_API_KEY -u NVIDIA_API_KEY -u LIVE_TEST -u LIVE_TEST_MODE -u TEST_LIVE_MODE uv run pytest -n auto --dist worksteal  
============================= test session starts ==============================
platform darwin -- Python 3.13.2, pytest-9.1.1, pluggy-1.6.0
rootdir: /Users/tgasser/projects/nemo_guardrails_worktree/fix/iorails-check-span-masked-capture
configfile: pytest.ini (WARNING: ignoring pytest config in pyproject.toml!)
testpaths: tests, benchmark/tests
plugins: langsmith-0.9.4, inline-snapshot-0.33.0, recording-0.13.4, cov-7.1.0, anyio-4.14.1, xdist-3.8.0, asyncio-1.4.0, httpx-0.36.2, profiling-1.8.1
asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=function, asyncio_default_test_loop_scope=function
created: 10/10 workers
10 workers [7681 items]

........................................................................ [  0%]
........................................................................ [  1%]
........................................................................ [  2%]
........................................................................ [  3%]
..................................................................ss.ss. [  4%]
........................................................................ [  5%]
........................................................................ [  6%]
..........................s............................................. [  7%]
......................s................................................. [  8%]
................................................................ss...... [  9%]
....................................s................................... [ 10%]
........................................................................ [ 11%]
........................................................................ [ 12%]
........................................................................ [ 13%]
........................................................................ [ 14%]
........................................................................ [ 14%]
........................................................................ [ 15%]
........................................................................ [ 16%]
........................................................................ [ 17%]
........................................................................ [ 18%]
......................s................................................. [ 19%]
.s...s.........................................ssss.ss..s............... [ 20%]
......s.....................................................s........... [ 21%]
........................................................................ [ 22%]
........................................................................ [ 23%]
...............s....ssssss.............................ss...ss..s..sss.. [ 24%]
........................................................................ [ 25%]
........................................................................ [ 26%]
........................................................................ [ 27%]
........................................................................ [ 28%]
........................................................................ [ 29%]
......sss.ss.sss........sss.ss.s...s.s..s......................sss.sssss [ 29%]
........................................................................ [ 30%]
........................................................................ [ 31%]
........................................................................ [ 32%]
........................................................................ [ 33%]
........................................................................ [ 34%]
........................................................................ [ 35%]
........................................................................ [ 36%]
........................................................................ [ 37%]
........................................................................ [ 38%]
........................................................................ [ 39%]
........................................................................ [ 40%]
................................................................s....... [ 41%]
........................................................................ [ 42%]
........................................................................ [ 43%]
........................................................................ [ 44%]
........................................................................ [ 44%]
........................................................................ [ 45%]
........................................................................ [ 46%]
.......................................ssss............................. [ 47%]
........................................................................ [ 48%]
.......................................................sss.ssssss.ss.sss [ 49%]
........................................................................ [ 50%]
........................................................................ [ 51%]
...............s.ssssss.sss............................................. [ 52%]
........................................................................ [ 53%]
........................................................................ [ 54%]
...s..ssss................s..s.s.s.s.s.ssssss.ss..ssss.................. [ 55%]
........................................................................ [ 56%]
........................................................................ [ 57%]
........................................................................ [ 58%]
...........sssss........................................................ [ 59%]
........................................................................ [ 59%]
..................s..................................................... [ 60%]
...s.............................................................ssssss. [ 61%]
..........................sss...................................ss...... [ 62%]
................ss...................................................... [ 63%]
........................................................................ [ 64%]
.........................................sssssssssss.ss................. [ 65%]
................................................s....................... [ 66%]
........................................................................ [ 67%]
........................................................................ [ 68%]
.....................................................s.................. [ 69%]
......................................................s................. [ 70%]
........................................................................ [ 71%]
........................ss.............s................................ [ 72%]
..................ss........................ss...............s.......... [ 73%]
....................................................................s... [ 74%]
........................................................................ [ 74%]
........................................................................ [ 75%]
........................................................................ [ 76%]
...............................................s........................ [ 77%]
....sssssssss.sssssssss.s............................................... [ 78%]
........................................................................ [ 79%]
........................................................................ [ 80%]
........................................................................ [ 81%]
........................................................................ [ 82%]
........................................................................ [ 83%]
........................................................................ [ 84%]
........................................................................ [ 85%]
........................................................................ [ 86%]
..........................................s.....s....................s.. [ 87%]
........................................................................ [ 88%]
........................................................................ [ 89%]
....s...........................s.....................ssssssss.......... [ 89%]
................ssss................s..ssss............................. [ 90%]
.......................................s.s.............................. [ 91%]
............ss..................ss...................................... [ 92%]
........................................................................ [ 93%]
........................................................................ [ 94%]
.............s.......................................................... [ 95%]
........................................................................ [ 96%]
...............s........................................................ [ 97%]
.........sssss..s....................................................... [ 98%]
........................................................................ [ 99%]
.................................................                        [100%]

════════════════════════════════════════════════════════════════════════════════ inline-snapshot ═════════════════════════════════════════════════════════════════════════════════
INFO: inline-snapshot was disabled because you used xdist. This means that tests with snapshots will continue to run, but snapshot(x) will only return x and inline-snapshot will 
not be able to fix snapshots or generate reports.


====================== 7467 passed, 214 skipped in 38.26s ======================

AI Assistance

• No AI tools were used.
• AI tools were used; a human reviewed and can explain every change (tool: ___).

Checklist

• I've read the CONTRIBUTING guidelines.
• This PR links to a triaged issue assigned to me.
• My PR title follows the project commit convention.
• I've updated the documentation if applicable.
• I've added tests if applicable.
• I've noted any verification beyond CI and any checks I couldn't run.
• I did not update generated changelog files manually.
• I addressed all CodeRabbit, Greptile, and other review comments, or replied with why no change is needed.
• @mentions of the person or team responsible for reviewing proposed changes.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Repository: NVIDIA-NeMo/Guardrails/.coderabbit.yaml
• Review profile: CHILL
• Plan: Enterprise
• Run ID: f62f9068-142d-42ad-a0ad-650273176817

📥 Commits

Reviewing files that changed from the base of the PR and between cabab5f and 5ddcc9c.

📒 Files selected for processing (11)

• docs/observability/tracing/content-capture.mdx
• docs/observability/tracing/span-reference.mdx
• docs/run-rails/using-python-apis/check-messages.mdx
• nemoguardrails/guardrails/guardrails_types.py
• nemoguardrails/guardrails/iorails.py
• nemoguardrails/guardrails/rails_manager.py
• nemoguardrails/guardrails/telemetry.py
• tests/guardrails/test_guardrails_types.py
• tests/guardrails/test_iorails_check.py
• tests/guardrails/test_rails_manager.py
• tests/guardrails/test_transform_rail_pipeline.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

---

📝 Walkthrough

Walkthrough

IORails now preserves rail rewrites in request content capture, including rewrites made before a rail blocks. Rail results carry the rewritten text, and documentation and tests describe and check capture behavior across checks, generation, and streaming.

Changes

Rewritten content capture

Layer / File(s)	Summary
Preserve rewrites in rail results nemoguardrails/guardrails/guardrails_types.py, nemoguardrails/guardrails/rails_manager.py, tests/guardrails/test_guardrails_types.py, tests/guardrails/test_rails_manager.py	RailResult adds optional rewrite_before_block data. Sequential rails retain prior rewrites in later blocking results, including an empty-string rewrite.
Capture messages left by rails nemoguardrails/guardrails/iorails.py, tests/guardrails/test_iorails_check.py	IORails updates request conversation state with rewrites during checks, generation, and streaming, including when a rail blocks. Tests check captured messages and verify that output rewriting does not mutate caller messages.
Document and test capture semantics nemoguardrails/guardrails/telemetry.py, docs/observability/tracing/*, docs/run-rails/using-python-apis/check-messages.mdx, tests/guardrails/test_transform_rail_pipeline.py	Telemetry descriptions and documentation specify which rewritten or caller-provided messages appear in request spans. Tracing tests cover masked content in allowed and blocked checks, generation, and streaming.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Bug fix

Suggested reviewers: tanushriya910

Merge Risk: ⚪ Minimal · up to 5ddcc

The change is mergeable after normal checks; no actionable issue remains in the supplied evidence.

🚥 Pre-merge checks | ✅ 5 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Test Results For Major Changes	⚠️ Warning	This PR makes a cross-cutting change to request-span content capture across check, generation, and streaming paths. It also changes how rail rewrites are preserved before a block. These privacy-sensit…	Update the PR description’s Verification section with the tests run and their final results. Include coverage for masked input/output and blocking behavior in check, generation, and streaming paths.

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	Docstring coverage is 98.25% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 57 functions across 8 files. (3 skipped: 3 …
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: recording transformed messages on the OpenTelemetry request span. It is concise and specific.

Full details: Test Results For Major Changes

Explanation

This PR makes a cross-cutting change to request-span content capture across check, generation, and streaming paths. It also changes how rail rewrites are preserved before a block. These privacy-sensitive behavior changes are not minor. The PR description’s Verification section contains only the template prompt. Its commit subjects mention tests expected to fail, but do not report final test execution or results.

• Fix all pre-merge checks with AI

✨ Finishing Touches 💡 1

⚔️ Resolve merge conflicts 💡

• Resolve merge conflict in branch fix/iorails-check-span-masked-capture

📝 Generate docstrings

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

Staged Fern docs preview: https://nvidia-preview-pr-2440.docs.buildwithfern.com/nemo/guardrails

[greptile-apps]

Confidence Score: 5/5

[Medium risk] Adds tracking of text rewrites that occur before a rail blocks a check.

The PR appears safe to merge; no actionable new issues were found.

What we checked:

• A block loses the mask: The manager keeps the last applied rewrite on the blocked result. The request paths apply that rewrite before recording the messages.
• Capture changes caller messages: The helper creates a new list and a new dictionary for the changed message. It selects the same last assistant message that the check reads.

Summary

Request spans now record messages after input and output rails rewrite them. A later block keeps any rewrite already applied.

• Adds rewrite_before_block without changing the rail verdict.
• Covers checks, blocked generation, and blocked streams with capture tests.
• Documents where other spans and unchecked messages can still contain sensitive text.

Tests were inspected but not run during this review.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Caller messages] --> B[Run rails]
  B --> C[Keep applied rewrites]
  C --> D{Later rail blocks?}
  D -->|Yes| E[Return refusal]
  D -->|No| F[Return checked content]
  E --> G[Record rewritten messages on request span]
  F --> G

Loading

_{Reviews (2) · Last reviewed commit: "Document that any rail blocking before t..." · Reviewed by Greptile}