Skip to content

fix(tracing): record the tool output on approval-gated function spans - #4866

Merged
seratch merged 2 commits into
openai:mainfrom
dfedoryshchev:fix/approval-function-span-output
Sep 7, 2026
Merged

fix(tracing): record the tool output on approval-gated function spans#4866
seratch merged 2 commits into
openai:mainfrom
dfedoryshchev:fix/approval-function-span-output

Conversation

@dfedoryshchev

@dfedoryshchev dfedoryshchev commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Summary

When a function tool is gated by needs_approval, the function span exports the internal FunctionToolResult wrapper into span_data.output instead of the tool's output.

_run_single_tool returns either a raw tool output or a FunctionToolResult, because the approval path short-circuits. Every other consumer discriminates the two with isinstance, including _build_function_tool_results in the same file; the span write does not. FunctionSpanData.export() then calls str() on the wrapper, so the trace carries a repr of the whole Agent object in the field documented to hold the tool output, including agent.instructions, the tool inventory with JSON schemas, the model settings and the raw arguments. On the repo's own test agent that is 2506 characters, under the default trace_include_sensitive_data=True.

The rejection message leaked by two separate routes: it was written to span_data.output from outside the sensitive-data gate, and it went onto SpanError.message, which Span.export() sends through unchanged. Both reached the backend with the flag off.

The fix reads the output off the wrapper the way the rest of the file already does, drops the ungated write, and routes the error message through _error_tracing.get_trace_error under the same flag.

Test plan

  • Three tests in tests/test_run_step_execution.py, all failing on main: the FunctionToolResult(...) repr where None is expected, assert error["message"] == "Tool execution rejected" against a leaked secret-denial-456, and the include-sensitive-data path keeping the message intact.
  • The redaction test asserts against the whole exported span rather than span_data.output alone, so a leak through any other field fails it too.
  • Suite 7656 -> 7659 passed, with the same 25 pre-existing environment failures as an unpatched run. ruff check, ruff format --check and mypy on the changed module are clean.

The behaviour change worth a second look: with trace_include_sensitive_data=False the exported rejection output goes from the message text to None, and the span error message to a fixed string. That is what the flag asks for and it matches the error paths, but it is visible to anyone reading rejection reasons out of a redacted trace.

redacted_message is passed inline at the call site, where _tool_errors.py hoists its equivalent to a module constant. Say the word and I will match that shape.

Issue number

None.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1be570c04b

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/agents/run_internal/tool_execution.py
sylvesterkaczmarek

This comment was marked as off-topic.

@seratch seratch left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The wrapper-output fix is useful, but the rejection message still reaches the exported span through SpanError.message with sensitive-data tracing disabled. This leak also exists on the base; the issue is that the rejection-redaction part of this PR remains incomplete. Please gate the error message at the same approval boundary and assert against the complete exported span, or split this into the wrapper-output fix alone.

@dfedoryshchev

Copy link
Copy Markdown
Contributor Author

@seratch Updated - the error message is gated the same way now. Have a look.

@sylvesterkaczmarek sylvesterkaczmarek left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The rejection message now follows the same sensitive-data tracing gate as tool output, so the exported span no longer leaks app-supplied rejection text when sensitive tracing is disabled. This resolves the remaining blocker.

@seratch

seratch commented Sep 7, 2026

Copy link
Copy Markdown
Member

@sylvesterkaczmarek Could you please refrain from posting this type of comment on other people's PRs?

@seratch seratch added this to the 0.22.x milestone Sep 7, 2026
@seratch
seratch enabled auto-merge (squash) September 7, 2026 12:24
@seratch
seratch merged commit ba18044 into openai:main Sep 7, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants