Skip to content

[cuda_xpu_alignment] F.gumbel_softmax(tau=0.0) silently produces all-NaN output instead of raising ValueError#33

Closed
Stonepia wants to merge 1 commit into
mainfrom
agent/issue-15
Closed

[cuda_xpu_alignment] F.gumbel_softmax(tau=0.0) silently produces all-NaN output instead of raising ValueError#33
Stonepia wants to merge 1 commit into
mainfrom
agent/issue-15

Conversation

@Stonepia

@Stonepia Stonepia commented May 26, 2026

Copy link
Copy Markdown
Collaborator

[cuda_xpu_alignment] F.gumbel_softmax(tau=0.0) silently produces all-NaN output instead of raising ValueError

Fixes https://github.com/intel-sandbox/agentic_xpu/issues/15

Root Cause: torch.nn.functional.gumbel_softmax at torch/nn/functional.py:2229 computes (logits + gumbels) / tau without validating tau. When tau=0.0, division by zero produces inf/nan that propagates through softmax, yielding all-NaN output silently. Negative tau values are similarly unguarded despite the docstring implying tau should be positive.


Diff stat:

torch/nn/functional.py | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

[cuda_xpu_alignment] F.gumbel_softmax(tau=0.0) silently produces all-NaN output instead of raising ValueError
@Stonepia

Copy link
Copy Markdown
Collaborator Author

Closing: issue #15 (gumbel_softmax tau=0 NaN) is device-agnostic. Upstream pytorch#184582 addresses this directly with a ValueError guard. No XPU-specific fix needed — this issue should be WAITING_UPSTREAM for pytorch#184582 to merge.

@Stonepia Stonepia closed this May 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant