Skip to content

fix: clear ModelOutput fields via mapping to free tensors - #1568

Closed
AngeloDanducci wants to merge 2 commits into
generative-computing:mainfrom
AngeloDanducci:ad-1549
Closed

fix: clear ModelOutput fields via mapping to free tensors#1568
AngeloDanducci wants to merge 2 commits into
generative-computing:mainfrom
AngeloDanducci:ad-1549

Conversation

@AngeloDanducci

Copy link
Copy Markdown
Contributor

Pull Request

Issue

Fixes #1549

Description

HuggingFace backend now properly frees tensors when nulling fields.

Testing

  • Tests added to the respective file if code was changed
  • New code has 100% coverage if code was added
  • Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

Attribution

  • AI coding assistants used

Adding a new component, requirement, sampling strategy, or tool?

If your PR adds or modifies one of the types below, check the matching box. A checklist of type-specific review items will be posted as a comment.

  • Component
  • Requirement
  • Sampling Strategy
  • Tool

NOTE: Please ensure you have an issue that has been acknowledged by a core contributor and routed you to open a pull request against this repository. Otherwise, please open an issue before continuing with this pull request.

Signed-off-by: AngeloDanducci <angelo.danducci.ii@ibm.com>
@AngeloDanducci
AngeloDanducci requested a review from a team as a code owner August 19, 2026 14:31
@github-actions github-actions Bot added the bug Something isn't working label Aug 19, 2026
@jakelorocco

Copy link
Copy Markdown
Contributor

@AngeloDanducci thanks for picking up the issue. I think the initial PR that spotted the issue in the generate from raw path might actually be fixing it in the chat path as well? #1518

Signed-off-by: AngeloDanducci <angelo.danducci.ii@ibm.com>
@AngeloDanducci

Copy link
Copy Markdown
Contributor Author

@jakelorocco yep looks like it's covered there, I will update the PR to auto close 1549 and close this PR.

@jakelorocco

Copy link
Copy Markdown
Contributor

Thank you; sorry for the duplicate work, I didn't realize the other PR was going to handle it when I opened the issue and didn't realize the duplication till after I saw your PR.

@AngeloDanducci

Copy link
Copy Markdown
Contributor Author

No worries - the nature of being proactive sometimes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(huggingface): nulling ModelOutput fields never frees the tensors (KV cache stays on the MOT) on chat path

2 participants