Skip to content

fix: HuggingFace backend mot.usage always None without telemetry enabled #694

Description

@planetf1

Bug

ModelOutputThunk.usage is always None for the HuggingFace backend unless telemetry tracing or MELLEA_METRICS_ENABLED=true is set.

Root cause

Introduced in #653 (token metrics hooks refactor). In huggingface.py::_post_process_async, token count extraction was gated behind (span is not None or metrics_enabled), so mot.usage is never populated in plain runs.

Fix

Remove the telemetry guard from token count extraction — usage is a standard mot field, not a telemetry concern. Telemetry reporting further down still checks span/metrics_enabled as before.

# Before
if (span is not None or metrics_enabled) and isinstance(hf_output, GenerateDecoderOnlyOutput):
    ...

# After
if isinstance(hf_output, GenerateDecoderOnlyOutput):
    ...

Impact

test/backends/test_huggingface.py::test_async_avalue fails asserting mot1.usage is not None.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions