Skip to content

fix throughput metric attribution order crash in metrics - #1

Open
geocine wants to merge 1 commit into
razerofficial:mainfrom
geocine:fix/metrics-throughput-sample
Open

fix throughput metric attribution order crash in metrics#1
geocine wants to merge 1 commit into
razerofficial:mainfrom
geocine:fix/metrics-throughput-sample

Conversation

@geocine

@geocine geocine commented Jan 30, 2026

Copy link
Copy Markdown
  • fixes metrics crashing when vllm:time_per_output_token_seconds appears before TTFT metrics by using trpt_sample
    labels instead of ttft_sample.
  • prevents UnboundLocalError and ensures throughput stats are attributed to the correct model.
  • scope is limited to throughput parsing; no behavior changes elsewhere.
  • run metrics with a Prometheus payload where vllm:time_per_output_token_seconds is emitted before
    vllm:time_to_first_token_seconds -> UnboundLocalError at metrics.py:66.

Repro (no vLLM Required)

from prometheus_client.parser import text_string_to_metric_families
import util.mlib as mlib
from src.metrics.metrics import metrics

RAW = """
# HELP vllm:time_per_output_token_seconds summary
# TYPE vllm:time_per_output_token_seconds summary
vllm:time_per_output_token_seconds_sum{model_name="m1"} 2
vllm:time_per_output_token_seconds_count{model_name="m1"} 1
# HELP vllm:time_to_first_token_seconds summary
# TYPE vllm:time_to_first_token_seconds summary
vllm:time_to_first_token_seconds_sum{model_name="m1"} 1
vllm:time_to_first_token_seconds_count{model_name="m1"} 1
"""

mlib.get_metrics = lambda: list(text_string_to_metric_families(RAW))
mlib.get_running_models = lambda: []
mlib.get_cuda_gpu_infos = lambda: []

metrics(None)

Before fix: UnboundLocalError at metrics.py:66 (throughput metrics processed before TTFT).
After fix: table renders without error and throughput shows for model m1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant