[logging] Publish training metrics and step/epoch to Prometheus

[eicherseiji]

What does this PR do?

Makes the training loop's metrics queryable in Prometheus alongside GPU and vLLM data, so loop state and GPU utilization join on one wall-clock axis in a single store.

New metrics:

• skyrl_<metric> (Prometheus, every backend): every numeric value logged through Tracking.log is mirrored to a gauge of the same name, e.g. timing/run_training becomes ray_skyrl_timing_run_training. Strings and bools are skipped.
• skyrl_trainer_global_step, skyrl_trainer_epoch (Prometheus, per step): the step and zero-indexed epoch the loop is working on, so any metric can be sliced by step or epoch.
• skyrl_step_start_unixtime, skyrl_step_end_unixtime (Prometheus, per step): wall-clock step boundaries.

The step gauges share the names Tracking.log produces for trainer/global_step and trainer/epoch, so a step gauge and the metric published under that name are the same series.

How

Tracking holds a ScalarGauges (from skyrl/train/utils/metrics.py) and mirrors every numeric log value to it. RayPPOTrainer._mark_step_start / _mark_step_end publish the step gauges through Tracking.log_gauge, which writes to those same gauges, in both the sync and async loops. The async loop marks step start from the step Timer so the gauge's timestamp matches the throughput window.

Tests

• tests/train/test_tracking.py (new): numeric metrics publish under sanitized names while strings and bools are skipped; a gauge set via log_gauge and the same-named published metric are one gauge object.

• End to end on a local Ray instance: the step gauges and a logged metric payload export from Ray's /metrics with the expected names, trainer/global_step maps to a single ray_skyrl_trainer_global_step series, and the string and bool produce no gauge:

  ray_skyrl_trainer_global_step  = 42.0
  ray_skyrl_trainer_epoch        = 3.0
  ray_skyrl_step_start_unixtime  = 1.78459e+09
  ray_skyrl_timing_run_training  = 1.5
  

• ruff and black clean on changed files.

[gemini-code-assist]

Code Review

This pull request introduces Prometheus-compatible scalar gauges to track training metrics in SkyRL. It adds a ScalarGauges utility that lazily creates Ray metrics gauges, integrates these gauges into the Tracking class to automatically publish numeric metrics, and adds step-start and step-end tracking to both synchronous and fully asynchronous trainers. The review feedback highlights two key improvements: first, using time.time() instead of step_timer.start_time in the asynchronous trainer to ensure correct Unix timestamps are published; second, replacing the numbers.Real check in the tracking class with a try-except float conversion to robustly support 0-dimensional PyTorch tensors.

[gemini-code-assist]

The _mark_step_start method expects a Unix timestamp (wall-clock time) because it publishes to skyrl_step_start_unixtime. However, Timer typically measures elapsed duration using monotonic clocks (such as time.monotonic() or time.perf_counter()), which do not represent Unix epoch time. Passing a monotonic timestamp will result in incorrect wall-clock metrics in Prometheus. To ensure correctness and consistency with the synchronous trainer, use time.time() directly.

Suggested change

	self._mark_step_start(epoch, step_timer.start_time)
	self._mark_step_start(epoch, time.time())

[gemini-code-assist]

Using isinstance(value, numbers.Real) to filter numeric values will silently exclude 0-dimensional PyTorch tensors (e.g., torch.tensor(1.5)), which do not inherit from numbers.Real. Since PyTorch tensors are extremely common in training/logging payloads, this will prevent many valid metrics from being published to Prometheus. Attempting to convert the value to float using a try-except block is much more robust and handles Python scalars, NumPy scalars, and 0-dimensional PyTorch tensors seamlessly, while safely ignoring strings, lists, and multi-element tensors.

        for key, value in data.items():
            if isinstance(value, bool):
                continue
            try:
                val_float = float(value)
            except (TypeError, ValueError):
                continue
            self._prometheus.set(f"skyrl_{_PROMETHEUS_NAME_RE.sub('_', key)}", val_float)