Skip to content

feat: add get_gpu_errors tool for ECC and XID error reporting #28

Description

@pmady

Summary

Add a get_gpu_errors MCP tool that surfaces GPU error state — ECC memory errors, XID errors, and driver health.

Background

GPUs can have silent errors that degrade inference accuracy or cause crashes. NVML exposes:

  • nvmlDeviceGetMemoryErrorCounter — ECC single-bit and double-bit errors
  • nvmlDeviceGetTotalEccErrors — total corrected/uncorrected errors
  • XID errors via event monitoring

Requirements

  • New get_gpu_errors tool returning per-device error counts
  • Fields: ecc_single_bit, ecc_double_bit, retired_pages, xid_errors (if available)
  • Graceful handling when ECC is not supported (consumer GPUs)

Suggested output

{
  "index": 0,
  "uuid": "GPU-aaaa-1111",
  "ecc_supported": true,
  "ecc_single_bit_errors": 0,
  "ecc_double_bit_errors": 0,
  "retired_pages_single_bit": 0,
  "retired_pages_double_bit": 0
}

Implementation hints

  • Add Errors() ([]ErrorInfo, error) to the Collector interface
  • Implement in gpu/nvml.go, stub in gpu/stub.go, add to gpu/mock.go
  • Register tool in server/server.go following existing pattern
  • Add tests in server/server_test.go using mock

Acceptance criteria

  • New tool registered and callable
  • Graceful fallback for non-ECC GPUs
  • Tests using mock (no GPU required)
  • README tools table updated

Difficulty: Intermediate — requires understanding NVML error APIs

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestgpuGPU hardware related

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions