Summary
Add a get_gpu_errors MCP tool that surfaces GPU error state — ECC memory errors, XID errors, and driver health.
Background
GPUs can have silent errors that degrade inference accuracy or cause crashes. NVML exposes:
nvmlDeviceGetMemoryErrorCounter — ECC single-bit and double-bit errors
nvmlDeviceGetTotalEccErrors — total corrected/uncorrected errors
- XID errors via event monitoring
Requirements
- New
get_gpu_errors tool returning per-device error counts
- Fields:
ecc_single_bit, ecc_double_bit, retired_pages, xid_errors (if available)
- Graceful handling when ECC is not supported (consumer GPUs)
Suggested output
{
"index": 0,
"uuid": "GPU-aaaa-1111",
"ecc_supported": true,
"ecc_single_bit_errors": 0,
"ecc_double_bit_errors": 0,
"retired_pages_single_bit": 0,
"retired_pages_double_bit": 0
}
Implementation hints
- Add
Errors() ([]ErrorInfo, error) to the Collector interface
- Implement in
gpu/nvml.go, stub in gpu/stub.go, add to gpu/mock.go
- Register tool in
server/server.go following existing pattern
- Add tests in
server/server_test.go using mock
Acceptance criteria
Difficulty: Intermediate — requires understanding NVML error APIs
Summary
Add a
get_gpu_errorsMCP tool that surfaces GPU error state — ECC memory errors, XID errors, and driver health.Background
GPUs can have silent errors that degrade inference accuracy or cause crashes. NVML exposes:
nvmlDeviceGetMemoryErrorCounter— ECC single-bit and double-bit errorsnvmlDeviceGetTotalEccErrors— total corrected/uncorrected errorsRequirements
get_gpu_errorstool returning per-device error countsecc_single_bit,ecc_double_bit,retired_pages,xid_errors(if available)Suggested output
{ "index": 0, "uuid": "GPU-aaaa-1111", "ecc_supported": true, "ecc_single_bit_errors": 0, "ecc_double_bit_errors": 0, "retired_pages_single_bit": 0, "retired_pages_double_bit": 0 }Implementation hints
Errors() ([]ErrorInfo, error)to theCollectorinterfacegpu/nvml.go, stub ingpu/stub.go, add togpu/mock.goserver/server.gofollowing existing patternserver/server_test.gousing mockAcceptance criteria
Difficulty: Intermediate — requires understanding NVML error APIs