GPU Lock Skill

gpu-lock-skill is a Codex skill for coordinating shared NVIDIA GPU access across multiple agents or tasks.

It is designed for workflows where agents spend most of their time editing, reasoning, or compiling, and only occasionally need exclusive GPU access for CUDA tests, benchmarks, profilers, or model/runtime checks.

What It Does

• Provides a local GPU lease manager in scripts/gpu_lock.py
• Serializes access to a selected NVIDIA GPU
• Detects active GPU processes before granting a lease
• Reclaims expired leases with a timeout watchdog
• Terminates stale same-user GPU processes associated with expired leases
• Optionally hides GPUs from agent shells by default with CUDA_VISIBLE_DEVICES=-1

This is a cooperative locking tool, not a kernel-level security boundary. For hard access control, run agents in containers or cgroups without /dev/nvidia* access and expose GPU access only through a trusted wrapper.

Quick Start

Hide GPUs in the agent shell by default:

source /path/to/gpu-lock-skill/scripts/codex_gpu_env.sh

Run a GPU command through the lock:

python /path/to/gpu-lock-skill/scripts/gpu_lock.py run --gpu 0 --timeout 30m -- make bench

The wrapped command receives CUDA_VISIBLE_DEVICES=0. The parent shell remains hidden after the command exits.

Manual Lease

Use a manual lease when multiple GPU commands should run under the same lease:

eval "$(python /path/to/gpu-lock-skill/scripts/gpu_lock.py acquire --gpu 0 --timeout 30m --format shell)"

Run the GPU commands, then release:

eval "$(python /path/to/gpu-lock-skill/scripts/gpu_lock.py release --gpu 0 --token "$CODEX_GPU_LOCK_TOKEN" --format shell)"

The shell-format release unsets the lock token and hides GPUs again.

Commands

python scripts/gpu_lock.py acquire --gpu 0 --timeout 30m
python scripts/gpu_lock.py release --gpu 0 --token "$CODEX_GPU_LOCK_TOKEN"
python scripts/gpu_lock.py refresh --gpu 0 --token "$CODEX_GPU_LOCK_TOKEN" --timeout 30m
python scripts/gpu_lock.py status --gpu 0
python scripts/gpu_lock.py cleanup
python scripts/gpu_lock.py run --gpu 0 --timeout 30m -- <command>

Use --format json for structured output, or --format shell with acquire and release when updating the current shell environment.

Lock State

Lock state is stored under:

1. CODEX_GPU_LOCK_DIR, if set
2. XDG_RUNTIME_DIR/codex-gpu-lock, if available
3. ~/.cache/codex-gpu-lock

Each lease records the owner, token, target GPU, expiry time, tracked process IDs, and tracked process groups.

Timeout Behavior

Every lease starts a lightweight watchdog by default. If the lease is still active after its expiry time, the watchdog reclaims it and terminates recorded same-user processes associated with the lease.

Unknown GPU processes without a matching lock are reported and block acquisition by default. To explicitly clean up old unknown same-user GPU processes:

python scripts/gpu_lock.py acquire --gpu 0 --timeout 30m --kill-unknown-stale-after 30m

Codex Skill

The skill instructions live in SKILL.md. When this skill is active, agents should acquire a GPU lease before running any command that may touch CUDA, NVIDIA profilers, GPU-enabled tests, or GPU benchmarks.