This repository was archived by the owner on Apr 7, 2026. It is now read-only.
Description Problem
ML training exploration in bashrs (Shell Safety Classifier) consumed significant GPU time:
6 training runs on RTX 4090
3 kill criteria triggered
Peaked at 32% accuracy
HuggingFace model ultimately deleted
Root cause (data distribution mismatch) identified late
Proposed Solution
probar experiment init " SSC v11 training"
probar experiment run --config train.toml --budget 2h-gpu
probar data-audit training_data.jsonl # Pre-flight data quality check
probar experiment status # Live training dashboard
probar experiment compare run-1 run-6 # Compare runs
Requirements
Pre-flight data audit : Check label distribution, detect class imbalance, validate domain coverage before burning GPU time
Budget gates : Set GPU-hour or cost limits per experiment
Early stopping : Monitor validation loss curve, stop when plateau detected (save remaining GPU budget)
Experiment tracking : Store hyperparameters, metrics, and artifacts per run
Kill criteria evaluation : Check accuracy thresholds at intermediate checkpoints, not just at the end
Cost reporting : Track actual GPU-hours and estimated cost per experiment
Acceptance Criteria
Cross-Reference
Filed from paiml-mcp-agent-toolkit#258 as part of sovereign stack tooling gap analysis.
Reactions are currently unavailable
Problem
ML training exploration in bashrs (Shell Safety Classifier) consumed significant GPU time:
Proposed Solution
Requirements
Acceptance Criteria
probar data-auditdetects class imbalance >3:1probar data-auditdetects domain mismatch between train/evalCross-Reference
Filed from paiml-mcp-agent-toolkit#258 as part of sovereign stack tooling gap analysis.