Skip to content

Publish verified REDD T0 leaderboard - #7

Merged
nipunbatra merged 24 commits into
mainfrom
agent/t0-real-leaderboard
Jul 17, 2026
Merged

Publish verified REDD T0 leaderboard#7
nipunbatra merged 24 commits into
mainfrom
agent/t0-real-leaderboard

Conversation

@nipunbatra

Copy link
Copy Markdown
Member

Outcome

Publishes the first automatically generated, real-data NILMbench T0 leaderboard.

  • corrected REDD T1 refrigerator task
  • 17 methods, including the NILMTK Mean baseline
  • 3 seeds per method: 51 accepted runs
  • 3 epochs, 3,960 training rows, 2,448 test rows
  • NVIDIA A100-SXM4-80GB execution
  • every aggregate is smoke-verified

The leaderboard is generated from committed result bundles; no metric was entered by hand. JSON and CSV are cross-checked, and the JSON commits to the exact CSV checksum.

Reproducibility

  • runner: 83fb39e
  • nilmtk-contrib: 825740b39bcd44b3f4bfaf146f4c0d944843b131
  • image: nilmbench:t0-83fb39e-cuda
  • image digest: sha256:dd977962c2e0d72e2d923f8f5c3e92e538f67a00495e545c2f22571001872e91
  • REDD SHA-256: 757e694a420157aa035c8fd345915080c2396cb7a37bfeb5f925bcb84f9532b8

Verification

  • 129 NILMbench tests passed
  • 4 container-contract tests passed
  • scoped Ruff, lock check, diff check, wheel and sdist passed
  • immutable-image pip check passed
  • full REDD/A100 preflight passed
  • all 51 candidate bundles independently reverified
  • finalizer confirmed 51 source runs, 17 aggregates, three exact seeds per method, and 105 expected publication paths

This is deliberately a T0 smoke leaderboard, not a converged accuracy claim. It establishes a reproducible, continuously extensible real-data baseline.

@nipunbatra
nipunbatra merged commit 6e25a9f into main Jul 17, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant