Skip to content

Benchmark ResidualMoE on real REDD data - #13

Merged
nipunbatra merged 2 commits into
mainfrom
agent/t0-residual-moe
Jul 18, 2026
Merged

Benchmark ResidualMoE on real REDD data#13
nipunbatra merged 2 commits into
mainfrom
agent/t0-residual-moe

Conversation

@nipunbatra

@nipunbatra nipunbatra commented Jul 18, 2026

Copy link
Copy Markdown
Member

What changed

  • register ResidualMoE as a residual mixture-of-experts family
  • advance the single model-agnostic CPU/CUDA image family to merged nilmtk-contrib commit 0769d00893a178277d06e83ccd2e92c03a5ba9ef
  • publish three immutable real-REDD result bundles and approve their exact runtime tuple
  • regenerate the checked-in JSON/CSV leaderboard from 72 source bundles

T0 result

Protocol: corrected T1 REDD, building 1 fridge, 60 s sampling, sequence length 299, 3 epochs, 3960-sample limit, seeds 10/20/42, NVIDIA A100-SXM4-80GB.

Rank Model MAE F1 Parameters Time Peak GPU memory Status
5/24 ResidualMoE 50.66 ± 0.95 0.755 ± 0.019 440,870 18.25 ± 0.94 s 6424.7 MiB smoke-verified

ResidualMoE improves over its TimesNet anchor on this slice (53.54 MAE, 0.728 F1) and over the earlier unconstrained NILMMoE (56.90 MAE, 0.707 F1). It is not the overall winner: NILMFormer remains rank 1 at 48.09 MAE. The result is retained without post-hoc tuning.

Provenance and reproducibility

  • all three publication runs passed on the first attempt
  • an independent second three-seed campaign reproduced every per-seed MAE and F1 value exactly
  • runner: e33b94d44d07b098f774e8569298c9853dd71422
  • nilmtk-contrib: 0769d00893a178277d06e83ccd2e92c03a5ba9ef
  • image: nilmbench:t0-e33b94d-cuda
  • image digest: sha256:3eb89c6d2bcdc8942426e359ed8b1b6ed74108060d86a293101a186844347c86
  • REDD SHA-256: 757e694a420157aa035c8fd345915080c2396cb7a37bfeb5f925bcb84f9532b8
  • final leaderboard: 72 bundles, 24 models, one ranking cohort, contiguous ranks 1–24, all entries provenance-verified

Verification

  • 130 tests passed; 1 dataset-dependent skip
  • Ruff passed for src and tests
  • uv lock --check passed
  • direct CUDA model/expert/router/zero-anchor contract passed on an A100
  • independently regenerated leaderboard.json and leaderboard.csv are byte-identical to the committed artifacts
  • git diff --check passed

@nipunbatra
nipunbatra marked this pull request as ready for review July 18, 2026 04:16
@nipunbatra
nipunbatra merged commit 9315de3 into main Jul 18, 2026
1 check passed
@nipunbatra
nipunbatra deleted the agent/t0-residual-moe branch July 18, 2026 04:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant