Skip to content

Benchmark TimesNet and SGN on real REDD data - #10

Merged
nipunbatra merged 2 commits into
mainfrom
agent/t0-timesnet-sgn
Jul 18, 2026
Merged

Benchmark TimesNet and SGN on real REDD data#10
nipunbatra merged 2 commits into
mainfrom
agent/t0-timesnet-sgn

Conversation

@nipunbatra

@nipunbatra nipunbatra commented Jul 18, 2026

Copy link
Copy Markdown
Member

Campaign definition

  • register TimesNet and SGN as distinct periodic-2D and subtask-gated model families
  • advance the single immutable nilmtk-contrib pin to c130293e24e16817b9859d1b78ae18bd988b1219
  • refresh the lockfile and CPU/CUDA release build contexts to the same reviewed contrib commit
  • document that both models share one batched campaign image
  • publish six immutable real-REDD result bundles and regenerate the living leaderboard

Real T0 campaign

  • task: corrected-t1-redd / fridge / 60 s / sequence 299 / 3 epochs / limit 3960
  • seeds: 10, 20, 42 per model
  • hardware: NVIDIA A100-SXM4-80GB
  • runner: 106af6b
  • contrib: c130293e24e16817b9859d1b78ae18bd988b1219
  • image: nilmbench:t0-106af6b-cuda
  • digest: sha256:4b233308c342556efd0be0d88f24586d2808cc9b7f86cde1d52cafccacb4c425
  • TimesNet: rank 6, MAE 53.54 +/- 3.48 W, F1 0.728 +/- 0.041
  • SGN: rank 8, MAE 56.58 +/- 5.61 W, F1 0.636 +/- 0.015
  • leaderboard: 63 result bundles, 21 models, one cohort, contiguous ranks 1-21

Verification

  • 130 tests passed; 1 dataset-dependent test skipped
  • focused ruff and uv lock checks passed
  • CI cpu-smoke passed
  • generated leaderboard JSON and CSV reproduce byte-for-byte
  • a separate six-job A100 rerun reproduced every MAE and F1 value exactly for all model/seed pairs
  • all bundles have verified source, image, hardware, dataset checksum, and result-ID provenance

@nipunbatra
nipunbatra marked this pull request as ready for review July 18, 2026 02:39
@nipunbatra
nipunbatra merged commit 9a3f2d5 into main Jul 18, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant