Skip to content

Latest commit

 

History

History
49 lines (36 loc) · 1.8 KB

File metadata and controls

49 lines (36 loc) · 1.8 KB

Model Evolution

Experiment Timeline

Version range Main idea Data scale Notes
v1 Context last-frame decoder 14 files Initial production-style ConvLSTM baseline
v3 Flatten ReLU baseline 14 files Lightweight smoke/baseline model
v4-v9 Multi-head attention variants 14 files BCE, Tversky, CBAM, stacked ConvLSTM trials
v10-v13 Spatial UNet family 120 files Streaming input, holdout sample, mask/time channels
v14-v16 365-file experiments 341 matched files Larger data regime, gated UNet, production v16

Current Production Model

  • Version: v16
  • Variant: multihead_spatial_unet_gated
  • Parameters: 335,741
  • Input shape: 12 x 66 x 66 x 5
  • Added channels: inflow, outflow, occupancy mask, time sin, time cos
  • Training monitor: val_ssim, maximize
  • Best epoch: 24
  • Epochs trained: 38

Improvement Highlights

Production v16 improved over the weakest observed model by:

  • R2: +873.8%
  • SSIM: +39.6%
  • Occupancy F1: +231.7%
  • Outflow MAE: +60.5% reduction from worst
  • Validation MAE: +66.5% reduction from worst

The best F1 model was v15 at 0.5608, while production v16 reached 0.5485 and achieved the best R2 and SSIM.

These improvements are based on actual-value evaluation: predictions and labels were inverse-transformed to the original AIS traffic scale before computing R2, SSIM, F1, and MAE. See actual_value_evaluation.md.

Why v16

v16 is a strong production candidate because it combines:

  • Larger 365-file training setup
  • Gated spatial UNet architecture
  • Two ConvLSTM layers
  • Route occupancy prior
  • Time-of-day channels
  • Monitoring based on SSIM rather than only loss

This improved structural similarity and global fit while keeping F1 close to the best model.