Skip to content

feat(training): expose map_fusion_mode as a train_il parameter - #192

Merged
riita10069 merged 1 commit into
autowarefoundation:mainfrom
ChristophBank:feat/train-il-map-fusion-mode
Aug 12, 2026
Merged

feat(training): expose map_fusion_mode as a train_il parameter#192
riita10069 merged 1 commit into
autowarefoundation:mainfrom
ChristophBank:feat/train-il-map-fusion-mode

Conversation

@ChristophBank

Copy link
Copy Markdown
Contributor

Problem

AutoE2E.__init__ has accepted map_fusion_mode since #94, and
MAP_FUSION_REGISTRY offers residual / cross_attn / deformable. But
train_il never forwarded it, so every run trained the constructor default —
residual — regardless of intent
, and nothing in the run log said so.

That makes the "Map Feature Fusion: Residual, Attention" axis of #168
unreachable through the sanctioned path. A contributor comparing the two either
compared residual against residual, or left train_il for a custom script and
lost comparability with everyone else. It is a silent failure: the run looks
healthy and produces a plausible number.

What this adds

  • MapFusion enum beside the existing Backbone enum
  • map_fusion_mode on train_il and wf_train_il
  • the value in checkpoint_config, so evaluation rebuilds the same
    architecture — _model_kwargs feeds that dict into AutoE2E(**config), so
    without the key a non-default run is reconstructed with the constructor default
    and loads mismatched weights
  • model/map_fusion_mode to MLflow, so runs are distinguishable after the fact

The default reproduces the previous behaviour exactly. A run that does not pass
the argument is unchanged.

Deliberately not included

planner_mode is left to #172, which already adds it to train_il along
with the compute_planner_loss wiring. Only the map-fusion half is unique to
this PR, so it stays narrow rather than fixing planner_mode as well.

BEV grid size is left to #188 (camera_bev_size), which touches the same
train_il signature block. This PR adds one parameter next to backbone and
should rebase cleanly either way; happy to rebase behind whichever lands first.

Tests

Model/tests/test_map_fusion_selection.py — a new file rather than an addition
to test_workflow_training_lifecycle.py, which both #172 and #188 modify. Could also integrate into test_workflow_training_lifecycle.py if prefered.

Covers: the pass-through into AutoE2E, the workflow wiring, the checkpoint
round-trip through _model_kwargs, and that every enum value is a registry key.
Verified to fail without the change (5 of 8 fail).

Note on CI: Model/tests currently has 40 pre-existing failures on main
(reasoning, planner, projection, auto_e2e), unrelated to this PR — the counts
are identical with and without it. Nothing in Platform/pipelines fails.

cross_attn is a trap at the contract resolution

Worth knowing before anyone selects it: MapCrossAttentionFusion.forward raises
above 4096 BEV tokens, and the KITScenes contract grid is 256×256 = 65,536. So
deformable is the attention-based option at that resolution. The enum exposes
cross_attn anyway because it is valid at smaller grids, and the guard fails
loudly rather than silently.

The model has selected map-BEV fusion by registry key since autowarefoundation#94
(MAP_FUSION_REGISTRY: residual / cross_attn / deformable) and AutoE2E.__init__
accepts map_fusion_mode, but train_il never forwarded it. Every run therefore
trained the constructor default, residual, whatever the experimenter intended —
and nothing in the run log said so.

That makes the 'Map Feature Fusion: Residual, Attention' axis of autowarefoundation#168
unreachable through the sanctioned training path: a contributor comparing the
two either compared residual against residual, or left train_il for a custom
script and lost comparability with everyone else.

Adds a MapFusion enum beside the existing Backbone enum, threads it through
train_il and wf_train_il, and records it in the checkpoint config so evaluation
rebuilds the same architecture — _model_kwargs feeds that dict into
AutoE2E(**config), so without the key a non-default run would be reconstructed
with the constructor default and load mismatched weights. Also logs
model/map_fusion_mode to MLflow so runs are distinguishable after the fact.

The default reproduces the previous behaviour exactly, so runs that do not pass
the argument are unchanged.

Tests cover the pass-through, the workflow wiring, the checkpoint round-trip and
that every enum value is a registry key; they fail without the change.
@riita10069

Copy link
Copy Markdown
Collaborator

Thanks for the fix, this was a clear bug. Great.

@riita10069
riita10069 merged commit 53a6614 into autowarefoundation:main Aug 12, 2026
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants