Federated Model Training

[m-zain-khawaja]

Description

Working group contributors can train various versions of the Reactive branch of AutoE2E on the KITScenes dataset. We should adopt a 90:10 train:val split since the data is relatively small. Contributors should share their training results and we can iteratively improve to achieve the best performance.

Variations

• Backbone: SwinV2Tiny, ConvNextV2Tiny, ResNet50
• Map Feature Fusion: Residual, Attention
• Trajectory Planner: Bezier, Flow Matching

Report key metrics: ADE (Average displacement error) and FDE (final displacement error) at 3s, 5s and the end of the trajectory which is at 6.4s

Data

Please also consider input data formats, e.g. applying noise to the input images or the map to make the network more robust.

Please also consider map representation, rendering etc. we want to use navigation maps not Lanelet2 maps for now.

Losses

Please also consider loss function

[m-zain-khawaja]

@gcordova10 @atanasko @intisar1020

[Innomer]

I will be contributing as well (Mann here)

[m-zain-khawaja]

@Alejasolu

[m-zain-khawaja]

Data download

The KITScenes dataset is over 4TB in size, this may be too large for local developer workstations. We can leverage the HuggingFace streaming API to stream the data in chunks for model training without downloading the entire 4TB dataset - see more here

Instructions on streaming data in a training loop - here

[CodeName-Detective]

@CodeName-Detective I will look into this.

[docjag]

I would do some experiment.

[coefinger]

@coefinger I will also do some experiments

[ChristophBank]

@ChristophBank I will also join in