wavelet-transformer

The traditional downsampling (e.g., average pooling) can reduce the resolution. Nonetheless, it may suffer from the dropping of detailed information. In this work, we propose an Efficient Wavelet Attention (EWA), which injects the wavelet transform and a Mean GELU (MGELU) function.

A wavelet-based Transformer leveraging time-frequency domain cross-attention on rs-fMRI data for Autism Spectrum Disorder (ASD) and ADHD diagnosis. Outperforms competing baselines on ABIDE-I and ADHD-200 (MICCAI '24).