Summary
Implement Transolver, which introduces Physics-Attention that decomposes discretized domains into learnable physical-state slices.
Reference
- Wu et al., "Transolver: A Fast Transformer Solver for PDEs on General Geometries," ICML 2024. arXiv:2402.02366
Description
Transolver introduces "Physics-Attention" that groups mesh points into learnable slices based on similar physical states (rather than spatial proximity). Attention is applied to these physics-aware tokens, which better captures intricate physical correlations than raw mesh-point attention. Works on general geometries without requiring regular grids.
Key innovation: Instead of attending over all mesh points (quadratic cost), it learns to partition the domain into a small number of physically meaningful slices and operates on those.
Summary
Implement Transolver, which introduces Physics-Attention that decomposes discretized domains into learnable physical-state slices.
Reference
Description
Transolver introduces "Physics-Attention" that groups mesh points into learnable slices based on similar physical states (rather than spatial proximity). Attention is applied to these physics-aware tokens, which better captures intricate physical correlations than raw mesh-point attention. Works on general geometries without requiring regular grids.
Key innovation: Instead of attending over all mesh points (quadratic cost), it learns to partition the domain into a small number of physically meaningful slices and operates on those.