Pinned Loading

1. transformer-symmetries transformer-symmetries Public

   We investigate symmetries in single layer transformers used to represent boolean functions and construct bounds in the number of attention heads required to represent them.

2. rkhs-and-deep-learning rkhs-and-deep-learning Public

   An intuitive derivation of smoothing splines from variational calculus, demonstrating their relationship to reproducing kernel Hilbert spaces (RKHS) and regularized neural networks.

   Python

3. interpretable-transformer interpretable-transformer Public

   We construct a fully interpretable single-attention-head transformer, which we train to predict Markov chains. The transformer represents a typical architecture, but we are able to solve it analyti…

   Python

4. taylor-green-vortex-pinn taylor-green-vortex-pinn Public

   Physics-informed neural network for the 2D Taylor-Green vortex, comparing a learned Navier-Stokes solution against the analytic flow.

   Python