This project is an implementation of the (encoder/decoder) transformer architecture, as described in "Attention is All You Need" (AIAYN). There is also an implementation of the decoder-only architecture.
Aside from being an interesting learning exercise, the main goal of the project is to be as faithful as possible to the description in AIAYN. This includes the choice of terminology and notation/variable names. Any assumptions are clearly documented.
git clone https://github.com/dscamiss/yati && cd yati
uv pip install .git clone https://github.com/dscamiss/yati && cd yati
pip install -r requirements.txt .See examples directory.
- Add more training examples
- Part 1 of the Annotated Transformer.