derope

Here is 1 public repository matching this topic...

fahadsid1770 / Aesop-LLM

custom Bengali Language Model built from scratch using PyTorch, featuring advanced transformer architectures including Multi-Head Latent Attention, RoPE positional encodings, and Mixture of Experts layers.

tokenizer gpt llm multi-head-latent-attention derope

Updated Mar 13, 2026
Jupyter Notebook

Add this topic to your repo

To associate your repository with the derope topic, visit your repo's landing page and select "manage topics."

Learn more