| Category | Description | Famous Open-Source Models |
|---|---|---|
| Encoder-Only | Understanding-focused models for tasks like classification, QA (Extractive), embeddings | BERT, RoBERTa, ALBERT, DistilBERT, ELECTRA, DeBERTa, ERNIE |
| Decoder-Only | Generation-focused models for text completion and open-ended output | GPT (1-4), GPT-Neo, GPT-J, GPT-NeoX, LLaMA, Mistral, Phi, Gemma |
| Encoder-Decoder (Seq2Seq) | Input-to-output models for translation, summarization, etc. | T5, mT5, FLAN-T5, BART, mBART, UL2, MarianMT, ByT5 |