self-attention-implementation

Here are 12 public repositories matching this topic...

ranfysvalle02 / ai-self-attention

This repository provides a basic implementation of self-attention. The code demonstrates how attention mechanisms work in predicting the next word in a sequence. It's a basic implementation that demonstrates the core concept of attention but lacks the complexity of more advanced models like Transformers.

python machine-learning ml python3 rnn attention-mechanism rnn-model attention-is-all-you-need self-attentive-rnn self-attention transformer-architecture self-attention-implementation

Updated May 21, 2026
Python

bhattbhavesh91 / self-attention-python

Sponsor

Star

This repository will guide you to implement a simple self-attention mechanism using the Python's NumPy library

python numpy attention-mechanism self-attention self-attention-implementation

Updated Jan 19, 2023
Jupyter Notebook

khwajawisal / Attention-augmented-Convolutional-Neural-Networks

Star

self-attention-implementation attention-in-neural-networks

Updated Dec 10, 2019
Python

Ayush-2703 / FT-Transformer

Star

PyTorch implementation of the FT-Transformer (Gorishniy et al., 2021) for tabular regression — benchmarked against classical baselines on California Housing, with ablation studies, Optuna tuning, 98% test coverage, and CI/CD.

deep-learning tabular-data regression pytorch transformer self-attention pytorch-implementation self-attention-implementation tabular-deep-learning feature-tokenizer

Updated Aug 13, 2026
Python

dariush-bahrami / mytransformers

Star

My implementation of transformers

nlp deep-learning neural-network transformers attention-mechanism attention-is-all-you-need self-attention pytorch-implementation self-attention-implementation

Updated Feb 20, 2024
Python

nexus-4 / self-attention-mechanism

Star

Implementation of self-attention mechanism based on the "Attention is all you need" article.

python numpy artificial-intelligence self-attention-implementation tutotial

Updated May 29, 2026
Jupyter Notebook

sumony2j / SeedGPT

Star

SeedGPT is a minimal decoder-only LLM from scratch, inspired by andrej karpathy's tutorials, generating text token-by-token based on input word prompts.

Updated Aug 15, 2025
Python

SolarBeamRed / DL_Original-Transformer-from_scratch

Star

PyTorch implementation of of the original Transformer from the 2017 "Attention Is All You Need" Paper. Also included notes about the working where I felt it was necessary

pytorch transformer-architecture positional-encoding self-attention-implementation masked-attention

Updated May 16, 2026
Jupyter Notebook

simranjeet97 / Build_LLM_from_Scratch

Star

A from-scratch implementation of a small language model, built part by part, from a text file to a system capable of generation, classification, instruction following, parameter-efficient adaptation, and reasoning-style preference training.

Updated Sep 3, 2026
Python

ENAYATULLA / Mini-LLM-From-Scratch

Star

An educational GPT-style Transformer Language Model built from scratch in Python and PyTorch featuring custom tokenization, embeddings, positional encoding, causal masked self-attention, multi-head attention, autoregressive text generation, checkpointing, and temperature/top-k sampling.

python nlp machine-learning natural-language-processing deep-learning pytorch artificial-intelligence transformer educational attention gpt language-model from-scratch educational-project self-attention self-attention-implementation llm mini-llm

Updated Jul 25, 2026
TypeScript

MuzzammilShah / GPT-TransformerModel-1

Star

A step-by-step implementation of a GPT-style language model on a combined harry potter novels dataset, inspired by Andrej Karpathy’s lecture.

python3 pytorch gpt language-model transformer-architecture self-attention-implementation

Updated Nov 13, 2025
Jupyter Notebook

Razvan48 / Proiect-Natural-Language-Processing-NLP

Star

Proiect Natural Language Processing (NLP), Anul 3, Semestrul 2, Facultatea de Matematica si Informatica, Universitatea din Bucuresti

natural-language-processing natural-language pytorch transformer bart confusion-matrix bert natural-language-understanding f1-score precision-recall-curve self-attention naturallanguageprocessing pytorch-implementation bert-model gpt2 bert-fine-tuning self-attention-implementation bart-model

Updated Oct 8, 2025
Jupyter Notebook

Add this topic to your repo

To associate your repository with the self-attention-implementation topic, visit your repo's landing page and select "manage topics."

Learn more