BuildingMakemore

Following Andrej Karpathy's "The spelled-out intro to language modeling: building makemore".

A character-level bigram language model trained on names.txt, built two ways:

1. Counting - count all bigram occurrences into a 27x27 matrix, normalize rows into probabilities, and sample new names from it.
2. Neural network - a single linear layer (27x27 weights) trained with gradient descent on the negative log likelihood, which converges to the same result as counting.

Everything is in build_makemore_yay.ipynb.

Setup

pip install -r requirements.txt
jupyter notebook build_makemore_yay.ipynb