#

fineweb

Here are 11 public repositories matching this topic...

kmkrofficial / LiteGPT

LiteGPT: A 124M Small Language Model (SLM) pre-trained on FineWeb and fine-tuned on Alpaca.

pytorch alpaca gpt2 llm nanogpt fineweb

Updated Dec 16, 2025
Python

evanatyourservice / llm-jax

Train a SmolLM-style llm on fineweb-edu in JAX/Flax with an assortment of optimizers.

optimization flax gpt jax huggingface large-language-models llm fineweb

Updated Jul 24, 2025
Python

lmmx / bbcfw

Exploring the BBC News subset of the FineWeb dataset (via HuggingFaceFW/fineweb's dated subsets on HF), originally a Common Crawl dataset.

news bbc-news fineweb

Updated Feb 17, 2025
Python

nazmiefearmutcu / awareness

Ingest the public web (Common Crawl + HF FineWeb + RSS + GDELT) onto your laptop. Queryable from DuckDB. Single Python process — no Spark, no cloud.

python osint data-engineering web-scraping gdelt data-pipeline iceberg common-crawl duckdb fineweb

Updated May 31, 2026
Python

aidendorian / Marcella-66M-SLM

A 66M parameter decoder-only transformer language model implemented from scratch in PyTorch. Features a custom SentencePiece tokenizer, RoPE positional embeddings, SwiGLU feed-forward network, per-layer KV cache for efficient autoregressive inference, and a Svelte-based streaming chat interface.

transformers torch pytorch pretrained-models slm language-model rope alpaca sdpa finetuning sentencepiece transformer-models kv-cache small-language-models fineweb flashattention marcella

Updated May 13, 2026
Python

MasterSkepticista / gpt2

Training GPT-2 on FineWeb-Edu in JAX/Flax

flax jax gpt2 flash-attention fineweb

Updated May 29, 2026
Python

chris-colinsky / openai-parameter-golf

OpenAI Parameter Golf competition entry — adaptive Hessian-sensitivity GPTQ clipping on PR #1855's stack. 1.06310 BPB (3-seed mean), PR #1962.

machine-learning transformer quantization language-model ai-engineering llm gptq fineweb claude-code openai-challenge

Updated May 6, 2026
Python

elevonix-labs / finest-data-ui

subnet bittensor fineweb elevonix

Updated Mar 6, 2025
TypeScript

aditaa / llm

Decoder-only LLM from scratch with reproducible data pipelines, tokenizer/sharding workflows, and GPU training.

nlp machine-learning pytorch zim training-pipeline llm fineweb

Updated Apr 13, 2026
Python

mn-cs / fineweb-spark

FineWeb-Edu dataset analysis using Apache Spark - DSC 232R group project

python nlp machine-learning big-data apache-spark text-classification word2vec distributed-computing pyspark fineweb

Updated Mar 24, 2026
Jupyter Notebook

Cubix1729 / gpt-training

A minimal GPT model pretraining pipeline (mostly taken from Andrej Karpathy's build-nanogpt repo)

gpt-2 llm llm-training fineweb pretraining-llms gpt-2-small

Updated May 18, 2026
Python

Improve this page

Add a description, image, and links to the fineweb topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the fineweb topic, visit your repo's landing page and select "manage topics."