Mini-Mack is a tiny domain-specific language model and semantic retrieval system built from scratch with PyTorch, Bash, Ollama embeddings, jq, awk, and plain text files.
- 1,035,921 parameters
- Trained from scratch
- Word-level tokenizer
- 128-dimensional embeddings
- 8 attention heads
- 4 Transformer layers
- 64-token context
- Custom PyTorch causal Transformer
Mini-Mack combines several small tools:
- Custom PyTorch Transformer for Richmack framework concepts
- Bash routing for deterministic queries
- awk for exact mathematical calculations
- Ollama embeddinggemma for semantic embeddings
- jq for vector similarity calculations
- Plain-text documents and
.vecfiles for semantic retrieval - Top-3 chunk retrieval
- Semantic paragraph reranking
Question → Bash router → deterministic rule, calculator, semantic retrieval, or Mini-Mack Transformer
Semantic retrieval:
Document → chunk → embedding → vector similarity → top matching chunks → paragraph reranking → answer
The included wallet_train.txt contains Richmack Improvement Framework definitions, formulas, calculations, scenarios, recommendations, and concept comparisons.
Run:
python3 train_word.py
The resulting model is stored as:
wallet_word.pt
./mini-mack "What is stable repetition?"
./mini-mack "If N=68 and r=0.5, what is S?"
./mini-mack "Why does complexity matter to the Richmack Weissman Score?"
Mini-Mack deliberately uses deterministic Unix tools where they are more reliable than a neural model.
The model handles learned language and framework concepts.
Bash handles routing.
awk handles exact arithmetic.
Embeddings handle semantic retrieval.
The project is an experiment in building a useful hybrid AI system from small, understandable primitives.