Three end-to-end systems projects: deploying speech models to microcontrollers, optimizing LLM inference with custom GPU kernels, and implementing alternative DNN primitives with compression analysis. Each project includes code, experiments, reported results, and a project-level README with full setup details.
These were built as graduate coursework for the Machine Learning Systems and Hardware course at Cornell University and are intended to demonstrate practical ML systems engineering: memory-constrained deployment, kernel-level optimization, and measuring accuracy–speed trade-offs with hardware-aware metrics.
Python · PyTorch · Triton · CUDA · TensorFlow Lite Micro · Arduino · NumPy · Matplotlib · pytest · Jupyter · Hugging Face
| Project | What it covers | Key techniques | Highlight |
|---|---|---|---|
| LLM Deployment to TinyML | Keyword spotting on Arduino Nano 33 BLE | Audio preprocessing, TinyConv, PTQ/QAT, pruning, TFLite Micro | ~93% test accuracy; QAT recovers ~20 pp over PTQ at 3-bit |
| LLMs with Triton | Decoder-only LLM runtime optimization | torch.compile, fused projections, Triton INT4 head, sparse attention |
2.81× overall score; 9.7× prefill speedup (R3) |
| DNN Primitives Implementation and Approximation | Conv/matmul primitives + compression | im2col, Winograd, FFT, SVD, log-domain matmul | ~6× MNIST compression at ~98% accuracy (fc1+fc2) |
End-to-end keyword spotting pipeline: train a small TinyConv model on Google Speech Commands, convert PyTorch → ONNX → TFLite → TFLite Micro, and deploy to an Arduino Nano 33 BLE Sense. Covers flash/RAM sizing, quantization-aware training vs. post-training quantization, and structured/unstructured pruning.
Skills: TinyML deployment, audio feature extraction (Mel/MFCC), model compression, MCU profiling, on-device vs. offline accuracy analysis.
Optimize a standalone decoder-only LLM inference stack in pure PyTorch and Triton, preserving reference logits while improving throughput across four regimes (decode with/without context, batched/single prefill). Five guided baselines build up to open-ended optimizations in submission.py.
Skills: GPU kernel programming, torch.compile, fused linear layers, INT4 decode kernels, sparse attention, correctness-preserving runtime optimization.
Implement and benchmark alternative formulations of core DNN operations — im2col, Winograd, and FFT convolutions; SVD and log-domain matrix multiply — then study numerical precision, rank truncation, and end-to-end SVD compression of an MNIST MLP.
Skills: kernel implementation from scratch, numerical error analysis, low-rank approximation, execution-based benchmarking, pytest correctness validation.
- Python 3.10+
- Project-specific hardware where noted:
- TinyML: Arduino Nano 33 BLE Sense (optional for full deployment)
- Triton / DNN primitives: CUDA GPU recommended
Dependencies are not shared at the repo root. cd into a project folder and follow its README:
| Project | Quick start |
|---|---|
| TinyML | Open notebooks 1–6 in Jupyter/Colab; run pytest tests/ for size/quant helpers |
| LLMs with Triton | pip install torch triton … → download artifacts → python local_eval.py --submission submission.py |
| DNN Primitives | pip install torch torchvision pytest → pytest tests/ → open 1_numerical_precision.ipynb |