local-llms-workshop

Materials for "Local LLMs for Fast, Cheap, and Reproducible Inference"

Introduction

Open or Closed?

• Do you really need a zoom lens?

Advantages of Open Source

• Security
• Reproducibility
• Cost

Disadvantages of Open Source

• Go Your Own Way
• Performance?
• Option Overload

The Holy Trinity

• Simple Tasks
• Complex Tasks
• Agentic Tasks

Background

• Hugging Face
  • Model Hub
  • transformers
• vLLM
• openai
• torchtune
• LiteLLM
• dspy
• ellmer
• AI Sandbox - https://researchcomputing.princeton.edu/support/knowledge-base/ai-sandbox

Blackfish

Features

• CLI
• UI
• Open OnDemand (OOD)
• Python API

Services

• vLLM

Jobs

• TigerFlow + TigerFlow-ML

Limitations

• Multi-service
• Multi-node
• Quantization
• Fine-tuning

Examples

• Getting Started (OOD, CLI)
• Interactive LLMs (OOD)
• LLMs + Python (CLI) - Protest Analysis
• LLMS + Python (Client) - "Stochastic Parrot"
• Batch Translation (UI) - News Articles
• Batch OCR (CLI) - Scientific Articles

Conclusion

Future Directions

• TBD

Help Wanted

• Resources
• Stars
• Issues
• Use-cases