LLMFit

Find the best local LLM your PC can actually run.

LLMFit scans your machine, recommends local models that should fit, downloads with beginner-friendly Ollama when possible, and keeps Hugging Face GGUF + llama.cpp available for advanced control.

Stop guessing which 4 GB, 8 GB, 20 GB, or 40 GB model file to download.

llmfit start

Step 1 Check this PC and choose a recommendation target
Use: chat

Step 2 Recommend models that should run locally
Recommended models for this PC
Your PC: 31.5 GiB RAM, CPU-only
Use: chat

#  Fit       Model                         Size      Quality   Why
1  Best      Llama 3.2 1B Instruct          1.2 GiB   High      Best balance
2  Best      Qwen3 VL 2B Instruct           1.7 GiB   High      Best balance
3  Possible  Qwen3 30B A3B Instruct        19.6 GiB   Good      May need lower context

Next:
  llmfit get 1     download with the easiest available path
  llmfit show 1    inspect advanced GGUF files

Why LLMFit Exists

Running local LLMs is still harder than it should be.

People waste time and bandwidth trying to answer:

• Can this PC run a 7B, 13B, 30B, or 70B model?
• Which GGUF quantization should I download?
• Is CPU-only realistic, or do I need GPU offload?
• Why did this model crash after a huge download?
• Should I use the easy Ollama path or the advanced GGUF path?

LLMFit turns that into a guided local workflow:

llmfit check
llmfit find
llmfit get 1
llmfit run 1

Features

• PC-aware model recommendations for ordinary Windows, Linux, and macOS machines.
• Recommendation-first llmfit find with no model name required.
• Beginner-friendly Ollama download and run path.
• Advanced Hugging Face GGUF search and download support.
• Fit ratings based on RAM, VRAM, file size, quantization, and context size.
• Cleaner model shortlist first, detailed file table only when you ask for it.
• Guided first-run command: llmfit start.
• Local dashboard: llmfit status.
• llama.cpp command generation for run and benchmark workflows.
• No telemetry, no hidden cloud inference, no benchmark upload.

Install

Recommended user install after the first package release:

pipx install llmfit

Until a package is published, install from a checked-out repository:

pipx install .

From a local checkout for development:

python -m pip install -e ".[dev]"

With uv:

uv pip install -e ".[dev]"

LLMFit is still early. PyPI packaging is planned after the CLI behavior and docs settle.

Quickstart

Check your setup:

llmfit doctor

Run the guided journey:

llmfit start

Or use the individual commands:

llmfit check
llmfit find
llmfit get 1
llmfit run 1 --prompt "Explain quantization simply."
llmfit status
llmfit doctor

Need a specific use case?

llmfit find --use chat
llmfit find --use coding
llmfit find --use fast
llmfit find --use low-ram

Already know the model family you want?

llmfit find qwen
llmfit find llama
llmfit find mistral

Want the rawer search view?

llmfit find qwen --variants
llmfit find --all

Want the advanced GGUF path?

llmfit show 1
llmfit get 1 --source hf
llmfit run 1 --backend llama.cpp
llmfit bench 1

How It Works

1. llmfit check detects OS, CPU, RAM, GPU, VRAM, and accelerator hints.
2. llmfit find searches Hugging Face for model candidates and filters out embeddings, rerankers, mmproj, and auxiliary files by default.
3. LLMFit estimates whether each candidate should fit with the selected context size.
4. llmfit get 1 uses Ollama when a beginner-friendly match is available and Ollama is installed.
5. llmfit show 1 expands a model choice into concrete GGUF files for advanced Hugging Face downloads.
6. llmfit run 1 uses the local registry and chooses Ollama or llama.cpp based on how the model was downloaded.

Example Commands

# See your machine profile
llmfit check

# Get recommended local chat models for this PC
llmfit find

# Get coding-oriented recommendations
llmfit find --use coding

# Download the first recommendation with the easiest available path
llmfit get 1

# Run the latest downloaded model
llmfit run 1 --prompt "Write a haiku about local AI."

# Advanced: inspect GGUF files and download from Hugging Face
llmfit show 1
llmfit get 1 --source hf

# Advanced: benchmark a GGUF model through llama.cpp
llmfit bench 1

# See local state
llmfit status

Longer aliases remain available for compatibility:

Short command	Alias
check	scan
find	search
show	files
get	download
bench	benchmark

Runtime Setup

Beginner Path: Ollama

For the easiest local run experience, install Ollama from ollama.com/download.

On Windows:

winget install Ollama.Ollama

On macOS:

brew install ollama

On Linux, use your distribution package manager or the installer from Ollama.

When Ollama is installed, llmfit get 1 can pull a matching Ollama model and llmfit run 1 will use ollama run.

You can also pull an Ollama tag directly:

llmfit get llama3.2:3b --source ollama
llmfit run 1

Advanced Path: Hugging Face + llama.cpp

For exact GGUF file and quantization control, use Hugging Face downloads and llama.cpp.

llmfit show 1
llmfit get 1 --source hf
llmfit run 1 --backend llama.cpp

llmfit run --backend llama.cpp and llmfit bench require a llama.cpp executable such as llama-cli.

LLMFit looks for llama-cli on PATH, or you can set:

LLMFIT_LLAMA_CPP=/path/to/llama-cli

On Windows PowerShell:

$env:LLMFIT_LLAMA_CPP = "C:\path\to\llama-cli.exe"

Run this any time setup feels unclear:

llmfit doctor

Contributor Bootstrap

From a local checkout:

.\scripts\dev.ps1

./scripts/dev.sh

Both scripts install the package in editable development mode and print the next command to run.

What Makes This Different

Manual Hugging Face search often shows dozens of files per model, unclear quantization trade-offs, and no clue whether the download will run on your machine.

LLMFit focuses on the decision:

• this PC
• this memory budget
• this model category
• this beginner or advanced download path
• this next command

The goal is not to hide details. The goal is to show the right detail at the right time.

Current Limitations

• Fit estimation is conservative and heuristic-based.
• GGUF metadata is mostly inferred from filenames and Hugging Face file sizes.
• Ollama and llama.cpp must be installed separately.
• Ollama model matching is currently heuristic-based and supports common families first.
• GPU detection is best-effort and platform-dependent.
• Benchmarking is currently for Hugging Face GGUF + llama.cpp models.

Roadmap

• Better GGUF metadata extraction.
• More real-machine hardware probe fixtures.
• Optional live Hugging Face integration tests.
• Optional real llama.cpp integration tests.
• Safer estimator tuning from community feedback.
• Better Ollama model matching and setup guidance.
• Assisted llama.cpp installation.
• Desktop UI after the CLI is solid.
• Explicit-consent benchmark sharing.

Contributing

LLMFit is built for contributors who care about making local AI more practical.

Good first areas:

• add hardware probe fixtures from your machine
• improve model filtering heuristics
• improve fit estimation
• improve CLI wording and docs
• add troubleshooting examples
• test llama.cpp behavior on different platforms

Start here:

• Contributing guide
• CLI commands
• File architecture
• Project positioning
• Launch checklist
• Implementation plan
• TODO

Suggested GitHub topics:

llm local-llm gguf llama-cpp huggingface model-recommendation ai-tools cli python

License

MIT