Skip to content

Docs: Add how-to article for running models (API and local) #4

Description

@kosar

Goal

Add a short, practical how-to article that helps users choose and run models (LLMs) for their agents: metered API providers and local/self-hosted options.

Context

We already reference docs/MODEL-CHOICE.md and mention Ollama in the README. Users need a clear path to (1) use API-backed models and (2) run models locally (e.g. Ollama) as an alternative to metered APIs. This how-to should set them up to make that choice and get it working.

What the how-to should cover

  • Using metered API models: Where to configure API keys and model names (e.g. OpenAI, Moonshot, Anthropic), and how the agent uses them.
  • Running models locally: How to "roll up your sleeves" and run a local model (e.g. Ollama) that can be used in place of metered API providers.
  • Practical note: Include a neutral note that local models can be much slower than API-based ones and may cause agent responses to take minutes rather than seconds—so users can set expectations and choose accordingly.

Format

Keep it concise: how-to style, not a long reference. No bias toward either approach; the article should support both. It can live under docs/ (e.g. docs/HOWTO-MODELS.md) and be linked from the main README or MODEL-CHOICE where appropriate.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions