Skip to content

README.md

Latest commit

 

History

History
60 lines (39 loc) · 1.31 KB

README.md

File metadata and controls

60 lines (39 loc) · 1.31 KB
Raw

Local Ollama Assistant

This project implements a local AI assistant using:

  • Ollama local model operation via ollama.chat
  • Python Flask API exposing OpenAI-compatible endpoints
  • Compatibility with AnythingLLM chat clients
  • Assistant role design through assistant_config.py SYSTEM_PROMPT

Setup

  1. Install dependencies:
pip install -r requirements.txt
  1. Make sure Ollama is installed and the model exists locally, e.g.:
ollama models
  1. Optionally edit assistant_config.py to change:
  • MODEL_NAME
  • OLLAMA_OPTIONS
  • SYSTEM_PROMPT
  • Web search settings

Run the API

python main.py

The server will start on http://127.0.0.1:8000.

AnythingLLM Configuration

In AnythingLLM, point the chat endpoint to:

  • URL: http://127.0.0.1:8000/v1/chat/completions
  • Model: llama3.2 (or the value in assistant_config.py)

AnythingLLM can send messages in OpenAI-compatible format, and this API preserves conversation history and returns assistant responses.

Assistant Role

Customize the assistant role in assistant_config.py by editing SYSTEM_PROMPT.

Example:

SYSTEM_PROMPT = """
You are a friendly Taiwan Travel Assistant for international students.
..."""

This prompt defines the assistant's behavior, tone, language rules, and response style.