Jarvis — Personal AI Agent Starter

A barebones starter template for building a personal 24/7 AI assistant that runs on your machine and communicates via Telegram.

What This Is

Jarvis is a personal AI agent that:

Runs 24/7 as a background process
Communicates through Telegram
Uses Claude as its reasoning engine
Orchestrates tasks via MCP servers (filesystem, browser, code sandbox, and more)
Maintains conversation history and long-term memory
Requires explicit approval for risky operations

Key Differences from the Production Version

This starter is intentionally simplified for clarity:

Removed complex retry/event logic (kept the basics)
Simplified error handling
Removed production features (voice calls, smart home, job tracker, etc.)
Streamlined MCP server setup (filesystem, playwright, sandbox, memory only)
Minimal but functional scheduler and memory system

This is a working foundation — you can run it immediately after adding your API keys. Build additional features on top as needed.

Architecture

main.py (entry point / orchestrator)
  ├── PersonalAgent (Claude Agent SDK)
  │   ├── MCP Servers (filesystem, playwright, sandbox, memory)
  │   ├── Risk classification hook
  │   └── Context loader (personality, skills, memory)
  ├── TelegramBot (aiogram)
  │   ├── Handlers (/start, /task, /status, /model)
  │   ├── Approval flow (inline keyboard)
  │   └── Message splitting
  ├── TaskStore (Redis)
  ├── Scheduler (basic cron)
  └── Memory system (ChromaDB for vector search)

Quick Start

1. Prerequisites

Python 3.12+
Redis (local or Docker)
Claude API key
Telegram bot token

2. Clone & Install

git clone <this-repo> jarvis
cd jarvis

# Install dependencies
uv sync --all-packages

# Or with pip
pip install -r requirements.txt

3. Environment Setup

Copy .env.example to .env and fill in your credentials:

cp .env.example .env

Then edit .env:

ANTHROPIC_API_KEY=sk-ant-...
TELEGRAM_BOT_TOKEN=123456:ABC-DEF...
TELEGRAM_CHAT_ID=your_user_id
TELEGRAM_ALLOWED_USER_IDS=[your_user_id]
REDIS_URL=redis://localhost:6379

Get your Telegram credentials:

Bot Token: Create a bot via @BotFather, get the token
Chat ID: Send a message to your bot, then visit https://api.telegram.org/bot<TOKEN>/getUpdates to see your user ID

4. Start Redis

Option A: Local (requires redis-server installed)

redis-server

Option B: Docker

docker run -d -p 6379:6379 redis:7-alpine

5. Run the Agent

uv run python main.py

You should see:

INFO: redis_connected url=redis://localhost:6379
INFO: starting_agent model=claude-sonnet-4-6
INFO: telegram_bot_starting

Open Telegram and send a message to your bot. It should respond!

Usage

Send commands to your bot via Telegram:

/task <description> — Submit a task for the agent to complete
```
/task find me a recipe for chocolate chip cookies
```
/status — View currently running tasks
```
/status
```
/model <name> — Switch AI model (haiku, sonnet, opus)
```
/model opus
```
Regular message — Chat with the agent
```
What's the weather like in NYC?
```

Project Structure

jarvis-starter/
├── main.py                              # Entry point
├── pyproject.toml                       # Root workspace config
├── compose.yaml                         # Docker Compose (optional)
├── .env.example                         # Environment template
├── README.md                            # This file
├── packages/
│   ├── core/src/core/
│   │   ├── __init__.py
│   │   ├── agent.py                     # PersonalAgent (Claude Agent SDK wrapper)
│   │   ├── config.py                    # Settings (pydantic)
│   │   ├── context.py                   # ContextLoader (personality + skills)
│   │   ├── state.py                     # TaskStore + ConversationBuffer (Redis)
│   │   ├── risk.py                      # Risk classifier (2-tier)
│   │   ├── humanize.py                  # Human-readable tool descriptions
│   │   └── logging.py                   # Structured logging
│   ├── interfaces/src/interfaces/telegram/
│   │   ├── __init__.py
│   │   ├── bot.py                       # TelegramBot class
│   │   ├── handlers.py                  # Command handlers
│   │   ├── middleware.py                # Auth middleware
│   │   ├── approval.py                  # Inline approval flow
│   │   └── attachments.py               # File downloads
│   ├── mcp_servers/src/mcp_servers/
│   │   ├── __init__.py
│   │   ├── memory/server.py             # Memory MCP server (ChromaDB)
│   │   └── memory/__init__.py
│   ├── pyproject.toml                   # Package workspace config
│   └── pyproject.toml (each package)
├── tools/memory/
│   ├── __init__.py
│   ├── store.py                         # MemoryStore class
│   └── __main__.py                      # CLI (store/search/list)
├── data/
│   ├── agent_context/
│   │   ├── personality.md               # Agent identity & tone
│   │   ├── schedules.md                 # Cron schedule entries
│   │   ├── memory/
│   │   │   ├── preferences.md           # User preferences (auto-written)
│   │   │   ├── projects.md              # Active projects (auto-written)
│   │   │   ├── chat_history.md          # Session summaries (auto-written)
│   │   │   └── journal.md               # Task journal (auto-written)
│   │   └── skills/
│   │       ├── _index.md                # Skill registry
│   │       ├── _manager.md              # Skill routing logic
│   │       └── example-skill.md         # Example skill definition
│   ├── memory/chroma/                   # Vector store (auto-created)
│   └── attachments/                     # Downloaded files
├── configs/
│   ├── agent.yaml                       # Runtime configuration
│   └── risk_policy.yaml                 # Risk classification overrides
└── tests/
    ├── conftest.py                      # pytest fixtures
    └── unit/
        └── test_example.py              # Example test

Configuration

Main Config (configs/agent.yaml)

agent:
  model: "claude-sonnet-4-6"             # Primary model
  max_turns: 50                          # Max conversation turns
  approval_timeout_seconds: 300          # How long to wait for approval
  task_timeout_seconds: 600              # Task execution timeout
  timezone: "America/New_York"           # For scheduled tasks

workspace:
  allowed_paths:                         # Paths the agent can access
    - "/Users/YOUR_NAME/Documents"
    - "/Users/YOUR_NAME/Downloads"
  context_dir: "data/agent_context"      # Where context files live

logging:
  level: "INFO"                          # DEBUG, INFO, WARNING, ERROR
  format: "json"                         # json or console

Risk Policy (configs/risk_policy.yaml)

Define which operations require approval:

risk_overrides:
  mcp__gmail__send_email: "approval"     # Always ask before sending email
  mcp__filesystem__write_file: "autonomous"  # Auto-write files (careful!)

Core Concepts

PersonalAgent

Wraps the Claude Agent SDK with:

MCP server configuration
Risk classification hook (approve/deny decisions)
Context loading (personality + skills + memory)
Fallback handling (retries with different models)

Risk Classification

Two-tier system:

Autonomous: Safe operations (reading files, web search, code execution) — execute immediately
Require-Approval: Risky operations (sending emails, deleting files) — ask user via Telegram inline keyboard

Override in configs/risk_policy.yaml to customize.

TelegramBot

Handles:

Command routing (/task, /status, /model, etc.)
Approval flow (inline keyboard with Approve/Deny buttons)
Message splitting (Telegram max 4096 chars)
Auth middleware (allowlist user IDs)

Memory System

Three levels:

Conversation Buffer (Redis, TTL 1hr) — Recent chat history for context
Chat History (file, auto-trimmed) — Session summaries (personality.md loads this)
Vector Memory (ChromaDB) — Searchable summaries from past tasks

Use the memory CLI:

uv run python -m tools.memory store "summary text" --topics "topic1,topic2"
uv run python -m tools.memory search "what did I do about X?"
uv run python -m tools.memory list --last 10

Skills

Define repeatable workflows in data/agent_context/skills/:

## Daily News Briefing

**Trigger:** Scheduled daily at 9 AM

**Steps:**
1. Search for tech news
2. Compile top stories
3. Send summary via Telegram

**Tools Required:** WebSearch, Telegram

**Tags:** autonomous, scheduled

**Composable With:** email-digest

Register in skills/_index.md for the agent to find and use them.

Extending

Add a New MCP Server

Edit packages/core/src/core/agent.py → _build_mcp_servers():

servers["my-server"] = {
    "type": "stdio",
    "command": "npx",
    "args": ["-y", "@example/my-mcp-server"],
}

Add a New Telegram Command

Edit packages/interfaces/src/interfaces/telegram/handlers.py:

async def cmd_mycommand(message: Message, task_store: TaskStore) -> None:
    # your handler logic
    await message.answer("Result!")

def create_router(...):
    router.message.register(cmd_mycommand, Command("mycommand"))

Customize Agent Personality

Edit data/agent_context/personality.md — everything here loads into the system prompt.

Store Long-Term Memory

After completing a task, use:

from tools.memory import MemoryStore

store = MemoryStore()
store.store(
    "Fixed the deployment bug by rolling back schema migration",
    topics=["devops", "database"],
    files_touched=["migrations/001_fix.sql"],
)

Running Tests

# All tests
uv run pytest tests/ -v

# Single test file
uv run pytest tests/unit/test_example.py -v

# With coverage
uv run pytest tests/ --cov=packages/

Environment Variables

Variable	Required	Default	Purpose
`ANTHROPIC_API_KEY`	Yes	—	Claude API key
`TELEGRAM_BOT_TOKEN`	Yes	—	Telegram bot token
`TELEGRAM_CHAT_ID`	Yes	—	Your Telegram user ID
`TELEGRAM_ALLOWED_USER_IDS`	Yes	—	List of allowed user IDs (JSON)
`REDIS_URL`	No	`redis://localhost:6379`	Redis connection string
`CONFIG_PATH`	No	`configs/agent.yaml`	Path to config file
`RISK_POLICY_PATH`	No	`configs/risk_policy.yaml`	Path to risk policy file
`OPENAI_API_KEY`	No	—	For memory embeddings (optional)

Troubleshooting

"redis_connected" never appears

Redis isn't running. Start it:

redis-server  # or docker run -d -p 6379:6379 redis:7-alpine

Bot doesn't respond to messages

Check TELEGRAM_CHAT_ID is correct (not bot ID)
Check bot token is valid
Check logs: uv run python main.py 2>&1 | grep -i error
Make sure your user ID is in TELEGRAM_ALLOWED_USER_IDS

Task times out

Increase approval_timeout_seconds or task_timeout_seconds in configs/agent.yaml.

"API unavailable" errors

Claude API is rate-limited or down. The agent will retry with a cheaper model (haiku). Check your API quota and billing.

ChromaDB errors on startup

Delete data/memory/chroma/ and restart — it will recreate the database.

Next Steps

Personalize personality.md — Update identity, tone, and rules to match your preferences
Add your first skill — Create a repeatable workflow in data/agent_context/skills/
Enable a new MCP server — Integrate Google Calendar, email, or custom tools
Set up scheduled tasks — Add entries to data/agent_context/schedules.md
Build context — Let the agent run and it will auto-populate preferences.md, projects.md, chat_history.md

Architecture Notes

Claude Agent SDK subprocess model — Clean, isolated agent execution
Local ChromaDB — No cloud vector DB dependency
Redis for state — Task store, conversation buffer, approval flow
Async/await throughout — Efficient, non-blocking operations
Haiku fallback — Cheaper model for retries on API issues

License

This starter is provided as-is. Modify and extend freely.

Support

CLAUDE.md in the main repo — Full documentation of the production version
tests/ — Example unit tests showing how to test each component
Inline code comments — Each module explains its purpose and design

Happy building! 🚀

Name		Name	Last commit message	Last commit date
configs		configs
data/agent_context		data/agent_context
packages		packages
tests		tests
tools/memory		tools/memory
.env.example		.env.example
.gitignore		.gitignore
GETTING_STARTED.md		GETTING_STARTED.md
MANIFEST.md		MANIFEST.md
README.md		README.md
STRUCTURE.md		STRUCTURE.md
compose.yaml		compose.yaml
index.html		index.html
main.py		main.py
pyproject.toml		pyproject.toml
requirements.txt		requirements.txt

Latest commit

History

Folders and files

Repository files navigation