Home

eyespot

The open-source framework that turns cameras, microphones, and speakers into native Agent Skills.

Website · Quick Start · Architecture · Built-in Skills · Hardware · Contributing

---

What is eyespot?

AI Agents can write code, make plans, and manage schedules — but they still can't see, hear, or speak in the physical world.

eyespot bridges that gap. It is an open-source framework that packages cameras, microphones, and speakers as first-class Agent Skills — callable like any other tool. Give your Agent real-world perception in minutes, not months.

Your Agent  ──calls──▶  eyespot Skill  ──perceives──▶  Physical World
                              │
                    structured JSON output
                    (not video streams)

Why eyespot?

Feature	Traditional Camera	eyespot
Output	Video stream / screenshots	Structured JSON (Agent-callable)
Operating Mode	Passive recording	Proactive sensing + event-driven push
Agent Integration	None (build your own)	Native Skill, 3 lines of code
Intelligence	Rule-based (motion detection)	Semantic understanding
Developer Friendly	Closed SDK	MIT open source + RESTful + MCP

---

Quick Start

pip install eyespot-core

import eyespot

# Connect your camera
camera = eyespot.Camera.from_rtsp(url)

# Register as an Agent Skill
agent.add_skill(camera.as_skill())

That's it — your Agent can now see the physical world.

Works with any Agent framework: LangChain, AutoGen, LiveKit, OpenAI, Claude, and OpenClaw.

---

Architecture

eyespot uses a three-layer architecture:

┌─────────────────────────────────────────────────┐
│                  The Bridge                      │
│   Agent framework adapter (LangChain, AutoGen,   │
│   LiveKit, MCP protocol, RESTful API)            │
├─────────────────────────────────────────────────┤
│                  The Brain                       │
│   Local multimodal vision-language model +       │
│   scene-specific lightweight models              │
├─────────────────────────────────────────────────┤
│                  The Eye                         │
│   RGB camera + microphone array + speaker +      │
│   wide-angle lens + night vision + edge AI chip  │
└─────────────────────────────────────────────────┘

Key design principles:

• Hardware as Native Skills — Cameras, mics, and speakers are first-class Agent Skills. Your Agent calls them the same way it calls any tool.
• Plug In Any Hardware — ONVIF cameras, RTSP streams, USB microphones, Bluetooth speakers — all abstracted behind a unified API.
• Flexible Inference — Local inference on capable hardware (data stays on device), or cloud inference for cost-sensitive deployments.

---

Built-in Skills

eyespot ships with six core Skills out of the box:

Skill	Description
Scene Observation	Ask in natural language, get real-time scene descriptions
Object Recognition	Identify people, pets, packages — structured JSON output
Event Detection	Always-on monitoring, proactive event push to Agent
Visual Tasks	Set complex tasks with visual conditions via natural language
Timeline Recall	AI-understood event timeline, not traditional video playback
Open API	RESTful API, Webhook, deep Home Assistant integration

Example: Object Recognition

result = camera.detect()
# Returns structured JSON:
# { "type": "person", "label": "delivery", "conf": 0.97 }

Example: Proactive Event Detection

@camera.on_event("smoke_detected")
def handle_smoke(event):
    # { "event": "smoke_detected", "ts": 1714000383 }
    agent.alert(event)

---

Compatible Hardware

eyespot is verified compatible with a wide range of hardware:

Device	Type	Status
Any ONVIF camera	IP Camera	Supported
Any RTSP stream	IP Camera	Supported
USB webcams	USB Camera	Supported
Birdie Lite	eyespot Reference HW	Coming Soon
Birdie Pro	eyespot Reference HW	Coming Soon
Birdie Wild	eyespot Reference HW	Coming Soon

Supports ONVIF / RTSP / USB / WebRTC and more. Additional hardware verified continuously.

---

Ecosystem Compatibility

eyespot is framework-agnostic. If your Agent can call tools, it can use eyespot Skills.

• OpenClaw
• LangChain
• AutoGen
• LiveKit
• OpenAI Function Calling
• Claude Tool Use
• Home Assistant
• MCP Protocol

---

Roadmap

• Core SDK architecture
• ONVIF / RTSP camera support
• Scene observation & object recognition Skills
• Event detection with proactive push
• Timeline recall
• Birdie reference hardware release
• eyespot Cloud (managed hosting + fleet management)
• Model Store (community-contributed scene models)

---

Contributing

We welcome contributions! eyespot is MIT-licensed and community-driven.

# Clone the repo
git clone https://github.com/Vibetool/eyespot.git
cd eyespot

# Install dev dependencies
pip install -e ".[dev]"

# Run tests
pytest

Please read our Contributing Guide before submitting a PR.

---

License

MIT License — free forever, for everyone.

---

Links

• Website: eyespot.ai
• Email: team@vibetool.ai
• Twitter/X: @vibetool

---

_{Built with ❤️ by Vibetool}