🪿 WingmanAI

Alpha — actively in development. Expect rough edges.

WingmanAI is a macOS ambient AI overlay powered entirely by local inference. Its AI persona — Goose — watches your screen, understands what you are doing in any app, and surfaces proactive advice without you ever having to ask. When Goose wants to act on your behalf, it proposes an action and waits for your approval. You can talk to it by voice or text at any time.

Named after Goose from Top Gun — always watching your back, speaking up when it matters, executing when asked.

Nothing leaves your Mac. Ever.

---

What Makes This Different

Every AI assistant tool available today is reactive — you open it, you ask, it answers, you close it. None of them watch your screen. None of them understand what you are doing right now in real time. None of them speak up without being asked.

WingmanAI is ambient. Goose lives in a compact always-on-top overlay in your chosen corner. It watches. It thinks. When it has something useful to say, it says it. When it wants to do something, it asks your permission first.

┌─────────────────────────────────────────┐
│ 🪿 WingmanAI    ▶ ⏸ ⏹   👁️  🔕  ⚙️  │
│ ─────────────────────────────────────── │
│ Looks like you're writing an email.     │
│ The subject line could be sharper.      │
│ [✏️ Fix it]  [💡 Explain]  [Dismiss]   │
│ ─────────────────────────────────────── │
│ 🧠 14K/128K ████░  RAM 38G  CPU 12%    │
└─────────────────────────────────────────┘

---

Features

👁️ Screen Watching

• Captures your screen every 5 seconds (configurable: 3/5/10/30s)
• Perceptual hash diff — inference only fires when the screen meaningfully changes, keeping your GPU cool
• Watch full screen, active window only, or a user-defined region
• Privacy exclusions — list apps Goose should never watch (1Password, banking apps, etc.)
• Vision transparency log — see exactly what Goose saw and what it concluded

🎯 Focus Awareness

• Mouse tracking — Goose knows where your cursor is and prioritises that area of the screen
• Text selection — highlight any text in any app and Goose immediately reads it, understands it in context, and generates relevant suggestions (not hardcoded buttons — AI-generated based on what you've been doing)
• Approve an action and Goose rewrites the text in-place via AppleScript or keyboard simulation, without you switching apps

🎙️ Full Two-Way Voice

• Speak to Goose — SFSpeechRecognizer, 50+ languages, offline-capable
• Goose speaks back — AVSpeechSynthesizer, macOS neural voices, user picks
• Global hotkey (⌘⇧Space default) to activate from anywhere
• "Hey Goose" wake word — optional, toggleable

🖥️ Transport Controls — ▶ ⏸ ⏹

• Play — Goose is active, watching, models loaded in GPU RAM
• Pause — capture stops, inference freezes, models stay warm — instant resume
• Stop — full resource release, models evicted from VRAM — use when you need GPU for something else

🤖 AI Models

• Reasoning: IBM granite3.1-dense:8b — 128K context, same model Bob uses
• Vision: llava:7b — reads your screen
• Both bundled via Ollama — zero external installation required
• Both configurable — swap for any Ollama-compatible model
• VisionProvider abstract interface — swap LLaVA for moondream2 or better with one class change

🌩️ Frontier Model Escalation (Off by Default)

• When local Granite isn't enough, optionally escalate to GPT-4o, Claude, or Grok
• Priority order, per-minute throttle, and full failback chain — all user-configured
• Attribution always visible: "🌩️ Claude says:" vs "🪿 Goose says:"
• API keys stored locally, never transmitted except to the configured provider

🌐 Web Search

• [SEARCH: query] — DuckDuckGo, no API key needed
• [FETCH: url] — direct URL fetch
• Rate limited (default 10/min, configurable)
• Proactive research mode — tell Goose to research something while you keep working; it surfaces a summary card when done

🖥️ Action Execution

• Shell commands — Goose proposes, you approve, it executes
• AppleScript — Goose controls any app with your approval
• Every action requires explicit approval and is logged to the audit log
• Timed tasks — "work on this for 5 minutes then stop"

🧠 Memory & Profiles

• Persistent memory across sessions — Goose remembers your preferences and projects
• Project profiles — named context bundles (folder + pinned files + persona)
• One-tap profile switch from the overlay
• Snippet library — save great outputs, searchable

📁 Session Continuity

• Token counter always visible: 🧠 14K/128K ████░ 11% — green → amber → red
• When context fills (90%), Goose writes a concise summary ("state of the world") and starts fresh
• Loads previous session summaries intelligently — more recent sessions loaded first within token budget
• Session files: gooseSession1-2025-07-14.md — readable and editable by you

📊 Performance Monitor

• Always visible: RAM 38G CPU 12% GPU 48%
• Auto-throttle: when thermals hit red, Goose slows capture interval automatically
• Full progress bars in expanded view

🔧 Developer Features

• Git awareness — git status, git diff, auto commit message generation
• Error message explainer — stack traces and compiler errors detected on screen
• Terminal command suggester — failed commands → fix suggestions
• Code review on demand, draft improver, translation, tone analyzer
• Drag files onto the overlay — Goose reads and comments
• macOS Services "Ask Goose" — right-click selected text in any app
• URL scheme wingmanai://ask?q=... — trigger from Keyboard Maestro, Alfred, Shortcuts

🛡️ Reliability

• LaunchAgent — auto-restarts on crash
• Crash recovery — state snapshot before each inference
• Audit log — every action logged with timestamp, tool, argument, approval status
• Local-only badge — visible confirmation nothing has left your Mac
• Update check on launch

---

Competitive Landscape

WingmanAI occupies a position no other tool currently holds.

Capability	Ollama	Jan.ai	LM Studio	Unsloth	OpenClaw	WingmanAI
Core purpose	Inference runtime	Local chat UI	Local chat + dev API	Model training	Messaging gateway	Ambient desktop AI
Screen watching	❌	❌	❌	❌	❌	✅
Proactive advice	❌	❌	❌	❌	❌	✅
Always-on overlay	❌	❌	❌	❌	Menu bar only	✅
App control / keyboard injection	❌	❌	❌	❌	❌	✅
Works in any app	❌	❌	❌	❌	❌	✅
Voice two-way	❌	❌	❌	❌	TTS only	✅
Live screen vision	❌	❌	❌	❌	❌	✅
Shell execution	❌	❌	❌	❌	✅	✅ + approval gate
Web search	❌	❌	❌	✅	✅	✅
Persistent memory	❌	❌	❌	❌	MEMORY.md	✅
Git awareness	❌	❌	❌	❌	❌	✅
Session continuity files	❌	❌	❌	❌	❌	✅
100% local / offline	✅	✅	✅	✅	❌ requires API keys	✅
Zero API cost	✅	✅	✅	✅	❌	✅ (frontier optional)
Bundled — no setup	❌	✅	✅	✅	❌ npm + Node.js	✅
IBM Granite first-class	Runs it	Runs it	Runs it	Runs it	❌	✅ Default
Thermal management	❌	❌	❌	❌	N/A	✅ diff-gated inference

The one-liner on each competitor

• Ollama — the engine WingmanAI runs on top of. Infrastructure, not competition.
• Jan.ai — beautiful local ChatGPT. You go to it. It doesn't watch you.
• LM Studio — great local model runner and developer API. You go to it.
• Unsloth — the best tool for training models. Different job entirely.
• OpenClaw — your AI in your messaging apps, from anywhere. Needs cloud API keys. No screen awareness.
• WingmanAI — the only tool that watches what you're doing right now and helps without being asked, entirely on your machine.

---

Requirements

• macOS — Apple Silicon recommended (M1/M2/M3/M4)
• RAM — 16GB minimum, 32GB+ recommended (models need ~10GB total)
• Storage — ~8GB for default models
• macOS permissions — Screen Recording + Accessibility (guided at first launch)
• No Ollama installation needed — bundled inside the app

---

Alpha Status

This is early alpha software. Here's what that means honestly:

Working:

• Core overlay UI with Play/Pause/Stop transport controls
• Screen capture and perceptual hash diffing
• LLaVA vision inference on screen changes
• Granite reasoning engine with tool call loop
• ApprovalGate for shell and AppleScript actions
• Full two-way voice (STT + TTS)
• Web search via DuckDuckGo
• Memory persistence across sessions
• Session summary files
• Performance monitor strip

In progress / rough:

• First-launch flow (model downloads, permission setup)
• Text selection action card (Focus Awareness)
• Frontier model escalation
• Git awareness and commit message generation
• Some intelligence features (meeting mode, stuck detection) need real-world tuning

Known issues:

• Occasional crash on startup if Ollama port conflict exists
• Vision transparency log UI needs polish
• Settings screen not fully wired to all runtime settings yet

---

Building from Source

# Clone the repo
git clone https://github.com/YOUR_USERNAME/WingmanAI.git
cd WingmanAI

# Install Flutter dependencies
flutter pub get

# Build debug
flutter build macos --debug

# Run
flutter run -d macos

The bundled Ollama runtime is included in macos/Runner/ollama/. The Xcode build phase copy_ollama.sh copies it into the .app bundle automatically.

---

Architecture

WingmanAI.app
├── Swift (~200 lines)           — Window control, STT, TTS, mouse/selection tracking
└── Dart / Flutter               — All business logic
    ├── core/                    — Pure Dart, zero Flutter imports
    │   ├── ollama/              — Bundled Ollama lifecycle + streaming inference
    │   ├── vision/              — VisionProvider abstraction (LLaVA default)
    │   ├── screen/              — ScreenWatcher + perceptual hash diff
    │   ├── reasoning/           — ReasoningEngine (Granite + tool calls)
    │   ├── frontier/            — Optional cloud escalation (GPT-4o, Claude, Grok)
    │   ├── tasks/               — Serial task queue (parallel hook for future)
    │   ├── memory/              — MemoryStore + ProjectProfiles
    │   ├── session/             — Session summary writer + loader
    │   ├── intelligence/        — StuckDetector, MeetingMode, ErrorDetector, etc.
    │   └── ...                  — 90+ additional pure Dart modules
    └── ui/                      — Flutter widgets
        ├── overlay/             — CompactCard, ExpandedChat, all cards
        ├── firstlaunch/         — Startup flow screens
        └── settings/            — Full settings screen

State management: pure Dart streams + setState(). No Riverpod, no GetX, no Redux.

See DESIGN.md for the full design document and PLAN.md for the implementation plan with competitive analysis.

---

Contributing

Alpha stage — the codebase is moving fast. If you want to contribute:

1. Read DESIGN.md and PLAN.md first — they document the full intended design
2. All lib/core/ files must have zero Flutter imports (pure Dart)
3. All lib/ui/ files use Flutter normally
4. Follow the existing AppColors / AppTheme throughout
5. Every destructive action (shell, AppleScript, file write) must go through ApprovalGate

---

License

MIT

---

WingmanAI — named after Goose, Maverick's wingman who always had his back.