◈ O P E N A V A T A R ◈

AI-Powered Virtual Agent Framework

Neural cognition · Emotional resonance · Digital embodiment

---

▸ System Overview

OpenAvatar is an open-source virtual agent framework that fuses large language model cognition with 3D avatar embodiment. Built on Unity, it creates a real-time interactive digital persona capable of emotional expression, contextual reasoning, and natural conversation.

┌─────────────────────────────────────────────────────┐
│                   OPENAVATAR CORE                    │
│                                                      │
│  ┌──────────┐   ┌──────────────┐   ┌─────────────┐ │
│  │  LLM      │──▶│  Emotion     │──▶│  VRM Avatar │ │
│  │  Service  │   │  Engine      │   │  Renderer   │ │
│  │          │   │              │   │             │ │
│  │ · Ollama │   │ · Neutral    │   │ · BlendShape│ │
│  │ · Custom │   │ · Thinking   │   │ · Animation │ │
│  │ · Mock   │   │ · Happy      │   │ · LipSync   │ │
│  │          │   │ · Warning    │   │             │ │
│  │          │   │ · Comforting │   │             │ │
│  └──────────┘   └──────────────┘   └─────────────┘ │
│       │                │                  │          │
│       ▼                ▼                  ▼          │
│  ┌──────────────────────────────────────────────┐   │
│  │            Agent Response Pipeline            │   │
│  │   Input → Analyze → Emote → Animate → Speak  │   │
│  └──────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────┘

▸ Core Architecture

Neural Interface — LLMService

Multi-provider LLM orchestration layer with intelligent model resolution:

• Ollama — Local inference with auto-discovery of installed models (qwen2.5, gemma3, etc.)
• Custom JSON API — Pluggable endpoint for any REST-based LLM service
• Mock Mode — Rule-based fallback for offline development and testing

Emotion Engine — AgentEmotion / EmotionExpressionMapper

Five-dimensional emotional state machine with smooth VRM BlendShape transitions:

State	BlendShape Mapping	Visual Cue
Neutral	Baseline	Calm idle
Thinking	Surprised / Sorrow blend	Analytical focus
Happy	Joy @ full weight	Positive engagement
Warning	Angry @ full weight	Alert posture
Comforting	Fun + Joy blend	Empathetic warmth

Avatar Controller — AvatarAgentController

Full-stack avatar orchestration with dual animation pipeline:

• Animator Controller — Trigger-based state machine transitions
• Playable Graph Fallback — Procedural clip blending with smooth crossfade
• Auto-return to idle with configurable buffer timing
• Root motion support per-action

Presentation Engine

Procedurally generated cyberpunk HUD environment:

• Camera auto-framing with FOV calculation
• Runtime-generated backdrop (chamber glow, tech rings, grid overlay)
• Dynamic lighting with cool-tone ambient tuning
• System font resolution for crisp CJK typography

▸ Project Structure

Assets/
├── 00_Scenes/          → Demo scene (AvatarAgent_DemoScene)
├── 02_Audio/           → Voice clips
├── 03_Scripts/
│   ├── Avatar/         → Core avatar systems
│   │   ├── AvatarAgentController.cs    → Master orchestrator
│   │   ├── EmotionExpressionMapper.cs  → VRM emotion driver
│   │   └── SimpleAudioMouthSync.cs     → Audio lip-sync
│   ├── Core/           → Agent intelligence
│   │   ├── LLMService.cs               → Multi-provider LLM client
│   │   ├── AgentEmotion.cs             → Emotion enum
│   │   ├── AgentState.cs               → State enum
│   │   ├── AgentResponse.cs            → Response data model
│   │   └── AgentResponseFactory.cs     → Response construction
│   ├── Demo/           → Demo utilities
│   └── UI/             → HUD controllers
│       └── AgentStateUIController.cs   → Status panel + reply bubble
├── 09_Animations/      → FBX action clips (idle, happy, angry, sad, think, greet)
├── Models/             → VRM avatar (Avatar_Sunny)
└── TextMesh Pro/       → TMP resources

▸ Quick Start

Prerequisites

• Unity 2022.3 LTS or later
• Ollama (optional, for LLM-powered responses) — Install Guide

Setup

# 1. Clone the repository
git clone https://github.com/<owner>/OPEN-AVATAR.git
cd OPEN-AVATAR

# 2. Open in Unity Hub
# Add the project folder to Unity Hub and open with Unity 2022.3+

# 3. (Optional) Start Ollama and pull a model
ollama pull qwen2.5:3b

# 4. Open the demo scene
# Assets/00_Scenes/AvatarAgent_DemoScene.unity

▸ Documentation

Guide	Purpose
Setup	Unity, Mock mode, Ollama, and custom API setup
Architecture	Runtime pipeline, state mapping, and animation strategy
Custom API Contract	JSON request/response contract for external LLM services
Asset Policy	What belongs in version control and what must stay private
Contributing	Pull request expectations and project hygiene
Security	Private reporting and secrets handling

Configuration

In the AvatarAgentController Inspector:

Parameter	Description	Default
Provider Mode	Ollama / CustomJsonApi / Mock	Ollama
Ollama Endpoint	Local Ollama API URL	http://127.0.0.1:11434/api/generate
Ollama Model	Target model name	qwen2.5:3b
Auto Resolve	Auto-detect installed models	true
Custom Endpoint	Your REST API URL	http://127.0.0.1:8000/agent-response

Custom LLM API

OpenAvatar accepts JSON responses from any HTTP endpoint:

{
  "text": "I understand your concern. Let me analyze this.",
  "emotion": "Thinking",
  "state": "Thinking",
  "action": "analyze_input"
}

Allowed values:

• emotion: Neutral, Thinking, Happy, Warning, Comforting
• state: Idle, Thinking, Speaking, Warning, Comforting

▸ Android Build

# Command-line batch build (requires Unity Editor installed)
Unity.exe -quit -batchmode -nographics \
  -projectPath "path/to/OpenAvatar" \
  -executeMethod UnityEditor.BuildPipeline.BuildPlayer \
  -buildTarget Android \
  -buildPath "Build/OpenAvatar.apk"

Or use File → Build Settings in the Unity Editor.

▸ Tech Stack

Layer	Technology
Engine	Unity 2022.3 LTS
Avatar Format	VRM 0.x (via UniVRM)
LLM Runtime	Ollama / Custom REST API
Lip Sync	uLipSync
UI Framework	TextMesh Pro
Shader	MToon (VRM toon shading)
Animation	Unity Playable API + Animator Controller
Target Platform	Android / Windows

▸ Extending OpenAvatar

Adding a New Emotion

1. Add entry to AgentEmotion enum
2. Add corresponding AgentState if needed
3. Map BlendShape weights in EmotionExpressionMapper.BuildProfile()
4. Add animation trigger/clip in AvatarAgentController

Adding a New LLM Provider

1. Add entry to LLMService.ProviderMode enum
2. Implement IEnumerator RequestFrom{Provider}(...) method
3. Wire into GenerateResponse() switch block

Swapping Avatars

Replace the VRM file in Assets/Models/ and update the VRMBlendShapeProxy reference in the scene. OpenAvatar auto-resolves BlendShape mappings at runtime.

▸ License

This project is licensed under the MIT License — see LICENSE for details.

Third-party assets retain their respective licenses:

• UniVRM — MIT License
• uLipSync — MIT License
• Avatar_Sunny.vrm — Replace with your own VRM model for redistribution

---

OpenAvatar — Where silicon meets soul.