Skip to content

DEVELOPER_ONBOARDING.md

Latest commit

 

History

History
164 lines (131 loc) · 7.15 KB

DEVELOPER_ONBOARDING.md

File metadata and controls

164 lines (131 loc) · 7.15 KB
Raw

👨‍💻 Developer & Contributor Onboarding Guide

Welcome to the DreamBees MLX Studio technical contributor onboarding guide. This document covers system architecture, IPC protocol contracts, environment resolution mechanics, and development workflows.

🤖 Operating as an AI Agent? See docs/AGENT_ONBOARDING.md for specialized AI agent operating protocols, stdio/MCP tool interfaces, dynamic VRAM contracts, and BroccoliDB handoffs.

🏗️ Architectural Overview

The application is structured into four main layers:

+-------------------------------------------------------------------+
|                        REACT RENDERER (Vite)                       |
|   DreamBeesStudio.tsx • StudioCanvas.tsx • ModelHub.tsx • React 19   |
+-------------------------------------------------------------------+
                                  │
                          Electron IPC Bridge
                                  │
+-------------------------------------------------------------------+
|                       ELECTRON MAIN PROCESS                        |
|   broccolidb/ Substrate • sidecar_supervisor.ts • database.ts     |
+-------------------------------------------------------------------+
                                  │
                        stdio JSON-RPC IPC
                                  │
+-------------------------------------------------------------------+
|                     PYTHON MLX SIDECAR DAEMON                     |
|    mlx_image_daemon.py • mlx-lm Gemma 4 • TAESD • Metal Core      |
+-------------------------------------------------------------------+
  1. Frontend UI (React 19 + Vite):
    • Located at src/. Renders the ChatGPT-style interface (DreamBeesStudio.tsx), Studio Canvas, Model Hub, Gallery View, and Hardware Diagnostic Widget.
  2. Onboard Gemma 4 Prompt Rewriter Engine:
    • Located at src/core/gemmaPromptEngine.ts. Manages GemmaIntentFunnel, GemmaCircuitBreaker, single-pass XML tag parsing (parseGemma4Response), and V8 monomorphic hidden class shape factories.
  3. Electron Main Process & BroccoliDB Substrate:
    • Located at electron/ and broccolidb/. Manages window lifecycle, BroccoliDB operational substrate (Connection, Workspace, AgentContext), isolated SQLite database (better-sqlite3), and process supervision (sidecar_supervisor.ts).
  4. Touchless Environment Resolver (electron/environment_resolver.ts):
    • Probes system Python binaries and constructs isolated python_env virtualenvs automatically on demand.
  5. Python MLX Daemon (electron/mlx/mlx_image_daemon.py):
    • Runs native Apple Silicon Metal GPU inference, onboard Gemma 4 LLM text generation (mlx-lm), computes flow-matching velocity ($v_t$) and SNR telemetry, and streams sub-millisecond TAESD latent preview frames.

🚀 Setting Up Your Local Development Environment

Prerequisites

  • macOS on Apple Silicon (M1/M2/M3/M4)
  • Node.js >= 20
  • Python 3 >= 3.10

1. Clone & Install Dependencies

git clone https://github.com/DreamBees/DreamBeesMLX.git
cd DreamBeesMLX
npm install

2. Start Application in Dev Mode

npm run dev

3. Verification Command Menu

Scope Command Description
Production Build npm run build Full React client & Electron main production bundle
Type Check npx tsc -p tsconfig.json --noEmit Strict TypeScript compilation & type safety check
Python Daemon Check python3 -m py_compile electron/mlx/mlx_image_daemon.py Verify Python sidecar syntax & dependencies
MCP Server Build npm run build:mcp Build stdio MCP Server for AI Agents
Benchmark Suite python3 tests/benchmarks/run_benchmarks.py Run 10-category Metal GPU automated test suite

🛰️ IPC Protocol Contracts

Stdio JSON-RPC Messages (Sidecar Daemon <-> Electron)

Request Payload:

{
  "action": "generate",
  "payload": {
    "prompt": "Cyberpunk neon bee hovering over futuristic Tokyo night",
    "model_id": "flux2-klein-4b",
    "width": 512,
    "height": 512,
    "steps": 2,
    "guidance_scale": 1.0,
    "seed": 42,
    "output_path": "/tmp/test.png"
  }
}

Response Stream (Step Telemetry):

{
  "type": "progress",
  "payload": {
    "step": 1,
    "total_steps": 2,
    "progress_pct": 50,
    "elapsed_ms": 3074,
    "step_ms": 97,
    "its_per_sec": 10.31,
    "sigma_level": 0.5,
    "flow_velocity": 0.5,
    "snr_db": 0.0,
    "vram": { "active_mb": 2087, "peak_mb": 3569, "cache_mb": 3047 }
  }
}

🔒 Memory Leak & Storage Persistence Architecture

  1. SQLite Storage & Disk Pressure Guard:

    • Base64 images are offloaded to PNG files under userData/generations/, keeping DB rows small (~200 bytes).
    • 2 GB byte-quota engine evicts oldest LRU files when disk cache budget is exceeded.
    • Atomic file swapping (.tmp -> fsyncSync -> renameSync) prevents corrupted files on crash.

  2. Sidecar & Main IPC Lifecycles:

    • Line-buffered stream decoding (stdoutBuffer) in SidecarSupervisor prevents unparsed JSON chunks.
    • Singleton supervisor IPC binding (ensureSidecarSupervisor()) prevents duplicate event listeners.
    • Process cleanup (removeAllListeners()) is invoked on child exit and app shutdown.

  3. Metal VRAM & Ironclad Memory Policy:

    • Single Active Model Enforcement: _LOADED_MODELS retains only 1 active image model; loading a new model card automatically evicts previous models from memory.
    • Onboard Gemma LLM cache (_GEMMA_CACHE) is purged immediately after prompt rewriting to release ~2 GB of text model VRAM.
    • Dynamic Tiered Memory Limits (mx.set_memory_limit dynamically auto-scaled at 25%–30% for 4B models and 30%–40% for 6B/9B models like Z-Image Turbo / FLUX 9B, mx.set_cache_limit 64MB–256MB) strictly reserve 60% to 75% of system RAM for macOS WindowServer.
    • Stepwise Graph Collapsing & GPU Sync: mx.eval(latents), mx.synchronize(), and mx.clear_cache() run on every sampling step to reclaim intermediate Metal arrays.
    • Hardware VRAM Safety Circuit Breaker: Automatically halts generation safely (RuntimeError) if active VRAM hits the 75% system hardware RAM safety limit or MLX memory ceiling.

  4. React DOM Memory Controls:

    • All IPC progress and completion listeners in useEffect return explicit unsubscription functions.
    • Step previews in React state are capped to 16 items and base64 strings are evicted on unmount/completion.
    • DOM image elements use loading="lazy" and decoding="async" to prevent V8 offscreen bitmap memory starvation.

🛠️ Live VRAM & Memory Profiling Diagnostics

Run these diagnostic commands to verify live memory health and VRAM telemetry on macOS:

# 1. Daemon Hardware Diagnostics & Metal Availability Check
python3 electron/mlx/mlx_image_daemon.py --test

# 2. Inspect Total System Hardware Memory via sysctl
sysctl -n hw.memsize

# 3. Monitor Process Memory & Niceness Priority
ps aux | grep -E "mlx_image_daemon|Electron" | grep -v grep

# 4. Strict TypeScript & Python Compiler Checks
npx tsc -p tsconfig.json --noEmit
python3 -m py_compile electron/mlx/mlx_image_daemon.py