AI Factory

AI Factory is a local-first browser RPA and AI workflow platform. It turns natural-language automation goals or hand-written DSL into executable browser workflows, then runs them with Playwright, structured LLM planning, browser profiles, run snapshots, and notification delivery.

The current MVP focuses on one useful path: define a workflow, render it visually, run it locally, inspect every block, rerun from a failed block, and send results through email or Feishu.

中文总览

AI Factory 的目标是让普通业务人员用自然语言描述目标，就能创建可持续运行的“数字员工”。用户不需要理解代码、选择器或浏览器自动化细节，只需要说明要打开哪个后台、提取什么数据、执行什么操作、把结果发送到哪里，系统会生成可视化工作流，并把工作流转换为可执行的浏览器任务。

项目当前主要面向相对稳定、但仍会受页面变化影响的运营流程。典型场景包括自媒体内容平台后台运营、电商后台运营、周期性数据提取、页面状态巡检、运营结果通知等。当页面结构或外部输入变化导致流程失败时，平台会通过运行记录、节点快照、结构化上下文和 AI 执行器提供自愈与修复入口，逐步降低传统 RPA 维护成本。

这个想法来自作者过去在 iPaaS 和 RPA 项目实施过程中遇到的客户问题与交付经验：业务流程需要足够透明，AI 决策需要有边界，运行过程需要可追踪，失败后的修复也应该尽量产品化。

UI Examples

Workflow Editor

Run Detail

Why AI Factory

Most browser automation tools force you to choose between brittle scripts and opaque AI agents. AI Factory keeps the layers explicit:

A product-friendly DSL that users and the frontend can understand.
A canonical graph workflow model that can be validated and versioned.
A runtime layer that records every block run, output, log, error, and screenshot.
A strict boundary where LLMs only decide structured next actions or structured extraction results.
A local-first desktop mode for login sessions, browser profiles, and private credentials.

Features

Visual workflow editor with dynamic panels: Copilot, Editor, Browser, and Overview.
Workflow DSL with canonical block types such as goto_url, action, extraction, ai_browser_task, wait, notification/send_email, notification/send_feishu, and output.
Local Electron runtime backed by SQLite and Playwright Chromium.
Browser Profile capture for logged-in websites using Playwright storage_state.
Non-visual AI browser task execution through page facts and structured actions.
Structured extraction with JSON Schema validation.
Model configuration for SiliconFlow models such as deepseek-ai/DeepSeek-V4-Flash and zai-org/GLM-5.2.
Notification connection adapters for SMTP email and Feishu custom-bot webhooks.
Run history, block timeline, node snapshots, logs, outputs, cancellation, deletion, and rerun-from-block.
FastAPI backend with the same domain boundaries for future remote deployment.

Architecture

flowchart TB
  UI["React + Vite UI"] --> Electron["Electron Main Process"]
  UI -. future remote mode .-> API["FastAPI Backend"]

  Electron --> Gateway["LocalApi / RemoteClient"]
  Gateway --> Store["SQLite Local Store"]
  Gateway --> Runner["LocalWorkflowRunner"]
  Runner --> Executors["Block Executors"]

  Executors --> Runtime["Browser Runtime"]
  Runtime --> BrowserEngine["Playwright / Browser Engine"]

  Executors --> AI["AI Executors"]
  AI --> LLMGateway["LLM Provider Interface"]
  LLMGateway --> DeepSeek["DeepSeek Provider"]
  LLMGateway --> OpenAI["OpenAI Provider"]
  LLMGateway --> OtherLLM["Other Model Providers"]

  Executors --> Notify["Notification Service"]
  Notify --> Delivery["Delivery Channel Interface"]
  Delivery --> Email["Email Provider"]
  Delivery --> Feishu["Feishu Webhook"]
  Delivery --> IM["IM Messenger / Slack / Other Channels"]


  API --> Services["Workflow / Profile / LLM / Notification Services"]
  Services --> DB["SQLite now, PostgreSQL-ready"]

Executors are intentionally provider-agnostic. AI-capable blocks depend on a common LLM provider contract, so the same workflow runtime can use DeepSeek, OpenAI, or other compatible model providers without changing workflow semantics.

Notification delivery follows the same pattern. Workflow blocks produce canonical notification requests, while delivery adapters map those requests to email, Feishu, IM messengers, Slack, or other channels.

Read the full architecture document: docs/dev/hu-design-docs/overall-design.md.

Repository Layout

factory/
  app/                         FastAPI backend and Python runtime services
  alembic/                     Backend database migrations
  scripts/                     Backend helper scripts
  data/                        Local backend data, ignored by git

factory-frontend/
  src/                         React UI
  electron/                    Electron local API, SQLite store, runner, Playwright runtime

docs/
  dev/hu-design-docs/          Architecture and design notes
  design/                      Product concept images

openspec/                      Archived and active change proposals
tests/                         Python backend tests

Quick Start: Desktop MVP

The desktop app is the recommended MVP entry point. It does not require the Python backend in local mode.

cd factory-frontend
npm install
npx playwright install chromium
npm run dev

npm run dev starts Vite and Electron together. The default account email in the local app is operator@....

Useful desktop commands:

npm run build
npm run test:electron
npm run test:e2e-local
npm run test:e2e-notifications

Optional: Run the Python Backend

The backend mirrors the same workflow, profile, LLM, and notification boundaries for future remote mode.

uv sync
cd factory
uv run alembic upgrade head
uv run uvicorn app.main:app --reload --port 8000

Health check:

curl http://127.0.0.1:8000/health

Configuration

Create a repository-level .env for backend demos and secrets. Do not commit it.

SILICONFLOW_API_KEY=your_siliconflow_key
SILICONFLOW_MODEL=zai-org/GLM-5.2
SILICONFLOW_BASE_URL=https://api.siliconflow.cn/v1

JWT_SECRET_KEY=replace-with-at-least-32-bytes
LLM_CONFIG_ENCRYPTION_KEY=replace-with-a-deployment-secret
BROWSER_PROFILE_ENCRYPTION_KEY=replace-with-a-deployment-secret
NOTIFICATION_CONFIG_ENCRYPTION_KEY=replace-with-a-deployment-secret

Electron local mode stores secrets through the OS-backed Electron safeStorage API. Backend mode encrypts persisted model keys, browser profile state, and notification credentials with deployment secrets.

Storage mode:

# Default local-first mode
AIFACTORY_STORAGE_MODE=local npm --prefix factory-frontend run dev

# Future remote mode
AIFACTORY_STORAGE_MODE=remote AIFACTORY_API_URL=http://127.0.0.1:8000 npm --prefix factory-frontend run dev

Workflow DSL Example

{
  "unique_id": "toutiao_hotlist_workflow",
  "display_name": "今日头条热榜提取",
  "version": 1,
  "inputs": {},
  "runtime": {
    "browser_profile_id": null
  },
  "blocks": [
    {
      "unique_id": "open_toutiao",
      "display_name": "打开今日头条",
      "block_type": "goto_url",
      "config": {
        "url": "https://www.toutiao.com/",
        "wait_until": "domcontentloaded"
      }
    },
    {
      "unique_id": "extract_data",
      "display_name": "提取头条热榜前5条",
      "block_type": "extraction",
      "config": {
        "kind": "structured",
        "instruction": "提取页面中“头条热榜”的前5条记录。每条包含排名、标题和热度。",
        "schema": {
          "type": "object",
          "properties": {
            "hot_list": {
              "type": "array",
              "minItems": 5,
              "maxItems": 5,
              "items": {
                "type": "object",
                "properties": {
                  "rank": { "type": "number" },
                  "title": { "type": "string" },
                  "hotness": { "type": ["string", "null"] }
                },
                "required": ["rank", "title"]
              }
            }
          },
          "required": ["hot_list"]
        }
      }
    },
    {
      "unique_id": "send_feishu",
      "display_name": "发送飞书通知",
      "block_type": "notification/send_feishu",
      "config": {
        "content": "工作流已执行完成：{{$blocks.extract_data.output}}",
        "content_type": "text"
      }
    },
    {
      "unique_id": "output",
      "display_name": "输出结果",
      "block_type": "output",
      "config": {
        "values": "$blocks.extract_data.output"
      }
    }
  ],
  "edges": [
    { "from_unique_id": "open_toutiao", "to_unique_id": "extract_data" },
    { "from_unique_id": "extract_data", "to_unique_id": "send_feishu" },
    { "from_unique_id": "send_feishu", "to_unique_id": "output" }
  ],
  "entry": "open_toutiao",
  "outputs": {},
  "standalone_blocks": []
}

Notification variables intentionally support only block output references:

{{ $blocks.extract_data.output }}
{{$blocks.extract_data.output}}
$blocks.extract_data.output

API Surface

The desktop LocalApi currently exposes the primary /api/v1/... contract used by the UI. The FastAPI backend implements the same core domain boundaries and is being kept aligned for future remote mode.

POST /api/v1/auth/login
GET /api/v1/llm/models
POST /api/v1/llm/models
GET /api/v1/browser-profiles
POST /api/v1/browser-profiles
GET /api/v1/notification-connections
POST /api/v1/notification-connections
POST /api/v1/notification-connections/{id}/test
POST /api/v1/workflows/generate
GET /api/v1/workflows
POST /api/v1/workflows
POST /api/v1/workflows/{id}/runs
GET /api/v1/workflow-runs
GET /api/v1/workflow-runs/{id}
POST /api/v1/workflow-runs/{id}/blocks/{block_id}/rerun
POST /api/v1/workflow-runs/{id}/cancel

Backend Demo

factory/app/main.py includes a Playwright + SiliconFlow demo that opens Toutiao, extracts the first five hot-list items, and returns the final output.

uv run python factory/app/main.py

To run a Browser Profile-bound demo, set:

DEMO_BROWSER_PROFILE_ID=<profile-id>
DEMO_BROWSER_PROFILE_OWNER_USER_ID=<owner-user-id>
DEMO_AUTHENTICATED_URL=https://example.com/account

Testing

Backend:

uv run pytest

Frontend and Electron:

cd factory-frontend
npm run build
npm run test:electron

More targeted Electron tests are available in factory-frontend/package.json.

Security Notes

Never commit .env, SQLite databases, browser profile exports, API keys, SMTP passwords, webhook URLs, or generated user data.
Workflow DSL must reference runtime.browser_profile_id; it must not include cookies, authorization headers, or storage-state JSON.
Model keys and notification secrets are never returned by read APIs.
Run snapshots are local artifacts and may contain page data. Treat them as private user data.

Roadmap

Stronger graph execution for branches and loops.
More deterministic browser actions and richer locator strategies.
Remote service mode with sync from local Electron storage.
Queue-based workers for concurrent workflow runs.
Plugin-style notification channels and LLM providers.
Optional Vision / CUA capabilities after the non-visual runtime is stable.
Possible BPMN export or engine bridge after the MVP workflow runtime matures.

Project Status

AI Factory is an early MVP. The local desktop path is the primary working path today; the Python backend provides the matching service architecture and APIs for future remote deployment.

License

License has not been selected yet. Add a LICENSE file before publishing as a public open-source project.

Name		Name	Last commit message	Last commit date
Latest commit History 8 Commits
.github/workflows		.github/workflows
docs		docs
factory-frontend		factory-frontend
factory		factory
openspec		openspec
tests		tests
.gitignore		.gitignore
AGENTS.md		AGENTS.md
README.md		README.md
alembic.ini		alembic.ini
pyproject.toml		pyproject.toml
uv.lock		uv.lock

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

AI Factory

中文总览

UI Examples

Workflow Editor

Run Detail

Why AI Factory

Features

Architecture

Repository Layout

Quick Start: Desktop MVP

Optional: Run the Python Backend

Configuration

Workflow DSL Example

API Surface

Backend Demo

Testing

Security Notes

Roadmap

Project Status

License

About

Releases

Packages

Contributors

Languages

Folders and files

Latest commit

History

Repository files navigation

AI Factory

中文总览

UI Examples

Workflow Editor

Run Detail

Why AI Factory

Features

Architecture

Repository Layout

Quick Start: Desktop MVP

Optional: Run the Python Backend

Configuration

Workflow DSL Example

API Surface

Backend Demo

Testing

Security Notes

Roadmap

Project Status

License

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages