🤖 Awesome LLM Agent Protocols

A curated list of protocols, standards, and frameworks for integrating LLM agents with applications, tools, and APIs.

Curated by Yiling Li — author of an in-progress CCF-B paper on frontend agent page-scheduling protocols.

---

🤔 Why this list?

LLM agent integration is fragmented across at least 4 distinct layers (frontend, tool, framework, cross-process), each with its own emerging standards. Engineers often confuse "OpenAI Function Calling vs MCP vs LangChain Tools" without realizing they solve different problems.

This list:

• Categorizes by layer, not by vendor
• Honestly compares what each protocol does well and badly
• Highlights gaps where no good protocol exists (frontend page-level orchestration)
• Updated for 2026 state-of-art (post-MCP, post-Function-Calling stabilization)

---

📋 Table of Contents

1. Layer Map: Where Each Protocol Lives
2. Tool / Function Layer (Backend)
3. Cross-Process Layer
4. Frontend / UI Layer
5. Agent Framework Layer
6. Browser Automation Layer
7. Comparison Matrix
8. Decision Tree: Which Should I Use?
9. Standards in Progress
10. Further Reading
11. Contributing

---

🗺️ Layer Map: Where Each Protocol Lives

┌──────────────────────────────────────────────────────────┐
│              User · Natural Language Input               │
└──────────────────────────┬───────────────────────────────┘
                           │
┌──────────────────────────▼───────────────────────────────┐
│  Agent Framework Layer                                   │
│  (ReAct · AutoGen · CrewAI · LangGraph · Mastra)         │
│  → orchestrates reasoning loops, plans multi-step tasks  │
└──────────────────────────┬───────────────────────────────┘
                           │
            ┌──────────────┴──────────────┐
            ▼                             ▼
┌────────────────────────┐    ┌──────────────────────────┐
│  Frontend / UI Layer   │    │  Tool / Function Layer   │
│  (Manifest · Browser   │    │  (Function Calling ·     │
│   Use · ARIA-AI)       │    │   Anthropic Tool Use)    │
│  → operates Web UI     │    │  → calls in-process fn   │
└─────────┬──────────────┘    └────────────┬─────────────┘
          │                                │
          ▼                                ▼
┌──────────────────────────────────────────────────────────┐
│  Cross-Process Layer                                     │
│  (MCP · OpenAI Plugins · ChatGPT Actions)                │
│  → invokes remote tool servers                           │
└──────────────────────────────────────────────────────────┘

Critical observation: most engineers default to Tool/Function Calling for everything. But frontend page-level orchestration (cross-page workflows, DOM timing, state awareness) is a distinct problem that Tool Calling solves badly. This is the gap Manifest-style protocols fill.

---

🔧 Tool / Function Layer (Backend)

In-process tool invocation. The LLM picks a function name + arguments; your code executes; you return JSON.

OpenAI Function Calling

Maintainer: OpenAI Spec status: Stable, widely adopted as de-facto standard Best for: Backend tool calls in OpenAI / Azure OpenAI / compatible APIs (vLLM, LiteLLM) Strengths: Mature SDK across all major languages; structured output via JSON schema Weaknesses: No state between calls; no native multi-step orchestration; no UI awareness

tools = [{"type": "function", "function": {"name": "get_weather", "parameters": {...}}}]

Anthropic Tool Use

Maintainer: Anthropic Spec status: Stable since Claude 3 Best for: Backend tool calls with Claude models Strengths: Better at tool selection reasoning than GPT-4; supports parallel tool use; computer-use mode for UI Weaknesses: Vendor-locked; slightly different schema from OpenAI

Google Genkit Tools

Maintainer: Google / Firebase Spec status: GA 2024, evolving Best for: Vertex AI / Gemini production deployments Strengths: Tight Firebase integration; built-in observability Weaknesses: Less ecosystem maturity vs OpenAI

Pydantic AI

Maintainer: Pydantic team Spec status: Stable Best for: Type-safe Python agent code with Pydantic validation Strengths: Best-in-class type safety; multi-vendor LLM support Weaknesses: Python-only

---

🌐 Cross-Process Layer

Tool invocation across process / network boundaries. The LLM client and tool server run separately and communicate over a defined wire protocol.

Model Context Protocol (MCP) ⭐

Maintainer: Anthropic (open spec) Spec status: Stable v1, rapidly growing ecosystem Best for: Tool servers exposed to multiple LLM clients (Claude Desktop, Cline, Cursor, etc.) Strengths: Open spec; exec-time tool discovery; reusable across clients; secure capability scoping Weaknesses: Initial setup complexity; not designed for in-process or frontend use Reference servers: github.com/modelcontextprotocol/servers

ChatGPT Actions (formerly OpenAI Plugins)

Maintainer: OpenAI Spec status: Stable for GPT customizations Best for: Exposing REST APIs to ChatGPT custom GPTs Strengths: OpenAPI-based, low learning curve Weaknesses: Limited to ChatGPT ecosystem; OpenAI Plugins predecessor was deprecated

---

🖥️ Frontend / UI Layer

Letting LLM agents operate web/desktop UIs. This layer is the most fragmented because UI semantics are richer than function signatures.

Manifest Architecture ⭐ (this maintainer's work)

Maintainer: @Yukibei Spec status: Production-validated on 21 pages; CCF-B paper in progress Best for: Vue/React webapps where AI agents need to operate multiple pages Strengths: Page self-description; reference-counted reactive registry; 9-action universal adapter (O(1) onboarding); 4-layer DOM timing resilience Weaknesses: Vue-first (React port not yet open-sourced); requires data-agent-key markers Repo: github.com/Yukibei/manifest-architecture-demo Paper: 《基于用户意图认知行为的前端智能体页面调度诱导系统研究》(in submission)

Anthropic Computer Use

Maintainer: Anthropic Spec status: Beta Best for: General-purpose desktop / browser automation via screenshots Strengths: No app-side instrumentation needed Weaknesses: Slow (per-screenshot inference); high cost; non-deterministic vs declarative protocols

Browser Use

Maintainer: Browser-Use team (open source) Spec status: Active development Best for: Python-driven browser automation by LLM agents Strengths: Combines Playwright + DOM extraction + LLM planning Weaknesses: Headless-first (UX-side automation in real apps less natural)

ARIA + LLM Accessibility Tree

Maintainer: W3C Spec status: ARIA stable, AI integration emerging Best for: Using accessibility metadata as LLM input Strengths: Already deployed on most production sites; standardized Weaknesses: Not designed for AI orchestration (intent triggers, scripts, results); read-mostly

---

🧠 Agent Framework Layer

Higher-level orchestration: chain reasoning + tool use into autonomous loops.

LangChain / LangGraph

Maintainer: LangChain Inc. Spec status: Stable, large ecosystem Best for: Multi-step agent workflows in Python/JS Strengths: Largest ecosystem; LangGraph adds proper state machines Weaknesses: Abstraction overhead; debugging complexity; performance vs raw API

LlamaIndex Agents

Maintainer: LlamaIndex Best for: RAG-heavy agent workflows Strengths: First-class document/index integration

Microsoft AutoGen

Maintainer: Microsoft Research Best for: Multi-agent collaboration research Strengths: Explicit multi-agent conversation patterns Weaknesses: Steeper learning curve; production-readiness improving

CrewAI

Maintainer: CrewAI Best for: Role-based multi-agent teams Strengths: Intuitive role + task abstractions

Microsoft Semantic Kernel

Maintainer: Microsoft Best for: Enterprise .NET / Java / Python agent integration Strengths: Strong typing; planner abstractions

Mastra

Maintainer: Mastra Best for: TypeScript-first agent framework Strengths: Modern DX, integrates with Vercel AI SDK

Vercel AI SDK

Maintainer: Vercel Best for: Next.js / React applications Strengths: Streaming-first, edge-optimized

ReAct (Yao et al., 2023)

Type: Foundational technique, not a framework Best for: Understanding the reasoning + acting loop pattern Note: Most modern agent frameworks implement variations of ReAct internally

---

🤖 Browser Automation Layer

General-purpose programmatic browser control, increasingly used as LLM action surface.

Playwright

Maintainer: Microsoft Best for: Reliable cross-browser automation; LLMs can call Playwright API as tool Strengths: Most reliable selector engine; auto-waiting; trace viewer

Puppeteer

Maintainer: Google Chrome team Best for: Chrome-only automation Strengths: Mature; tight Chrome integration

Selenium

Maintainer: Selenium project Best for: Enterprise legacy / IE11 support Strengths: Largest historical ecosystem Weaknesses: Slower vs Playwright; flakier

---

📊 Comparison Matrix

Protocol	Layer	Stateful	Multi-Step	Open Spec	UI Awareness	Onboarding Cost
OpenAI Function Calling	Tool	❌	❌	✅	❌	O(N)
Anthropic Tool Use	Tool	❌	⚠️	✅	❌	O(N)
MCP	Cross-process	⚠️	⚠️	✅	❌	O(N)
Manifest ⭐	Frontend	✅ Reactive	✅ Pipeline Chain	✅ MIT	✅ Strong	O(1)
Anthropic Computer Use	Frontend	❌	⚠️	⚠️	✅ Vision	O(0)*
Browser Use	Frontend	⚠️	✅	✅	✅ DOM	O(0)*
LangChain Tools	Framework	⚠️	✅	✅	❌	O(N)
LangGraph	Framework	✅	✅	✅	❌	O(N)
AutoGen	Framework	✅	✅	✅	❌	O(N)
CrewAI	Framework	✅	✅	✅	❌	O(N)
Semantic Kernel	Framework	✅	✅	✅	❌	O(N)
Vercel AI SDK	Framework	⚠️	✅	✅	❌	O(N)
Playwright (as tool)	Browser	✅ Session	✅	✅	✅ DOM	O(0)*

* "O(0) onboarding" for vision/scraper-based approaches because they don't require app-side instrumentation. Tradeoff: significantly higher inference cost and lower reliability.

---

🌳 Decision Tree: Which Should I Use?

Q1: Are you building a backend that needs LLM to call functions?
    ├── Yes, in-process tool calls
    │   └── Use OpenAI Function Calling / Anthropic Tool Use
    │
    └── Yes, but tools are remote services
        └── Use MCP (if expecting reuse) or REST + Function Calling

Q2: Are you building a frontend webapp where LLM operates the UI?
    ├── Vue/React webapp with multiple pages, state matters
    │   └── Use Manifest Architecture (this maintainer's work)
    │
    ├── Generic browser automation, OK with screenshot inference cost
    │   └── Use Anthropic Computer Use or Browser Use
    │
    └── Highly accessible existing site, read-mostly agent
        └── Try ARIA-tree based prompting

Q3: Do you need multi-agent collaboration?
    ├── Role-based teams
    │   └── Use CrewAI
    │
    ├── Conversation-pattern research
    │   └── Use AutoGen
    │
    └── Production state machines
        └── Use LangGraph

Q4: Do you need RAG-heavy workflows?
    └── Use LlamaIndex Agents

Q5: Are you on a TypeScript/Next.js stack?
    └── Use Vercel AI SDK + Mastra

---

🚧 Standards in Progress

• W3C AI Agent Protocol Working Group — early-stage, watch for emerging standards
• IEEE 2840 — Standards for Autonomous Agents (mostly robotics-focused)
• CNCF AI Working Group — cloud-native agent infrastructure patterns

These are not yet production-ready but worth monitoring for long-term protocol convergence.

---

📚 Further Reading

Foundational Papers

• ReAct: Synergizing Reasoning and Acting in Language Models (Yao et al., 2023)
• Toolformer: Language Models Can Teach Themselves to Use Tools (Schick et al., 2023)
• Voyager: An Open-Ended Embodied Agent (Wang et al., 2023)
• Reflexion: Language Agents with Verbal Reinforcement Learning (Shinn et al., 2023)

Production Architecture Studies

• 《基于用户意图认知行为的前端智能体页面调度诱导系统研究》（CCF-B in submission, 2026）— 5/7 章已完成正文 / 12 数学公式 / 2 引理证明
• Anthropic: Building Effective Agents (2024)
• OpenAI: Practices for Governing Agentic AI Systems (2023)

Technical Blogs

• Manifest 架构设计哲学 — 4000 字深度文章 — by @Yukibei
• Anthropic Engineering Blog
• LangChain Blog

---

🤝 Contributing

Found a protocol that should be here? Caught an error? Disagree with a categorization?

Open a GitHub Issue with:

• Protocol name + maintainer + spec link
• Which layer it belongs to (per the layer map)
• 1-2 sentence "best for" + 1-2 strengths + 1 honest weakness

Maintainer commitments:

• All issues replied within 7 days
• Disagreements resolved by reference to public deployment evidence (not opinion)
• This list stays opinionated — no "everything is awesome", we honestly compare tradeoffs

---

📜 License

CC BY 4.0

You may share, adapt, and use commercially. Attribution required: link back to this repository.

Copyright © 2026 Yiling Li

---

⭐ If this list saved you a week of protocol research, a star helps other engineers find it too.

🌐 Production work: Hoop Pupil · 🤖 My demo: manifest-architecture-demo · 📝 My tech blog · 📧 Contact