A production-grade tool gateway for LLM applications.
Ferrata sits between your LLM and your tools. It handles the things that matter in production — tenant isolation, semantic tool selection, argument validation, permissions, rate limiting, retries, budget enforcement, and compliance-grade audit logging — so your tool functions stay plain async Python and know nothing about governance.
LLM tool call
│
▼
TenantIsolation → scopes tool set to what this tenant is permitted to see
SemanticRouter → selects the most relevant tools for this message
SchemaValidator → validates and coerces arguments; size-limits results
ExecutionSandbox → permission check, rate limit, timeout
│
▼
Tool executes
│
▼
RetryManager → classifies failure, retries or escalates
UsageTracker → budget check; emits OTel metrics and traces
AuditLogger → writes tamper-evident WAL record; emits OTel log
│
▼
Result returned to LLM
The core framework has no external dependencies. For semantic tool selection (recommended), install sentence-transformers:
pip install sentence-transformersfrom ferrata.gateway import ToolGateway
from ferrata.plugins.defaults import InMemoryTenantToolRepository
from ferrata.plugins.embedders import LocalEmbedder
repo = InMemoryTenantToolRepository()
repo.register_tool(my_tool, tenant_ids=["my_tenant"])
gateway = ToolGateway(
repository=repo,
embedding_backend=LocalEmbedder(), # downloads ~90MB on first use, cached after
)
await gateway.startup()
# Get tools for the LLM — semantically ranked for this message
tools = await gateway.get_tools(tenant_id="my_tenant", message="user's message")
# Execute a tool call from the LLM
result = await gateway.execute(
tenant_id="my_tenant",
tool_call=ToolCall(name="my_tool", args={"field": "value"}, id="call-1"),
message="user's message",
tool_fn=my_tool_function,
)Without embedding_backend, the gateway still works but returns all tenant tools
unranked — useful for testing, not recommended for production.
python3 tests/test_gateway.py # gateway integration tests
python3 tests/test_core_models.py # data model tests
# ... all 8 test suites, 480 tests totalNo test framework required — plain Python throughout.
A three-tenant financial services reference implementation with 14 tools, custom validators, SQL injection sanitization, RBAC, and budget enforcement:
python3 examples/fincorp_demo.pyTo run the LangChain + Gemini version (requires API key):
pip install langchain-google-genai langchain-core
export GOOGLE_API_KEY=your_key
python3 examples/fincorp_langchain.pySee examples/README.md for details on both examples.
LangChain is a first-class integration. Ferrata adopts LangChain's tool_calls
format as its canonical tool call shape — no conversion needed. Pass
AIMessage.tool_calls directly to gateway.execute() and feed results back
as ToolMessage objects.
Ferrata works as a drop-in governance layer inside any LangChain or LangGraph agent. LangChain handles orchestration; Ferrata handles who can call what, with what arguments, subject to what limits. The LLM provider — OpenAI, Anthropic, Gemini, Mistral, Ollama — is irrelevant to Ferrata. Swapping providers requires changing one line; nothing in the governance layer changes.
See the Infrastructure & Integration guide for complete LangChain and LangGraph code examples.
ferrata/
ferrata/
gateway.py # ToolGateway — the public entry point
core/ # data models, enums, exceptions
plugins/ # 7 plugins + interfaces + defaults
tests/ # 480 tests, no framework required
examples/
fincorp_demo.py # 3-tenant reference implementation
fincorp_langchain.py # same demo wired into a Gemini agent loop
Ferrata was inspired by:
OpenAI Function Calling Works Great, Until You Have 340 Tools, 12 Tenants, and Real Production Traffic by Teja Kusireddy https://medium.com/@teja.kusireddy23/openai-function-calling-works-great-until-you-have-340-tools-12-tenants-real-production-traffic-fe02da116e39
MIT