feat(inventory): publish introspected tool/permission inventory and honest cloud tiers

AGENTS.md

@@ -2,7 +2,7 @@
 
 ## Project Overview
 
-OpenDevOps Agent is an open-source AWS incident investigation tool powered by any LLM via LiteLLM. It runs a LangGraph ReAct loop (via DeepAgents) that calls 27 tools — 21 structured boto3 AWS tools plus bash, history analytics, skills, and a structured final-answer tool — then streams results to a React/Vite chat UI over SSE. Auth, multi-user RBAC, event-driven incident detection (EventBridge → SQS), and proactive anomaly polling are all built in and optional.
+OpenDevOps Agent is an open-source AWS incident investigation tool powered by any LLM via LiteLLM. It runs a LangGraph ReAct loop (via DeepAgents) that calls 26 tools — 20 structured boto3 AWS tools plus bash, history analytics, skills, and a structured final-answer tool — then streams results to a React/Vite chat UI over SSE. Auth, multi-user RBAC, event-driven incident detection (EventBridge → SQS), and proactive anomaly polling are all built in and optional.
 
 ---
 
@@ -48,6 +48,7 @@ to core are live without republishing.
 - **New env var:** if core reads it, add the Pydantic field to `CoreSettings` in `apps/core/src/opendevops_core/config.py`; if it's web/auth-only, add it to `Settings(CoreSettings)` in `apps/backend/src/config/appsettings.py`. Either way mirror it in `.env.example` (with a comment). Never read env vars directly — always go through `settings`.
 - **New DB column or table:** add a new numbered migration. Core-domain schema (tables core code reads/writes) goes in `apps/core/src/opendevops_core/migrations/` (e.g. `014_name.sql`); OSS-app-only schema goes in `apps/backend/migrations/`. The runner applies core-then-app, tracked in the `schema_migrations(source, version)` ledger. Never add columns inline in Python code.
 - **New tool:** add it to `ALL_TOOLS` in `apps/core/src/opendevops_core/agent/core.py`. Tool functions must be plain synchronous Python functions — DeepAgents infers the JSON schema from type hints and docstrings.
+- **Published tool/permission inventory:** the trust artifact (read-only `GET /api/inventory` + the generated `apps/documentation/tool_inventory.md`) is built **by introspection** in `opendevops_core.agent.inventory.build_inventory` — it never hand-maintains a list. Its sources of truth are `ALL_TOOLS`, the bash allowlist frozensets in `tools/bash_tool.py` (`_AWS_READONLY_VERBS`, `_AZ_READONLY_VERBS`, `_KUBECTL_SUBCOMMANDS`, `_DOCKER_SUBCOMMANDS`, blocked flags), and `providers/aws/permissions.py:PERMISSION_PROBES`. After changing any of those, regenerate the doc: `cd apps/backend && uv run python scripts/gen_tool_inventory.py`. Never edit `tool_inventory.md` by hand. Note: the introspected counts are **26 total tools / 20 structured AWS tools** (CloudWatch 6, CloudTrail 1, ECS 4, Lambda 3, EC2 2, RDS 2, IAM 2) — trust the inventory as the source of truth.
 - **New API route that matches a React Router path:** prefix it with `/api/` to avoid the SPA fallback conflict. The `/{full_path:path}` catch-all in `apps/backend/src/api/app.py` intercepts any GET that matches a registered FastAPI route first.
 - **New skill:** drop a `SKILL.md` file into `apps/core/src/opendevops_core/skills/<name>/SKILL.md`. It is picked up automatically at startup (and bundled into the core wheel) — no code changes needed. Use the frontmatter format (`name`, `description`) from the existing `lambda-throttling` skill.
 - **Docs sync:** if a feature has a corresponding file in `apps/documentation/`, update it when the feature changes. The `apps/documentation/` folder is the public documentation source.
@@ -103,14 +104,14 @@ Everything below is built and working in the codebase:
 
 ### Agent & Tools
 - **Framework:** DeepAgents (`create_deep_agent`) wrapping a LangGraph ReAct loop. `ChatLiteLLM` as the model interface — supports OpenRouter, Anthropic, OpenAI, Groq, Ollama, and any OpenAI-compatible endpoint via a single `LLM_MODEL` env var.
-- **27 tools total** registered in `ALL_TOOLS` in `apps/core/src/opendevops_core/agent/core.py`:
+- **26 tools total** registered in `ALL_TOOLS` in `apps/core/src/opendevops_core/agent/core.py`:
   - CloudWatch (6): `get_alarms`, `get_alarm_history`, `get_metric_data`, `get_log_events`, `describe_log_groups`, `query_logs_insights`
-  - CloudTrail (2): trail events + event lookup
+  - CloudTrail (1): event lookup
   - ECS (4): clusters, services, service detail, tasks
-  - Lambda (4): list, config, error rate, concurrent executions
+  - Lambda (3): list, config, error rate
   - EC2 (2): list instances, instance details
   - RDS (2): list DBs, DB details
-  - IAM (1): describe role + policies
+  - IAM (2): caller identity + describe role policies
   - Bash (1): `run_bash_command` — allowlisted read-only `aws`, `kubectl`, `docker` commands; never `shell=True`; 30s hard timeout
   - History (2): `get_investigation_history`, `search_past_investigations`
   - Skills (2): `list_skills`, `use_skill`

README.md

@@ -43,19 +43,31 @@ On a **reproducible 10-incident suite** (real AWS + Azure resources, scored agai
 |  | **OpenDevOps** | **AWS DevOps Agent / Q Developer** |
 |---|---|---|
 | **LLM** | Any (LiteLLM, Claude Code, Ollama) | Bedrock-managed only |
-| **Cloud coverage** | AWS + Azure (more coming) | AWS only |
+| **Cloud coverage** | AWS (structured tools + CLI) + Azure (CLI + skills) | AWS only |
 | **Data location** | Your DB / VPC | AWS-managed, not portable |
 | **Customization** | Open source — modify anything | Closed product |
 | **Pricing** | LLM at retail (or $0 via Ollama / Claude Code) | Per-investigation + Bedrock markup |
 | **Self-host** | Docker / Railway / on-prem / air-gapped | No |
 
 **When AWS is the better pick:** if you're 100% AWS, never plan to leave, and want zero infrastructure to run, Amazon Q Developer's native Console integration and AWS-only signals (Trusted Advisor, AWS Config, Compute Optimizer) are hard to beat. OpenDevOps is for everyone else.
 
+### Cloud coverage — honest tiers
+
+Multi-cloud means different things per provider. Here is exactly where each cloud stands today (numbers are introspected from the code and published in the [tool inventory](apps/documentation/tool_inventory.md)):
+
+| Cloud | Structured SDK tools | CLI investigation (`bash` tool) | Runbook skills | Event-driven + polling | Status |
+|---|---|---|---|---|---|
+| **AWS** | 20 (CloudWatch, CloudTrail, ECS, Lambda, EC2, RDS, IAM) | `aws` (read-only verbs) | ✅ | ✅ EventBridge → SQS + metric poller | **Complete** |
+| **Azure** | 0 | `az` + `kubectl` (AKS) | ✅ 4 (AKS, App Service, Monitor/KQL, VM) | ❌ | **CLI + skills** |
+| **GCP** | 0 | — | — | ❌ | **Not implemented** (stub returns no tools) |
+
+AWS is the complete path (structured tools **and** CLI). Azure is investigated CLI-first through the read-only `az`/`kubectl` allowlist plus runbook skills — there are **no** structured Azure SDK tools and **no** autonomous detection loop. GCP is a stub: the provider loads and returns zero tools.
+
 ## What's inside
 
 - **LangChain DeepAgents** as the agent framework — planning, tool orchestration, and session memory out of the box
-- **21 read-only AWS tools** across CloudWatch (6), CloudTrail (2), ECS (4), Lambda (4), EC2 (2), RDS (2), IAM (1), plus bash escape hatch, cross-session history analytics, skills, and `submit_investigation` — plain Python functions, schemas inferred automatically
-- **Azure support (CLI-first)** — investigates Azure through the read-only `az` CLI + `kubectl` (for AKS) and a set of Azure runbook skills (AKS debugging, App Service errors, Azure Monitor/KQL, VM diagnostics) — no separate SDK tools needed. Read-only; connect via a service principal or `az login` — see [apps/documentation/azure_setup.md](apps/documentation/azure_setup.md)
+- **20 read-only AWS tools** across CloudWatch (6), CloudTrail (1), ECS (4), Lambda (3), EC2 (2), RDS (2), IAM (2), plus bash escape hatch, cross-session history analytics, skills, and `submit_investigation` — plain Python functions, schemas inferred automatically. The complete, always-current list (with parameters, the bash allowlist, and the AWS permission matrix) is published as a trust artifact — see [apps/documentation/tool_inventory.md](apps/documentation/tool_inventory.md) or the read-only `GET /api/inventory` endpoint
+- **Azure support (CLI-first)** — investigates Azure through the read-only `az` CLI + `kubectl` (for AKS) and a set of Azure runbook skills (AKS debugging, App Service errors, Azure Monitor/KQL, VM diagnostics) — **no structured SDK tools and no event-driven/polling loop** (those are AWS-only). Read-only; connect via a service principal or `az login` — see [apps/documentation/azure_setup.md](apps/documentation/azure_setup.md)
 - **Sandboxed bash execution tool** — agent can run whitelisted read-only AWS CLI (`aws`), Azure CLI (`az`), kubectl, and docker commands as a last resort when the structured tools fall short; every command validated against an allowlist before execution; never uses `shell=True`; hard 30-second timeout
   - Includes **CloudWatch Logs Insights** (`query_logs_insights`) — full query language support: `fields`, `filter`, `stats`, `sort`, `limit`; results include scanned MB
 - **Streaming responses** — FastAPI SSE endpoint streams agent tokens in real time as the LLM reasons; tool calls appear as they complete

apps/backend/scripts/gen_tool_inventory.py

@@ -0,0 +1,142 @@
+"""Generate apps/documentation/tool_inventory.md from the live introspected inventory.
+
+This page is generated, never hand-edited — it is sourced from the same
+``build_inventory()`` introspection that backs the ``/api/inventory`` endpoint, so the
+docs and the runtime can never disagree. Regenerate after changing tools, the bash
+allowlist, or the AWS permission probes:
+
+    cd apps/backend && uv run python scripts/gen_tool_inventory.py
+"""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+from opendevops_core.agent.inventory import build_inventory
+
+_OUT = Path(__file__).resolve().parents[2] / "documentation" / "tool_inventory.md"
+
+
+def _param_cell(p: dict) -> str:
+    req = "required" if p["required"] else f"`{p['default']!r}`"
+    type_str = p["type"].replace("|", "\\|")  # don't break the markdown table
+    return f"| `{p['name']}` | `{type_str}` | {req} |"
+
+
+def _render(inv: dict) -> str:
+    out: list[str] = []
+    out.append("# Tool & Permission Inventory")
+    out.append("")
+    out.append(
+        "> **Generated file — do not edit by hand.** Produced by "
+        "`apps/backend/scripts/gen_tool_inventory.py` from the live code "
+        "(`opendevops_core.agent.inventory.build_inventory`), the same source that backs "
+        "the read-only `GET /api/inventory` endpoint. Regenerate with "
+        "`cd apps/backend && uv run python scripts/gen_tool_inventory.py`."
+    )
+    out.append("")
+    out.append(
+        "This is the trust artifact: exactly what the agent can inspect — every registered "
+        "tool and its parameters, the read-only bash command allowlist, the AWS "
+        "read-permission probe, and the per-cloud capability tiers. Everything is read-only."
+    )
+    out.append("")
+
+    out.append("## Capability by cloud")
+    out.append("")
+    out.append(
+        "| Cloud | Structured SDK tools | CLI access (`bash` tool) | Event-driven + polling |"
+    )
+    out.append("|---|---|---|---|")
+    for p in inv["providers"]:
+        active = " (active)" if p["active"] else ""
+        out.append(
+            f"| **{p['name'].upper()}**{active} | {p['structured_tools']} | "
+            f"{'yes' if p['cli_access'] else 'no'} | "
+            f"{'yes' if p['event_driven_and_polling'] else 'no'} |"
+        )
+    out.append("")
+    out.append(
+        f"Active provider: **{inv['active_provider']}** · "
+        f"total registered tools: **{inv['tool_count']}**."
+    )
+    out.append("")
+
+    out.append("## Registered tools")
+    out.append("")
+    # Group by module for readability, preserving registration order within a group.
+    seen: list[str] = []
+    by_mod: dict[str, list[dict]] = {}
+    for t in inv["tools"]:
+        mod = t["module"].split(".")[-1]
+        if mod not in by_mod:
+            by_mod[mod] = []
+            seen.append(mod)
+        by_mod[mod].append(t)
+    for mod in seen:
+        out.append(f"### `{mod}`")
+        out.append("")
+        for t in by_mod[mod]:
+            out.append(f"#### `{t['name']}` → `{t['returns']}`")
+            out.append("")
+            if t["description"]:
+                out.append(t["description"])
+                out.append("")
+            if t["parameters"]:
+                out.append("| Param | Type | Default |")
+                out.append("|---|---|---|")
+                for p in t["parameters"]:
+                    out.append(_param_cell(p))
+            else:
+                out.append("*No parameters.*")
+            out.append("")
+
+    out.append("## Bash command allowlist")
+    out.append("")
+    bash = inv["bash_allowlist"]
+    out.append(
+        f"`run_bash_command` runs only read-only commands, validated against this allowlist "
+        f"before execution. Shell chaining is **{bash['shell_chaining']}**, `shell=True` is "
+        f"never used, output is capped at {bash['max_output_chars']} chars, and every command "
+        f"has a hard {bash['timeout_seconds']}s timeout."
+    )
+    out.append("")
+
+    def _verbs(items: list[str]) -> str:
+        return ", ".join("`" + v + "`" for v in items) or "none"
+
+    aws = bash["aws"]
+    out.append(
+        f"- **aws** — {aws['note']}. Read-only verbs: {_verbs(aws['readonly_verbs'])}. "
+        f"Blocked global flags: {_verbs(aws['blocked_global_flags'])}."
+    )
+    out.append(
+        f"- **az** — {bash['az']['note']}. Read-only verbs: {_verbs(bash['az']['readonly_verbs'])}."
+    )
+    out.append(f"- **kubectl** — subcommands: {_verbs(bash['kubectl']['subcommands'])}.")
+    out.append(f"- **docker** — subcommands: {_verbs(bash['docker']['subcommands'])}.")
+    out.append("")
+
+    out.append("## AWS read-permission matrix")
+    out.append("")
+    out.append(
+        "One lightweight read call per service verifies the agent's credentials "
+        "(surfaced by the in-app permission checker)."
+    )
+    out.append("")
+    out.append("| Service | boto3 client | Read operation |")
+    out.append("|---|---|---|")
+    for r in inv["aws_permission_matrix"]:
+        out.append(f"| {r['service']} | `{r['boto3_service']}` | `{r['operation']}` |")
+    out.append("")
+    return "\n".join(out)
+
+
+def main() -> None:
+    inv = build_inventory()
+    _OUT.write_text(_render(inv), encoding="utf-8")
+    print(f"Wrote {_OUT} ({inv['tool_count']} tools)")
+
+
+if __name__ == "__main__":
+    main()

apps/backend/src/api/app.py

@@ -23,6 +23,7 @@
     evidence,
     history,
     integrations,
+    inventory,
     monitoring,
     sessions,
     settings,
@@ -195,6 +196,7 @@ async def lifespan(_app: FastAPI):
 app.include_router(integrations.router)
 app.include_router(init_router.router)
 app.include_router(monitoring.router)
+app.include_router(inventory.router)
 
 _DIST = Path(__file__).resolve().parents[3] / "frontend" / "dist"
 if not _DIST.is_dir():

apps/backend/src/api/routers/inventory.py

@@ -0,0 +1,19 @@
+"""Tool / permission inventory endpoint — a read-only trust artifact.
+
+Exposes exactly what the agent can do (registered tools + their parameters, the bash
+command allowlist, the AWS read-permission matrix, and per-provider capability tiers),
+introspected live from the code so it never drifts. Read-only; no AWS calls.
+"""
+
+from __future__ import annotations
+
+from fastapi import APIRouter
+from opendevops_core.agent.inventory import build_inventory
+
+router = APIRouter(prefix="/api/inventory", tags=["inventory"])
+
+
+@router.get("")
+async def get_inventory() -> dict:
+    """Return the introspected tool/permission inventory."""
+    return build_inventory()

apps/backend/tests/test_api/test_inventory.py

@@ -0,0 +1,110 @@
+"""Tests for GET /api/inventory — the published tool/permission inventory.
+
+The inventory is a trust artifact: it must reflect the *live* code (``ALL_TOOLS``, the
+bash allowlist, the AWS permission probes) with no hand-maintained duplicate that could
+drift. These tests assert exactly that correspondence.
+"""
+
+from __future__ import annotations
+
+import os
+
+import pytest
+
+os.environ.setdefault("CHECKPOINT_BACKEND", "memory")
+os.environ.setdefault("LLM_MODEL", "openrouter/anthropic/claude-3.5-sonnet")
+os.environ.setdefault("LLM_API_KEY", "test-key")
+
+
+@pytest.mark.asyncio
+async def test_inventory_reflects_all_tools():
+    """Every registered tool appears in the endpoint, by name, with no extras."""
+    from httpx import ASGITransport, AsyncClient
+    from opendevops_core.agent.core import ALL_TOOLS
+
+    from api.app import app
+
+    async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as client:
+        r = await client.get("/api/inventory")
+
+    assert r.status_code == 200
+    data = r.json()
+
+    expected = {t.__name__ for t in ALL_TOOLS}
+    returned = {t["name"] for t in data["tools"]}
+    assert returned == expected
+    assert data["tool_count"] == len(ALL_TOOLS)
+
+
+@pytest.mark.asyncio
+async def test_inventory_tool_parameters_introspected():
+    """Parameters are introspected from signatures, not hand-written."""
+    from httpx import ASGITransport, AsyncClient
+
+    from api.app import app
+
+    async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as client:
+        data = (await client.get("/api/inventory")).json()
+
+    tools = {t["name"]: t for t in data["tools"]}
+    # get_alarm_history(alarm_name: str, hours: int = 24)
+    hist = tools["get_alarm_history"]
+    params = {p["name"]: p for p in hist["parameters"]}
+    assert params["alarm_name"]["required"] is True
+    assert params["hours"]["required"] is False
+    assert params["hours"]["default"] == 24
+
+
+@pytest.mark.asyncio
+async def test_inventory_bash_allowlist_matches_source():
+    """The published allowlist is sourced from the bash-tool constants."""
+    from httpx import ASGITransport, AsyncClient
+    from opendevops_core.tools import bash_tool as bt
+
+    from api.app import app
+
+    async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as client:
+        data = (await client.get("/api/inventory")).json()
+
+    allow = data["bash_allowlist"]
+    assert set(allow["aws"]["readonly_verbs"]) == set(bt._AWS_READONLY_VERBS)
+    assert set(allow["az"]["readonly_verbs"]) == set(bt._AZ_READONLY_VERBS)
+    assert set(allow["kubectl"]["subcommands"]) == set(bt._KUBECTL_SUBCOMMANDS)
+    assert set(allow["docker"]["subcommands"]) == set(bt._DOCKER_SUBCOMMANDS)
+
+
+@pytest.mark.asyncio
+async def test_inventory_permission_matrix_matches_probes():
+    """The permission matrix is sourced from PERMISSION_PROBES."""
+    from httpx import ASGITransport, AsyncClient
+    from opendevops_core.providers.aws.permissions import PERMISSION_PROBES
+
+    from api.app import app
+
+    async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as client:
+        data = (await client.get("/api/inventory")).json()
+
+    matrix = data["aws_permission_matrix"]
+    returned = {(r["service"], r["boto3_service"], r["operation"]) for r in matrix}
+    expected = {(label, svc, op) for label, svc, op, _ in PERMISSION_PROBES}
+    assert returned == expected
+
+
+@pytest.mark.asyncio
+async def test_inventory_provider_tiers_honest():
+    """Capability tiers match the code: AWS has structured tools, Azure/GCP have none."""
+    from httpx import ASGITransport, AsyncClient
+
+    from api.app import app
+
+    async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as client:
+        data = (await client.get("/api/inventory")).json()
+
+    providers = {p["name"]: p for p in data["providers"]}
+    assert providers["aws"]["structured_tools"] > 0
+    assert providers["azure"]["structured_tools"] == 0
+    assert providers["gcp"]["structured_tools"] == 0
+    # Azure is CLI-first; GCP is a pure stub with no CLI path.
+    assert providers["azure"]["cli_access"] is True
+    assert providers["azure"]["event_driven_and_polling"] is False
+    assert providers["gcp"]["cli_access"] is False