feat: tool capping, skills system, conversation summarization, config centralization

.env.example

@@ -32,6 +32,10 @@ AWS_PROFILE=devops-agent-readonly   # optional
 MAX_TOOL_CALLS=20
 INVESTIGATION_TIMEOUT=120
 LOG_LEVEL=INFO
+# TOOL_RESPONSE_MAX_CHARS=40000  # cap large tool responses before they reach the LLM; 0 = disabled
+# SUMMARIZATION_ENABLED=true
+# SUMMARIZATION_THRESHOLD_CHARS=60000  # compact session when total message chars exceed this (~15K tokens)
+# SUMMARIZATION_KEEP_CHARS=20000       # preserve this many recent chars intact (~5K tokens)
 
 # Storage backend — uncomment exactly ONE block below:
 #

README.md

@@ -186,6 +186,7 @@ docs/
 | `AWS_PROFILE` | none | AWS named profile (e.g. `devops-agent-readonly`) |
 | `MAX_TOOL_CALLS` | `20` | Hard cap on tool calls per investigation |
 | `INVESTIGATION_TIMEOUT` | `120` | Timeout in seconds |
+| `TOOL_RESPONSE_MAX_CHARS` | `40000` | Truncate tool responses larger than this before feeding to the LLM; `0` disables |
 | `SLACK_WEBHOOK_URL` | none | Slack incoming webhook URL; leave unset to disable notifications |
 | `POLL_INTERVAL_MINUTES` | `0` | Proactive polling interval in minutes; `0` disables the poller |
 | `POLL_ERROR_THRESHOLD` | `5.0` | Lambda error rate % that triggers an automatic investigation |
@@ -206,9 +207,12 @@ docs/
 - [x] **Multi-provider LLM support** — 100+ providers via LiteLLM; swap models with a single `LLM_MODEL` env var change; supports OpenRouter, Anthropic, OpenAI, Groq, Ollama, and any OpenAI-compatible endpoint; see [docs/llm_providers.md](docs/llm_providers.md)
 - [x] **MCP integration** — expose the agent as an MCP server (`devops-agent mcp`); `investigate`, `ask`, and `list_sessions` tools available in Claude Desktop, Cursor, or any MCP-compatible client; stdio and HTTP+SSE transports; see [docs/mcp_server.md](docs/mcp_server.md)
 - [x] **Multi-backend storage** — `memory` (zero config), `sqlite` (local file, no external service), `postgres` (production); switch with one env var; see [docs/databases.md](docs/databases.md)
+- [x] **Skills system** — on-demand investigation skills loaded from `src/skills/*/SKILL.md`; skill names injected into system prompt at startup, full content loaded only when agent calls `use_skill(name)`; ships with `lambda-throttling` skill; add your own by dropping a `SKILL.md` into `src/skills/<name>/`
 - [ ] **Custom tools via URL** — register external tools by pointing at an OpenAPI/HTTP endpoint; agent discovers and calls them alongside built-in AWS tools
 - [x] **Bash CLI escape hatch (Phase 1)** — `run_bash_command` is implemented for read-only AWS CLI, kubectl, and docker commands with strict allowlist validation and timeout.
 - [ ] **Bash sandbox Phase 2** — run each bash command in an isolated throwaway container (`--network none`, read-only FS, non-root, resource limits).
+- [x] **Tool response capping** — truncates oversized AWS tool responses (CloudWatch logs, CloudTrail events) before they reach the LLM context window; configurable via `TOOL_RESPONSE_MAX_CHARS` (default 40 000 chars ≈ 10 K tokens)
+- [x] **Conversation summarization** — automatically summarize old messages when the session approaches the model's context limit; preserves recent exchanges and injects a structured summary so long investigations never fail mid-session; summarization events tracked in `usage_events.metadata` and surfaced in the dashboard
 - [ ] **Optimize tool loading** — pass only relevant tools per investigation context instead of the full 19-tool set
 - [ ] **Message middleware pipeline** — compaction, summarization, intent detection, context trimmer
 - [ ] **Guardrails** — input/output validation, PII scrubbing, query scope enforcement
@@ -218,6 +222,7 @@ docs/
 
 ### Later
 - [ ] **Observability** — OpenTelemetry traces for agent steps, tool call latency, LLM token usage
+- [ ] **Follow-up question suggestions** — after each investigation completes, generate 3 suggested follow-up questions in the background and surface them in the UI as clickable chips
 - [ ] **Session / user feedback loop** — thumbs up/down on investigations, feed signals back to the agent and to an internal quality dashboard
 - [ ] **Telegram integration** — bot that accepts `/investigate` commands and streams findings back to a chat or group
 - [ ] **Knowledge base** — attach internal runbooks, post-mortems, and architecture docs so the agent grounds answers in org-specific context

docs/conversation_summarization.md

@@ -0,0 +1,125 @@
+# Conversation Summarization
+
+## What it does
+
+Long investigations — many tool calls, large log dumps — can push a session's message
+history past the LLM's context limit. When that happens the next agent call either
+silently drops old messages or throws an error.
+
+Conversation summarization automatically compacts the oldest portion of a session's
+message history before each new LLM call. Old messages are replaced with a single
+structured summary, preserving recent context intact.
+
+## How it works
+
+`src/agent/summarizer.py` runs `maybe_summarize()` at the start of every chat turn,
+before the LangGraph agent call:
+
+```
+1. Read current session messages via agent.aget_state()
+2. Count total characters
+3. If total > SUMMARIZATION_THRESHOLD_CHARS:
+   a. Walk from the end, accumulate chars until SUMMARIZATION_KEEP_CHARS is reached
+   b. Split only at a HumanMessage boundary (never mid tool-call pair)
+   c. Call the LLM with a summarization prompt over the old messages
+   d. Replace old messages with one HumanMessage containing the structured summary
+   e. Record a usage_event with metadata.summarization = true
+4. Proceed with the normal agent call
+```
+
+The structured summary follows this template:
+
+```
+**Investigating:** <original incident or question>
+**Key findings:**
+- <finding with specifics — numbers, service names, timestamps>
+**Ruled out:** <list or "nothing ruled out yet">
+**Current state:** <where the investigation is now>
+```
+
+The system prompt is never touched — LangGraph manages it separately.
+
+## Configuration
+
+Set in `.env`:
+
+```env
+# Enable/disable summarization entirely (default: true)
+SUMMARIZATION_ENABLED=true
+
+# Total session chars that trigger a summarization pass (~15K tokens at 4 chars/token)
+SUMMARIZATION_THRESHOLD_CHARS=60000
+
+# How many recent chars to keep intact after compaction (~5K tokens)
+SUMMARIZATION_KEEP_CHARS=20000
+```
+
+| Variable | Default | Notes |
+|---|---|---|
+| `SUMMARIZATION_ENABLED` | `true` | Set to `false` to disable completely |
+| `SUMMARIZATION_THRESHOLD_CHARS` | `60000` | ~15K tokens; increase for larger context models |
+| `SUMMARIZATION_KEEP_CHARS` | `20000` | ~5K tokens of recent context preserved |
+
+## Dashboard tracking
+
+Each summarization run writes a `usage_events` record with:
+
+```json
+{
+  "summarization": true,
+  "messages_removed": 14,
+  "chars_removed": 42000
+}
+```
+
+The `/stats` endpoint (dashboard) exposes two aggregate fields in `summary`:
+
+| Field | Meaning |
+|---|---|
+| `total_summarizations` | How many times summarization has run across all sessions |
+| `total_chars_compacted` | Total characters removed (proxy for tokens saved) |
+
+These power future dashboard widgets ("N sessions compacted, ~X tokens saved").
+
+## Testing it
+
+**Unit test the split logic directly:**
+
+```bash
+uv run pytest tests/test_agent/ -v -k "summar"
+```
+
+**Trigger it in a real session:**
+
+Temporarily lower the threshold in `.env`:
+
+```env
+SUMMARIZATION_ENABLED=true
+SUMMARIZATION_THRESHOLD_CHARS=500   # very low — triggers after a couple messages
+SUMMARIZATION_KEEP_CHARS=200
+```
+
+Then send a few messages in the chat UI. Watch the logs for:
+
+```
+[abc12345] summarizer: session=1204 chars, removing 8 msgs (820 chars), keeping 4 msgs
+[abc12345] summarizer done: -820 chars, summary 312 chars, 1843ms
+```
+
+After that the LangGraph checkpoint for the session will contain the summary
+message instead of the old raw exchanges.
+
+**Check dashboard stats (SQLite or Postgres):**
+
+```bash
+curl http://localhost:8000/stats | python -m json.tool | grep -A2 summarization
+```
+
+## Edge cases
+
+| Scenario | Behaviour |
+|---|---|
+| Session has < 2 human turns | Summarization skipped — nothing meaningful to compact |
+| LLM summarization call fails | Warning logged, session continues without compaction |
+| State update fails | Error logged, session continues unchanged |
+| `SUMMARIZATION_ENABLED=false` | `maybe_summarize()` returns immediately without any DB or LLM call |

docs/skills.md

@@ -0,0 +1,103 @@
+# Skills
+
+Skills are on-demand investigation guides the agent can load during a session.
+Each skill is a markdown file (`SKILL.md`) that contains step-by-step investigation
+steps, key metrics to check, log patterns to look for, and a common root-cause table
+for a specific incident type.
+
+---
+
+## How it works
+
+1. At startup, the skill registry scans `src/skills/*/SKILL.md` and reads the
+   frontmatter (name + description) from each file.
+2. Available skill names and descriptions are injected into the agent's system prompt —
+   this costs only a few tokens regardless of how many skills exist.
+3. During an investigation, the agent calls `use_skill("lambda-throttling")` (or
+   whichever name matches the incident). Only then is the full skill content loaded
+   into the context window.
+4. Unrelated skills cost **zero tokens** — their content is never loaded.
+
+---
+
+## Built-in skills
+
+| Name | Description |
+|---|---|
+| `lambda-throttling` | Investigate Lambda function throttling — concurrency limits, burst limits, reserved concurrency misconfiguration |
+
+---
+
+## Adding a skill
+
+1. Create a directory under `src/skills/` named after the incident type:
+   ```
+   src/skills/ecs-oom/
+   ```
+
+2. Create `SKILL.md` inside it with the required frontmatter:
+   ```markdown
+   ---
+   name: ecs-oom
+   description: Investigate ECS task out-of-memory crashes and container restarts
+   ---
+
+   ## ECS OOM Investigation
+
+   ### Step 1 — Confirm OOM kills
+   ...
+   ```
+
+3. Restart the server (or the CLI). The new skill is auto-discovered and injected
+   into the system prompt — no code changes needed.
+
+---
+
+## Skill file format
+
+```
+src/skills/
+└── <incident-type>/
+    └── SKILL.md        ← required
+```
+
+**`SKILL.md` structure:**
+
+```markdown
+---
+name: <slug used in use_skill() calls>
+description: <one-line description shown in system prompt>
+---
+
+## Investigation steps go here
+...
+```
+
+- `name` — must be unique; this is what the agent passes to `use_skill()`
+- `description` — shown in the system prompt; keep it under 120 chars so it doesn't bloat the prompt
+- Body — plain markdown; use headers, bullet lists, and tables freely
+
+---
+
+## Agent tools
+
+Two tools are available to the agent:
+
+| Tool | Description |
+|---|---|
+| `list_skills()` | Returns all skill names and descriptions |
+| `use_skill(name)` | Loads the full skill content for the given name |
+
+The agent typically calls `use_skill()` directly (names are already in the system
+prompt) and only falls back to `list_skills()` if it needs to confirm what is available.
+
+---
+
+## Ideas for future skills
+
+- `ecs-oom` — ECS container out-of-memory crashes
+- `rds-high-connections` — RDS max connection limit exceeded
+- `api-gateway-5xx` — API Gateway 5XX spike investigation
+- `ec2-cpu-spike` — EC2 CPU utilisation runaway
+- `lambda-cold-start` — Lambda cold start latency spikes
+- `ecs-deployment-failure` — ECS service failing to stabilise after deploy

docs/tool_capping.md

@@ -0,0 +1,80 @@
+# Tool Response Capping
+
+## What it does
+
+AWS tool responses (CloudWatch Logs, CloudTrail events, ECS task lists, etc.) can return
+payloads that are hundreds of kilobytes. Feeding them untruncated into the LLM wastes
+tokens and can exhaust the context window mid-investigation.
+
+Tool response capping automatically truncates any tool response that exceeds a
+configurable character limit, appending a notice that explains what happened and
+suggests how to narrow the query.
+
+## How it works
+
+Every tool registered in `src/agent/core.py` is wrapped with `with_cap()` at startup:
+
+```python
+# src/agent/core.py
+tools = [with_cap(t) for t in ALL_TOOLS] if settings.tool_response_max_chars > 0 else ALL_TOOLS
+```
+
+The cap logic lives in `src/tools/_cap.py`:
+
+- Serialises the result to JSON and measures its length.
+- If within the limit → returns the result unchanged.
+- If over the limit → returns a replacement dict:
+
+```json
+{
+  "_capped": true,
+  "_original_chars": 184320,
+  "_notice": "Response truncated from 184,320 to 40,000 chars to protect the context window. Use more specific filters, shorter time ranges, or pagination to retrieve focused data.",
+  "_data": "...first 40,000 chars of the serialised result..."
+}
+```
+
+The agent sees the truncation notice and knows to refine its query.
+
+## Configuration
+
+Set in `.env` (or leave unset to use the default):
+
+```env
+# 0 = disabled (no capping); omit to use the default of 40,000 chars
+TOOL_RESPONSE_MAX_CHARS=40000
+```
+
+| Value | Behaviour |
+|---|---|
+| `0` | Capping disabled — all responses pass through unchanged |
+| `> 0` | Responses larger than this many characters are truncated |
+
+## Testing it
+
+Run the dedicated unit tests:
+
+```bash
+uv run pytest tests/test_tools/test_cap.py -v
+```
+
+To trigger capping manually:
+
+```python
+from tools._cap import cap_tool_result
+
+big = {"events": ["x"] * 10_000}
+result = cap_tool_result(big, max_chars=100)
+print(result["_capped"])      # True
+print(result["_notice"])
+```
+
+Or with a real tool response — set `TOOL_RESPONSE_MAX_CHARS=1` in `.env` and fire
+any CloudWatch query; the agent's reply will reference the truncation notice.
+
+## When capping does NOT fire
+
+- Non-dict return values are passed through unchanged (e.g., plain strings).
+- If `TOOL_RESPONSE_MAX_CHARS=0` the wrapper is never applied.
+- The cap applies only to what the LLM sees; the full raw response is **not** logged
+  or persisted — the agent stores only what it actually received.

frontend/src/pages/DashboardPage.tsx

@@ -1,13 +1,14 @@
 import { useEffect, useState } from 'react';
 import { Link } from 'react-router-dom';
-import { MessageSquare, Wrench, DollarSign, Clock, AlertTriangle, Zap } from 'lucide-react';
+import { MessageSquare, Wrench, DollarSign, Clock, AlertTriangle, Zap, Scissors } from 'lucide-react';
 import { fmtCost, fmtTok, relativeTime, cn } from '../lib/utils';
 
 interface Summary {
   total_sessions: number; total_queries: number;
   total_tool_calls: number; total_tool_errors: number;
   total_input_tokens: number; total_output_tokens: number;
   total_cost_usd: number; avg_latency_ms: number;
+  total_summarizations: number; total_chars_compacted: number;
 }
 interface ActivityDay   { date: string; sessions: number; }
 interface TopTool        { tool: string; count: number; errors: number; }
@@ -144,6 +145,24 @@ export default function DashboardPage() {
           />
         </div>
 
+        {/* Context management row */}
+        {(summary.total_summarizations > 0 || summary.total_chars_compacted > 0) && (
+          <div className="grid grid-cols-2 gap-3">
+            <StatCard
+              label="Sessions compacted"
+              value={summary.total_summarizations}
+              sub="conversation summarizations run"
+              icon={<Scissors size={15} />}
+            />
+            <StatCard
+              label="Context saved"
+              value={summary.total_chars_compacted > 0 ? `~${fmtTok(Math.round(summary.total_chars_compacted / 4))}` : '—'}
+              sub={`${summary.total_chars_compacted.toLocaleString()} chars compacted`}
+              icon={<Zap size={15} />}
+            />
+          </div>
+        )}
+
         {/* Activity chart */}
         <div className="bg-white dark:bg-[#18181C] border border-gray-200 dark:border-[#27272F] rounded-lg p-4 shadow-[0_1px_2px_rgba(0,0,0,0.04)]">
           <h2 className="text-[11px] font-semibold text-gray-400 dark:text-[#64748B] uppercase tracking-[0.07em] mb-4">

migrations/003_usage_events_metadata.sql

@@ -0,0 +1,4 @@
+-- Add metadata column to usage_events for tracking per-event context
+-- (e.g. summarization flag, chars_removed, etc.)
+ALTER TABLE usage_events
+    ADD COLUMN IF NOT EXISTS metadata jsonb NOT NULL DEFAULT '{}'::jsonb;