docs: Phase 7 slice 2 — mkdocs-material docs site

Full docs site with 10 pages covering the framework from first-touch
to adapter contribution. Builds clean with `mkdocs build --strict`.

- `mkdocs.yml`: Material theme with indigo palette, instant navigation,
  code-copy, light/dark toggle, pymdownx extensions (superfences,
  tabbed, tasklist, highlight with anchor line numbers).
- `docs/index.md`: ASCII pipeline diagram + tabbed offline/semi-online
  modes + link matrix to the rest of the site.
- `docs/getting-started.md`: Install, API key setup, Example 02
  walkthrough, explicit Windows+non-ASCII-path caveat for the
  `PYTHONPATH` + `PYTHONIOENCODING` workaround we hit in session 12.
- `docs/architecture.md`: The 4-axis table, 3-stage pipeline deep
  dive, data model, signal layer, async vs sync per plugin.
- `docs/harnesses/{index,claude-code,langchain,custom}.md`:
  `custom.md` has a ~30-line skeleton a new harness can start from.
- `docs/providers.md`: Every shipped provider + cost tracking +
  picking guidance table. Documents the `_inline_refs` and
  `_unstringify_json_fields` gotchas we had to solve for Gemini
  and Haiku respectively.
- `docs/evidence.md`: Extract-vs-classify rule (plan §13.8), the
  known `next_user_turns=[]` gap for LangSmith.
- `docs/rubrics.md`: No-regex rule, bilingual guidance.
- `docs/{roadmap,contributing}.md`: Summaries linking the canonical
  files at repo root.
- `[docs]` pyproject extra: mkdocs + mkdocs-material.
- `.github/workflows/docs.yml`: Build on PR, deploy to gh-pages on
  push to main. Needs the "GitHub Pages source = GitHub Actions"
  toggle in repo settings to go live.
- `/site/` gitignored.

No code change. 319 tests still pass. plan.md §0.5 snapshot + Phase 7
row updated.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: Hert4

.github/workflows/docs.yml

@@ -0,0 +1,60 @@
+name: docs
+
+on:
+  push:
+    branches: [main]
+    paths:
+      - "docs/**"
+      - "mkdocs.yml"
+      - ".github/workflows/docs.yml"
+  pull_request:
+    branches: [main]
+    paths:
+      - "docs/**"
+      - "mkdocs.yml"
+  workflow_dispatch:
+
+permissions:
+  contents: read
+  pages: write
+  id-token: write
+
+concurrency:
+  group: docs-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+          cache: pip
+
+      - name: Install docs extras
+        run: pip install -e ".[docs]"
+
+      - name: Build
+        run: mkdocs build --strict
+
+      - name: Upload pages artifact
+        if: github.event_name == 'push' && github.ref == 'refs/heads/main'
+        uses: actions/upload-pages-artifact@v3
+        with:
+          path: site/
+
+  deploy:
+    if: github.event_name == 'push' && github.ref == 'refs/heads/main'
+    needs: build
+    runs-on: ubuntu-latest
+    environment:
+      name: github-pages
+      url: ${{ steps.deploy.outputs.page_url }}
+    steps:
+      - id: deploy
+        uses: actions/deploy-pages@v4

.gitignore

@@ -43,6 +43,9 @@ Thumbs.db
 benchmarks/paper_replication/runs/
 benchmarks/paper_replication/workspaces/
 
+# mkdocs build output
+/site/
+
 # Local env / secrets
 .env
 .env.*

docs/architecture.md

@@ -0,0 +1,130 @@
+# Architecture
+
+The framework is built around one principle: **core never imports an adapter or a provider SDK**. Everything flows through four `Protocol` classes defined in [`trace2skill/core/protocols.py`](https://github.com/Hert4/trace2skill/blob/main/trace2skill/core/protocols.py). Users mix and match.
+
+## The 4 plugin axes
+
+```
+┌──────────────────────────────────────────────────────────────┐
+│                  trace2skill CORE                            │
+│         (harness-agnostic, zero domain logic)                │
+│                                                              │
+│   Stage 1 Rollout → Stage 2 Analyze → Stage 3 Merge          │
+│                    + Signal Layer                            │
+└───┬──────────────┬──────────────┬──────────────┬─────────────┘
+    │              │              │              │
+┌───▼────┐    ┌────▼────┐    ┌────▼────┐    ┌────▼──────┐
+│Harness │    │   LLM   │    │  Skill  │    │ Evidence  │
+│Adapter │    │Provider │    │ Format  │    │  Adapter  │
+└────────┘    └─────────┘    └─────────┘    └───────────┘
+```
+
+| Axis | What it does | Shipped reference implementations |
+|---|---|---|
+| `HarnessAdapter` | `async run_task(query, skill_dir, workspace, turn_budget) -> Trajectory`. Runs one task on an agent. | [Claude Code](harnesses/claude-code.md) (subprocess), [LangChain](harnesses/langchain.md) (AgentExecutor), SimpleReAct (in-process) |
+| `LLMProvider` | `complete` / `complete_structured` / `react`. Wraps one LLM API for the analyst, merger, and judge. | `AnthropicLLMProvider`, `OpenAICompatibleProvider` (generic — covers OpenAI, Gemini, OpenRouter, DeepSeek, Groq, Together, xAI) |
+| `SkillFormat` | `load(path) -> Skill` / `save(skill, path)`. On-disk skill representation. | `AnthropicSkillFormat` (`SKILL.md + resources/`) |
+| `EvidenceAdapter` | `async collect(session_id) -> Evidence`. Extracts raw signals from a real session. | `ClaudeCodeEvidenceAdapter` (JSONL sessions), `LangChainEvidenceAdapter` (LangSmith runs) |
+
+Two more plugin-adjacent types:
+
+- `Evaluator` — domain-specific scoring for batch mode. One is shipped: `SpreadsheetEvaluator` for the paper-replication benchmark.
+- `Rubric` — YAML-driven judge criteria (signal layer). One is shipped: `rubrics/generic.yaml`.
+
+## The 3-stage pipeline
+
+### Stage 1 — Rollout
+
+N tasks run in parallel through a `HarnessAdapter`. Defaults: `rollout_workers=32`. Each rollout produces a `Trajectory` (query, steps, final answer, artifacts). The `Evaluator` labels `y ∈ {0, 1}`.
+
+Async `asyncio.Semaphore(workers)` gating, JSONL checkpoint written after `gather()`.
+
+### Stage 2 — Analyze
+
+For every trajectory:
+
+- `y = 1` → `SuccessAnalyst` (single-pass, structured output).
+- `y = 0` → `AgenticErrorAnalyst` — ReAct loop with 6 tools:
+    - `inspect_skill_file(path)` — show a skill file with line numbers
+    - `read_ground_truth()` — show the expected answer (includes xlsx cell dump for spreadsheet domain)
+    - `try_patch(ops)` — dry-run a candidate patch, return unified diff or validation error
+    - `diff_vs_gt()` — compare trajectory output to ground truth
+    - `finish_with_patch(ops, rationale)` — emit verified fix
+    - `drop(reason)` — quality gate: cannot verify cause, drop trajectory
+
+Parallel dispatch (`analyst_workers=32` default; paper uses 128). `analyst_modes={"error"}` / `{"success"}` / both reproduces paper's +Error / +Success / +Combined conditions.
+
+!!! note "Why agentic"
+    Paper §4.3 shows single-LLM-call analysts propose surface-level edits ("be more careful"). The agentic loop can actually run candidate patches, diff against ground truth, and propose domain-specific fixes. This is Trace2Skill's main USP vs concurrent approaches.
+
+### Stage 3 — Consolidate
+
+Three deterministic guardrails drop bad patches before any LLM sees them:
+
+1. **File existence** — target file must exist in the frozen skill
+2. **Line-range conflict** — hunks within one patch must not overlap
+3. **Trial apply** — dry-run `Skill.apply_patch` on the base skill, reject if it raises
+
+Surviving patches feed a **hierarchical merge** (batch size 32, max depth 6 by default). Merge prompt instructs the LLM to keep only edits appearing ≥2 times across the pool — prevalence-weighted induction from paper §2.4.
+
+Output: one final `Patch` + the complete provenance chain (`source_traj_ids` for every op). Apply to the seed skill, save, done.
+
+## Data model at a glance
+
+```python
+@dataclass
+class Task:
+    task_id: str
+    query: str
+    inputs: dict[str, Path]
+    ground_truth: GroundTruth           # File | Value | Callable
+    metadata: dict
+
+@dataclass
+class Trajectory:
+    task_id: str
+    query: str
+    steps: list[ReActStep]
+    final_answer: str
+    y: int | None                       # 0 | 1, set by Evaluator
+    artifacts: dict[str, Path]
+    model: str
+    metadata: dict
+
+@dataclass
+class Skill:
+    root_md: str                        # SKILL.md content
+    resources: dict[str, bytes]         # {relative_path: content}
+
+    def freeze(self) -> FrozenSkill: ...
+    def apply_patch(self, patch: Patch) -> Skill: ...
+```
+
+Full definitions: [`trace2skill/core/models.py`](https://github.com/Hert4/trace2skill/blob/main/trace2skill/core/models.py).
+
+## Signal layer (semi-online mode)
+
+```
+EvidenceAdapter.collect(session_id)  ──→  Evidence (raw)
+                                           ↓
+                                       LLMJudge
+                                       + Rubric YAML
+                                           ↓
+                                       Judgment (y, confidence, failure_modes)
+                                           ↓
+                                       Pipeline.evolve_from_trajectories(...)
+```
+
+Rule of thumb (plan §13.8): **adapters extract, judges classify**. Never preprocess sentiment or categorize evidence inside the adapter — the judge + rubric decide what the signals mean.
+
+## Async choices
+
+| Plugin | Sync or async |
+|---|---|
+| `HarnessAdapter.run_task` | async (I/O) |
+| `LLMProvider.*` | async (I/O) |
+| `EvidenceAdapter.collect` | async (I/O) |
+| `SkillFormat.load` / `save` | sync (local files) |
+| `Evaluator.evaluate` | sync (pure computation) |
+
+Pipeline and CLI run on a single asyncio event loop. Use `asyncio.to_thread(...)` when wrapping a sync third-party SDK.

docs/contributing.md

@@ -0,0 +1,47 @@
+# Contributing
+
+The canonical guide is [`CONTRIBUTING.md`](https://github.com/Hert4/trace2skill/blob/main/CONTRIBUTING.md) in the repo root. Key points:
+
+## The three-check gate
+
+Before you push, these must pass:
+
+```bash
+python -m pytest tests/unit -q    # 319 tests (or current count)
+ruff check .
+pyright --strict trace2skill
+```
+
+CI runs the same three checks.
+
+## Adding a new adapter
+
+- Target **<300 LOC** per adapter. If you're going over, it probably belongs in core.
+- Lazy-import third-party SDKs inside method bodies; `TYPE_CHECKING` for type hints. Reason: keep core importable when the extra isn't installed.
+- Register in the axis's `__init__.py`, declare the optional dependency in `pyproject.toml`.
+- Test with mocks — no live API calls in `tests/unit/`.
+
+Reference implementations:
+
+| Axis | File |
+|---|---|
+| `HarnessAdapter` | [`trace2skill/harnesses/langchain.py`](https://github.com/Hert4/trace2skill/blob/main/trace2skill/harnesses/langchain.py) |
+| `EvidenceAdapter` | [`trace2skill/evidence_adapters/langchain.py`](https://github.com/Hert4/trace2skill/blob/main/trace2skill/evidence_adapters/langchain.py) |
+| `LLMProvider` | [`trace2skill/llm/openai_compatible.py`](https://github.com/Hert4/trace2skill/blob/main/trace2skill/llm/openai_compatible.py) |
+| `SkillFormat` | [`trace2skill/skill_formats/anthropic.py`](https://github.com/Hert4/trace2skill/blob/main/trace2skill/skill_formats/anthropic.py) |
+
+## The no-regex rule (signal layer)
+
+`trace2skill/signal/` must stay regex-free. All interpretation of evidence goes through the rubric + LLM judge. An ast-grep lint rule to enforce this is on the roadmap.
+
+## Adapters extract, judges classify
+
+An `EvidenceAdapter` pulls raw signals (user's next turn verbatim, tool errors as-is, git status text). It does **not** label success/failure or sentiment. The LLM judge + rubric YAML do that.
+
+## Commit style
+
+Imperative subject line, blank, body explaining *why*. Feature commits prefixed `feat:`, fixes `fix:`, docs `docs:`.
+
+## Good first issues
+
+See the [roadmap](roadmap.md) for the current "Near-term gaps" list.

docs/evidence.md

@@ -0,0 +1,97 @@
+# Evidence adapters (semi-online mode)
+
+Semi-online mode evolves the skill from **real** user sessions instead of benchmark tasks. An `EvidenceAdapter` extracts raw signals from one session; the LLM judge + rubric decides what those signals mean.
+
+## Protocol
+
+```python
+async def collect(self, session_id: str) -> Evidence: ...
+```
+
+Where `Evidence` is:
+
+```python
+@dataclass
+class Evidence:
+    trajectory: Trajectory
+    next_user_turns: list[str]          # what the user said after, verbatim
+    behavioral: dict[str, Any]          # cwd, git status, duration, tokens, ...
+    execution: dict[str, Any]           # tool call counts, error counts
+    explicit_feedback: dict | None      # 👍/👎 if the platform tracks it
+    session_metadata: dict              # session id, timestamps, project name
+```
+
+**Rule:** adapters extract, judges classify. Don't interpret sentiment or label success inside the adapter. The judge + rubric are the only place that decides y=0/1.
+
+## Shipped adapters
+
+### `ClaudeCodeEvidenceAdapter`
+
+Given a Claude Code session UUID, finds the JSONL under `~/.claude/projects/<slug>/`, parses it, and extracts:
+
+- `next_user_turns` — user messages after the first one (Claude Code's local-command + IDE-opened-file wrappers stripped)
+- `behavioral.cwd` + `behavioral.git_status_porcelain` if cwd is a git repo
+- `execution.tool_result_count` + `tool_error_count` + first few error messages
+
+```python
+from trace2skill.evidence_adapters import ClaudeCodeEvidenceAdapter
+
+adapter = ClaudeCodeEvidenceAdapter()
+evidence = await adapter.collect("abc-123-def-...")
+```
+
+`projects_dir` defaults to `~/.claude/projects`; override if you keep sessions elsewhere.
+
+### `LangChainEvidenceAdapter`
+
+Backed by LangSmith (LangChain's tracing service). `session_id` = LangSmith run UUID.
+
+- `trajectory` reconstructed by walking every descendant run of `run_type == "tool"` (agents nest tools inside sub-chains; flat inspection would miss them). Steps ordered by `start_time`.
+- `behavioral` — duration seconds, run_type, project name, token counts if present
+- `execution` — `tool_call_count`, `tool_error_count`, capped error snippets
+- `explicit_feedback` — `client.list_feedback(run_ids=[id])` parsed into a dict keyed by feedback key. Feedback API errors are swallowed (returns `None`) so evidence collection never fails on a flaky side channel.
+
+```python
+from trace2skill.evidence_adapters import LangChainEvidenceAdapter
+from langsmith import Client
+
+adapter = LangChainEvidenceAdapter(
+    project_name="my-agent",
+    client=Client(),               # lazy-builds from LANGSMITH_API_KEY if None
+    include_feedback=True,
+)
+evidence = await adapter.collect("run-uuid-...")
+```
+
+Requires `pip install -e ".[langchain]"` (langsmith comes transitively with langchain).
+
+#### Known gap
+
+`next_user_turns` returns `[]`. LangSmith does not natively track "threads" of related runs — this would need a user-side convention (e.g. `extra.metadata.thread_id` on each run you emit, then a filter in `collect`). Adapter extension welcome.
+
+## How evidence flows into the pipeline
+
+```
+SessionStore (SQLite)
+   session_id → unprocessed
+           ↓
+EvidenceAdapter.collect()
+           ↓
+       Evidence
+           ↓
+       LLMJudge + Rubric YAML
+           ↓
+       Judgment (y, confidence)
+           ↓
+  Filter: keep y=1 trajectories (or y=0 if domain has reliable GT)
+           ↓
+  Trace2SkillPipeline.evolve_from_trajectories(...)  ← skips Stage 1, goes straight to analyze + consolidate
+           ↓
+  AnthropicSkillFormat.save(evolved_skill, target_dir)  ← atomic deploy + timestamped backup
+```
+
+See `examples/03_claude_code_semi_online/` for a complete setup with Claude Code's `SessionEnd` hook.
+
+## Writing your own
+
+Implement one `async def collect` method. Keep it pure extraction. ~100-300 LOC is typical. Reference: `trace2skill/evidence_adapters/langchain.py`.