diff --git a/.claude/skills/cron-wrapup/SKILL.md b/.claude/skills/cron-wrapup/SKILL.md deleted file mode 100644 index 28cd748..0000000 --- a/.claude/skills/cron-wrapup/SKILL.md +++ /dev/null @@ -1,324 +0,0 @@ ---- -name: cron-wrapup -description: Use when running the nightly KB cron wrap-up — aggregating the previous day's usage reports, memory page, wiki-promote / TTL outcomes, and `data/raw/ops/cron/` per-run log exit states into a single durable Slack-digest-stable wiki summary plus run handoff. ---- - -# cron-wrapup - -## DB-Canonical Override - -DB rows are the source of truth. Write the wrap-up summary and run handoff -through the kb-mcp MCP tools (`upsert_page`, `create_handoff`), and write the -operation log through the kb-mcp `create_operation_log` tool. Markdown under -`data/` is generated export. If any older instruction below says to lint as a -write gate or commit `data/`, ignore that instruction and use -the kb-mcp tools instead. - -## Overview - -One skill, one job: at 05:00 KST, after the night's pipeline has finished, produce a deterministic operational snapshot of what the KB cron jobs did and what the user needs to notice. The output is the single source of truth for the 09:00 Slack digest — Slack must never read memory pages directly. - -Two artefacts per run: - -1. **Wiki summary** — `data/wiki/summaries/{YYYY}/{MM}/{TARGET}-cron-wrapup.md` (`type: summary`, `subtype: cron-daily`). Body uses fixed H2 section names (`Status`, `Jobs`, `Insights`, `Action Items`, `Anomalies`, `Counters`, `Links`) so downstream parsers stay stable as prose evolves. -2. **Run handoff** — `data/handoffs/{YYYY}/{MM}/cron-wrapup/{TARGET}_{role}_handoff_NN.md` via the `handoff-document` skill. - -This skill is the **operational command layer**. Content synthesis (what happened in the world) lives in `memory-report`'s daily memory page — do not duplicate it here. Wrap-up describes what happened in the KB *pipeline* and extracts the small set of user-facing insights and follow-up actions created by that pipeline. - -**Bundled reference (self-contained — do not consult docs/ at runtime):** - -- `reference/templates/cron-wrapup-summary.md` — wiki summary skeleton with seven canonical H2 sections pre-filled with placeholder rows. Copy this verbatim and fill cells. - -For handoff authoring, import `handoff-document`. This skill does not duplicate the handoff schema. - -If anything below contradicts the lint code (`src/kb/lint/wiki.py`), the lint code wins — file an issue and update this skill. - -## When to Use - -- Daily 05:00 KST cron fires via `scripts/cron/kb-cron-wrapup.sh`. -- Manual invocation: "Run the KB cron wrap-up for {YYYY-MM-DD}". - -**Do NOT use** for: - -- Content synthesis — use `memory-report` (daily memory page). -- Wiki promotion — use `wiki-approval` (separate cron). -- Raw ingestion — manual or external scripts only; never the wrap-up. -- Re-validating the whole KB history or using lint as analysis input. The wrap-up runs the required lint gate only after writing its own summary/handoff. - -## Target - -`TARGET = YYYY-MM-DD` (yesterday in KST). The cron wrapper supplies this: - -```bash -TARGET_DATE="$(TZ=Asia/Seoul date -d 'yesterday' +%F)" -``` - -If the user manually invokes "Run the KB cron wrap-up for 2026-05-20", `TARGET = 2026-05-20`. Never default to "today" — the night's pipeline writes for yesterday. - -## Repo Layout & read model - -Postgres is the source of truth; `data/` is a human-readable export. Read wiki, handoff, metrics, and cron-run inputs from the DB with `psql` (schema + recipes in `docs/db_informations/state-db-schema-reference.md`) — never open `data/**/*.md` during normal operation. Per-run cron evidence lives in `cron_runs.log_body`; archived files under `data/raw/ops/cron/` are export/debug copies only, not the source of truth. - -```bash -TARGET="$1"; psql_kb() { psql "${DATABASE_URL/+psycopg/}" -tAc "$1"; } -# Usage + memory summary bodies for the target (summary slugs are date-prefixed) -psql_kb "SELECT slug, body_md FROM pages WHERE type='summary' AND slug LIKE '$TARGET-%';" -# Canonical usage numbers when present (see docs/db_informations/state-db-schema-reference.md) -psql_kb "SELECT report_type, session_count, token_total, cost_usd, tool_error_count FROM metrics WHERE report_date='$TARGET';" -# Run handoffs from the night's pipeline -psql_kb "SELECT task_slug, handoff_id, status FROM handoffs WHERE status='ready';" -# Per-job outcomes when present (status/exit_code/timestamps) -psql_kb "SELECT job_name, status, exit_code, started_at, finished_at FROM cron_runs WHERE target='$TARGET';" -``` - -The export tree below shows where the same data also lands as human-readable markdown: - -``` -data/ -├── wiki/summaries/{YYYY}/{MM}/ # generated export of the summary pages (read via psql, above) -│ ├── {TARGET}-opencode-usage.md # DB-backed (kb-opencode-daily-report writes via kb-mcp / service layer) -│ ├── {TARGET}-claude-code-usage.md # DB-backed -│ ├── {TARGET}-hermes-usage.md # DB-backed -│ └── {TARGET}-memory.md # DB-backed (kb-memory-daily writes via kb-mcp / service layer) -├── handoffs/{YYYY}/{MM}/ -│ ├── wiki-daily-build/{TARGET}_*_handoff_*.md # run handoff for memory-daily -│ └── wiki-promote/... # if wiki-promote ran (folder per wiki-approval) -├── raw/ops/cron/{YYYY}/{MM}/ # per-run cron log files (one file per job per target) -│ ├── {TARGET}_kb-opencode-daily-report.log -│ ├── {TARGET}_kb-claude-code-daily-report.log -│ ├── {TARGET}_kb-hermes-daily-report.log -│ ├── {TARGET}_kb-memory-daily.log -│ ├── {TARGET}_kb-wiki-ttl-sweep.log -│ ├── {TARGET}_kb-wiki-promote.log -│ ├── {TARGET}_kb-memory-weekly.log # exists on Monday (after weekly run for prior ISO week) -│ └── {TARGET}_kb-memory-monthly.log # exists on day 1 (after monthly run for prior month) -│ └── {TARGET}_kb-cron-wrapup.log # committed by the shell wrapper AFTER session exits -└── log.md -``` - -> **Note:** `{TARGET}_kb-cron-wrapup.log` does not exist inside `data/` while the session is running. -> The shell wrapper records the cron run via the `kb-submit-cron-run` CLI after the session exits (not your job). -> Do not read or stage it during the session. - -`sources:` frontmatter paths are **relative to `data/`** (e.g. `wiki/summaries/...`, NOT `data/wiki/summaries/...`). - -Every wrapper uses `TARGET_DATE` (= yesterday in KST) as the filename prefix, even when its workflow target is a week or month — so the daily wrap-up's glob `{TARGET}_*.log` matches every wrapper that ran in last night's pipeline. The actual weekly/monthly period stays in the log body and the produced wiki page. - -Each run writes its own immutable log file, so the wrap-up reads exactly one file per job per target — no time-window grep required. - -## Wiki Summary Schema (the only thing lint ERRORs you for) - -```yaml ---- -type: summary -subtype: cron-daily -date: "YYYY-MM-DD" # the TARGET -created: "YYYY-MM-DD" # the run date (today in KST) -updated: "YYYY-MM-DD" -sources: - - wiki/summaries/YYYY/MM/{TARGET}-memory.md - - wiki/summaries/YYYY/MM/{TARGET}-opencode-usage.md - - wiki/summaries/YYYY/MM/{TARGET}-claude-code-usage.md - - wiki/summaries/YYYY/MM/{TARGET}-hermes-usage.md - - handoffs/YYYY/MM/wiki-daily-build/{TARGET}_*_handoff_*.md - # plus handoffs/YYYY/MM/wiki-promote/... if wiki-promote produced output that day -tags: [cron-wrapup, ops-daily] ---- -``` - -Required: `type, created, updated, sources, tags, date`. **No `review_status`** — summaries are exempt from approval. `subtype: cron-daily` is currently free-form (lint does not enforce subtype values) but use it consistently. - -## Body Contract (Slack-digest-stable — DO NOT rename sections) - -The seven H2 sections below are the parsing contract for the 09:00 Slack digest. Section names, order, and column shapes must remain stable. Authors may add prose under each section but must not rename or reorder them. - -```markdown -# Cron Wrap-up — {TARGET} - -## Status - -OK | DEGRADED | FAILED - -(One-word verdict on the first line, optional one-sentence elaboration on the second.) - -## Jobs - -| Job | Schedule | Exit | Output | Notes | -| --- | --- | --- | --- | --- | -| kb-opencode-daily-report | 10 3 * * * | 0 | wiki/summaries/2026/05/2026-05-20-opencode-usage.md | 1 session, 1.05 USD | -| kb-claude-code-daily-report| 20 3 * * * | 0 | wiki/summaries/2026/05/2026-05-20-claude-code-usage.md| 10/14 sessions, 69.77 USD | -| kb-hermes-daily-report | 15 3 * * * | 0 | wiki/summaries/2026/05/2026-05-20-hermes-usage.md | 3 sessions, zombie 0 | -| kb-memory-daily | 30 3 * * * | 0 | wiki/summaries/2026/05/2026-05-20-memory.md | 2 new pages | -| kb-wiki-ttl-sweep | 30 0 * * * | 0 | (log only) | 0 pages expired | -| kb-wiki-promote | 0 4 * * * | 0 | handoffs/2026/05/wiki-promote/... | 1 promoted | - -Exactly 5 columns. Missing jobs (e.g. weekly on non-Monday) → omit the row, do not invent. - -## Insights - -- none - -(Extract 1-5 user-facing signals from the daily memory page, usage reports, and run handoffs. Prefer durable or decision-relevant signals: new improvement pages, recurring error trends, cost spikes, completed work that needs review, newly blocked work, or promotion candidates. Use the literal line `- none` only when the KB produced no meaningful new signal.) - -## Action Items - -| Priority | Owner | Item | Source | -| --- | --- | --- | --- | -| high | user | Review frontend-review-console PR #20 CI and merge readiness | wiki/summaries/2026/05/2026-05-20-memory.md | - -(Extract concrete follow-ups from daily memory `Open Items`, `Next Run Notes`, run handoff `Next handoff instructions`, and unresolved improvement pages. Use owner `user`, `agent`, or `unknown`. If empty, use one row with `none | none | none | none`.) - -## Anomalies - -- none - -(Or bullet list of issues: non-zero exits, missing inputs, lint failures, lock skips, late starts > 10 min. Use the literal line `- none` when empty — never delete the section.) - -## Counters - -| Metric | Value | -| --- | --- | -| pages_created | 2 | -| pages_promoted | 1 | -| pages_ttl_swept | 0 | -| insights_count | 5 | -| action_items_count | 3 | -| lint_errors | 0 | -| lint_warnings | 12 | -| total_usd | 70.82 | -| sessions_total | 14 | - -Keep snake_case keys so the digest can match on them. `insights_count` and `action_items_count` make an empty-looking status easy to detect even when jobs all succeeded. - -## Links - -- memory: `wiki/summaries/2026/05/2026-05-20-memory.md` -- opencode: `wiki/summaries/2026/05/2026-05-20-opencode-usage.md` -- claude: `wiki/summaries/2026/05/2026-05-20-claude-code-usage.md` -- hermes: `wiki/summaries/2026/05/2026-05-20-hermes-usage.md` -- daily-handoff: `handoffs/2026/05/wiki-daily-build/2026-05-20_opencode_handoff_04.md` -- promote-handoff: `handoffs/2026/05/wiki-promote/...` (omit line if absent) -``` - -## Status Verdict — deterministic rules - -Compute in this order; the first match wins: - -| Verdict | Condition | -|---|---| -| `FAILED` | Any cron wrapper exited non-zero **OR** any *required* input is missing (memory page, all three usage reports). | -| `DEGRADED` | Any non-blocking anomaly: lint warnings appearing today that weren't there yesterday, lock skip, optional input missing (promote handoff on a day promote was supposed to run), or a per-run log file file-mtime far past its scheduled cron time. | -| `OK` | All expected logs show clean exit; Anomalies list is `- none`. | - -Late-start detection: per-run logs do not contain wrapper-side start timestamps, so use the log file's mtime (or first content line's timestamp if the underlying tool emits one) versus the scheduled cron time. Treat slips > 10 min as DEGRADED only when the signal is clearly available; otherwise omit the anomaly rather than guess. - -## Inputs the wrap-up reads — and what to extract from each - -| Input (read via `psql`; logs are files) | Extract | -|---|---| -| `pages` slug `{T}-opencode-usage` (numbers also in `metrics` report_type=opencode) | sessions, total USD | -| `pages` slug `{T}-claude-code-usage` (numbers also in `metrics`) | sessions, USD, tool_error_rate, hot files count | -| `pages` slug `{T}-hermes-usage` | sessions, zombie count | -| `pages` slug `{T}-memory` | key events, promotion candidates, open items, next-run notes; **do not copy the whole narrative** | -| `handoffs` task_slug=`wiki-daily-build`, status=`ready` | new pages count from §6 Outputs, risks, next handoff instructions | -| `handoffs` task_slug=`wiki-promote`, status=`ready` | promoted count, rejected count, expired count | -| `cron_runs` rows for target `T` | per-job status, exit code, timestamps, `log_body` diagnostics such as `ERROR:`, lock skips, and TTL sweep counts | - -Read each input at most once. Never use lint as a discovery source, never re-render existing summaries. The wrap-up MUST NOT read `data/raw/` or other `data/` exports for discovery; use `cron_runs.log_body` instead. Archived log files are debug copies only if DB submission itself is broken. The wrap-up may extract concise insights/actions from the daily memory and handoffs, but must not duplicate the memory page's full content narrative. - -## DB Write Order - -1. Write the wrap-up summary through the kb-mcp `upsert_page` tool (pass `type: summary`, structured `frontmatter`, `body_md` without the `---` fence, `slug`, and `export_path`). -2. Write the handoff through the kb-mcp `create_handoff` tool (structured `frontmatter` + `body_md`). -3. Write the operation note through the kb-mcp `create_operation_log` tool. -4. Confirm every tool result reports `export.status == success`. If a result carries an `error`/`code` (`missing_args`, `not_found`, `conflict`, `lint_failed`, `export_failed`), fix the cause and retry that write. - -## Common Lint Failures (preemptive fixes) - -| ERROR pattern | Cause | Fix before lint | -|---|---|---| -| `source file not found: wiki/summaries/...` | Cited a path that wasn't actually produced (e.g. promote handoff on day promote didn't run) | Drop the source line or change the path | -| `source file not found: data/wiki/...` | `sources:` paths must be `data/`-relative, not `data/data/` | Drop the `data/` prefix | - -## Handoff (defer to `handoff-document`) - -Every run writes one handoff. Use the `handoff-document` skill for filename grammar, frontmatter, body sections, and lint. - -This skill specifies only: - -- **Task folder**: `data/handoffs/{YYYY}/{MM}/cron-wrapup/` (one folder for the task; per-role NN keeps incrementing across dates, same shape as `wiki-daily-build/`). -- **Subject**: `{TARGET}` (the date being wrapped up). -- **Role**: whoever the cron wrapper invokes (currently `opencode`). -- **Content focus**: link to the produced wiki summary, list anomalies, set `status: ready` and a clear `## 9. Next handoff instructions` block for the next day or for the 09:00 digest. - -## Log Format (write through the kb-mcp `create_operation_log` tool) - -```markdown - -## YYYY-MM-DD (cron wrap-up — target: {TARGET}) - -- **fill**: {TARGET} cron wrap-up summary - - 소스: wiki/summaries/Y/M/{TARGET}-{opencode,claude-code,hermes}-usage.md, wiki/summaries/Y/M/{TARGET}-memory.md, raw/ops/cron/Y/M/{TARGET}_*.log - - 출력: wiki/summaries/Y/M/{TARGET}-cron-wrapup.md (신규) -- **handoff**: handoffs/Y/M/cron-wrapup/{TARGET}_{role}_handoff_NN.md -- **db_write**: summary + handoff + operation log exported successfully -``` - -## Workflow (10 steps) - -``` -1. Resolve TARGET = yesterday in KST (or accept explicit argument) -2. Query inputs via `psql`: usage + memory pages (`pages`), ready handoffs (`handoffs`), usage numbers (`metrics`), job outcomes (`cron_runs`); - then read per-run cron logs at `raw/ops/cron/{Y}/{M}/{TARGET}_*.log` (files). Note any MISSING required input → Status: FAILED later. -3. Extract per-input metrics (see Inputs table). Compute Counters. -4. Read every `raw/ops/cron/{Y}/{M}/{TARGET}_*.log` except `{TARGET}_kb-cron-wrapup.log` — that file does not exist during the session (the shell wrapper writes and commits it after the session exits, so it is never available to read here) - for non-zero exits, ERRORs, lock skips. Collect into Anomalies. -5. Compute Status verdict (FAILED > DEGRADED > OK, first match wins). -6. Fill `reference/templates/cron-wrapup-summary.md` and write it through the kb-mcp `upsert_page` tool (`type: summary`, structured `frontmatter` + `body_md`) with export path `wiki/summaries/{Y}/{M}/{TARGET}-cron-wrapup.md`. -7. Write run handoff via the kb-mcp `create_handoff` tool (subject = TARGET, role = invoker, status: ready). -8. Append the operation note via the kb-mcp `create_operation_log` tool. -9. Confirm every tool result reports `export.status == success` (retry any write that returns an `error`/`code`). -10. STOP. Do NOT commit or push `data/`; it is generated export. -``` - -## Data Persistence Policy - -DB rows are the durable checkpoint. Markdown, handoff files, `log.md`, and cron -logs under `data/` are generated export. Never stage, commit, or push from this workflow. - -## Idempotency - -- If `{TARGET}-cron-wrapup.md` already exists, **overwrite it** (the cron may rerun after a failure). Update `updated:` to today. -- For the handoff, use the next per-role `NN` in `handoffs/{Y}/{M}/cron-wrapup/`. Do not overwrite a prior handoff. - -## Failure path - -If a required input is missing (e.g. memory page never wrote): - -1. Still write the wrap-up. Set `Status: FAILED`. -2. List the missing input as a bullet under `## Anomalies`. -3. In the run handoff, set `status: ready` and put the failure cause in `§8. Risks / uncertainties` with a clear escalation note for §9 so the 09:00 digest surfaces it to Slack. -4. Exit non-zero from the wrapper so the cron log records the failure. - -## Workflow Discipline - -- **One target per run.** Never extend scope implicitly to "and also the day before". -- **Never edit raw files.** The wrap-up reads `data/raw/ops/cron/{Y}/{M}/{TARGET}_*.log` as read-only evidence, but must never modify their contents. Do not read other `data/raw/` subtrees. -- **Replay caveat.** A wrapper rerunning for an already-written TARGET will append to a persisted log entry. If you must replay a job for a previously-run TARGET, use the kb-mcp tools to manage the existing log entry before the wrapper runs. -- **Do not auto-promote** anything — wiki promotion is `wiki-approval`'s job. -- **Do not use lint as analysis input** — run only the required final gate: `kb-lint all` once. -- **Persist only through the kb-mcp tools.** Never commit outer repo files or generated `data/` export. -- **If blocked**, write the handoff with `status: ready` and the wrap-up with `Status: FAILED`, then exit non-zero. - -## Red Flags — STOP and re-check - -- About to read any cron log other than `raw/ops/cron/{Y}/{M}/{TARGET}_*.log` → STOP. The wrap-up summarizes exactly this target's per-run files; no time-window scan. (`{TARGET}_kb-cron-wrapup.log` does not exist during the session — the shell wrapper creates and commits it after the session exits; never read or stage it.) -- About to inspect `git status` or prepare a data commit → STOP. DB rows are the source of truth. -- About to re-render the full memory page into the wrap-up → STOP. Extract only concise insights and action items. -- About to rename or reorder one of the seven H2 sections → STOP. The Slack digest will break. -- About to create a wiki page by editing `data/wiki` directly → STOP. Write through the kb-mcp `upsert_page` tool. -- About to put `data/wiki/...` in `sources:` → drop the `data/` prefix -- About to mark Status: OK while Anomalies list is non-empty → re-read the Status table; non-empty anomaly = DEGRADED at minimum -- About to overwrite a prior cron-wrapup handoff with the same NN → bump NN, never overwrite handoffs -- About to invoke this skill twice in one cron firing → STOP. One run, one wrap-up, one handoff. -- About to run any `git commit` for generated KB data → STOP. Use the kb-mcp tools. diff --git a/.claude/skills/cron-wrapup/reference/templates/cron-wrapup-summary.md b/.claude/skills/cron-wrapup/reference/templates/cron-wrapup-summary.md deleted file mode 100644 index 59550a9..0000000 --- a/.claude/skills/cron-wrapup/reference/templates/cron-wrapup-summary.md +++ /dev/null @@ -1,82 +0,0 @@ ---- -type: summary -subtype: cron-daily -date: "YYYY-MM-DD" -created: "YYYY-MM-DD" -updated: "YYYY-MM-DD" -sources: - - wiki/summaries/YYYY/MM/YYYY-MM-DD-memory.md - - wiki/summaries/YYYY/MM/YYYY-MM-DD-opencode-usage.md - - wiki/summaries/YYYY/MM/YYYY-MM-DD-claude-code-usage.md - - wiki/summaries/YYYY/MM/YYYY-MM-DD-hermes-usage.md - - handoffs/YYYY/MM/wiki-daily-build/YYYY-MM-DD_role_handoff_NN.md -tags: [cron-wrapup, ops-daily] ---- - -# Cron Wrap-up — YYYY-MM-DD - -## Status - -OK - -(Replace with OK | DEGRADED | FAILED on the first line. Optional one-sentence elaboration on the second line.) - -## Jobs - -| Job | Schedule | Exit | Output | Notes | -| --- | --- | --- | --- | --- | -| kb-wiki-ttl-sweep | 30 0 * * * | 0 | (log only) | 0 pages expired | -| kb-opencode-daily-report | 10 3 * * * | 0 | wiki/summaries/YYYY/MM/YYYY-MM-DD-opencode-usage.md | N sessions, X.XX USD | -| kb-hermes-daily-report | 15 3 * * * | 0 | wiki/summaries/YYYY/MM/YYYY-MM-DD-hermes-usage.md | N sessions, zombie N | -| kb-claude-code-daily-report | 20 3 * * * | 0 | wiki/summaries/YYYY/MM/YYYY-MM-DD-claude-code-usage.md| N sessions, X.XX USD | -| kb-memory-daily | 30 3 * * * | 0 | wiki/summaries/YYYY/MM/YYYY-MM-DD-memory.md | N new pages | -| kb-wiki-promote | 0 4 * * * | 0 | handoffs/YYYY/MM/wiki-promote/... | N promoted | - -(Omit a row if a job did not run that day, e.g. weekly/monthly on non-eligible days. Do not invent rows.) - -## Insights - -- none - -(Replace with 1-5 user-facing signals from the daily memory page, usage reports, and run handoffs: new improvement pages, recurring error trends, cost spikes, completed work that needs review, newly blocked work, or promotion candidates. Use `- none` only when no meaningful new signal exists.) - -## Action Items - -| Priority | Owner | Item | Source | -| --- | --- | --- | --- | -| none | none | none | none | - -(Replace with concrete follow-ups from daily memory Open Items / Next Run Notes, run handoff Next handoff instructions, and unresolved improvement pages. Owner should be `user`, `agent`, or `unknown`.) - -## Anomalies - -- none - -(Replace with bullet list of anomalies — non-zero exits, missing inputs, lint failures, lock skips, late starts > 10 min. Use the literal line `- none` when empty.) - -## Counters - -| Metric | Value | -| --- | --- | -| pages_created | 0 | -| pages_promoted | 0 | -| pages_ttl_swept | 0 | -| insights_count | 0 | -| action_items_count | 0 | -| lint_errors | 0 | -| lint_warnings | 0 | -| total_usd | 0.00 | -| sessions_total | 0 | - -(Add new snake_case keys only if a follow-up Slack digest needs them.) - -## Links - -- memory: wiki/summaries/YYYY/MM/YYYY-MM-DD-memory.md -- opencode: wiki/summaries/YYYY/MM/YYYY-MM-DD-opencode-usage.md -- claude: wiki/summaries/YYYY/MM/YYYY-MM-DD-claude-code-usage.md -- hermes: wiki/summaries/YYYY/MM/YYYY-MM-DD-hermes-usage.md -- daily-handoff: handoffs/YYYY/MM/wiki-daily-build/YYYY-MM-DD_role_handoff_NN.md -- promote-handoff: handoffs/YYYY/MM/wiki-promote/... - -(Omit a line if the artefact was not produced that day.) diff --git a/.claude/skills/knowledgebase-initialize/SKILL.md b/.claude/skills/knowledgebase-initialize/SKILL.md index 5daced3..7b85ffd 100644 --- a/.claude/skills/knowledgebase-initialize/SKILL.md +++ b/.claude/skills/knowledgebase-initialize/SKILL.md @@ -1,6 +1,6 @@ --- name: knowledgebase-initialize -description: Use on a fresh clone, new machine, or new profile to create or repair the local data repository — verifying CLI tooling, choosing usage report mode, proposing cron jobs, and writing initialization handoff/log output. +description: Use on a fresh clone, new machine, or new profile to create or repair the local data repository — verifying CLI tooling and writing initialization handoff/log output. --- # KnowledgeBase Initialize @@ -15,10 +15,8 @@ Use this skill as the runtime contract for repository setup. Do not load `docs/` - `data/` is a generated export directory. Postgres (reached via `DATABASE_URL`) is the canonical store. Never add `data/` to the outer repo. - Never modify existing files under `data/raw/`. - Preserve existing `data/` contents; create only missing directories/files. -- Do not install or edit crontab until the user approves the exact entries. -- Memory cron jobs run lint and leave changes uncommitted for manual review. -- Wiki promotion writes through the kb-mcp tools; it does not commit or push from the AI session. -- Prefer relative paths from repo root. Do not hard-code machine-specific absolute paths except when showing final crontab examples. +- Wiki writes go through the kb-mcp tools; the skill does not commit or push from the AI session. +- Prefer relative paths from repo root. Do not hard-code machine-specific absolute paths. ## Required Data Layout @@ -33,7 +31,6 @@ data/ web/ manual/ ops/ - cron/YYYY/MM/ runs/YYYY/MM/ decisions/YYYY/MM/ traces/YYYY/MM/ @@ -47,14 +44,9 @@ data/ improvements/YYYY-MM/ checklists/ summaries/YYYY/MM/ - rejected/ - ops/ - reports/YYYY/MM/ log.md ``` -`data/rejected/` may be empty on a fresh install. It is populated by wiki rejection. - ## Phase 1: Inspect Check: @@ -65,8 +57,6 @@ psql "${DATABASE_URL/+psycopg/}" -tAc "SELECT 1" >/dev/null && echo "DB reachabl test -f data/log.md uv --version uv run kb-lint --help -uv run kb-db-ttl-sweep --help -find scripts/cron -maxdepth 1 -type f -name 'kb-*.sh' | sort ``` If a command fails because dependencies are missing, run: @@ -150,61 +140,7 @@ bash .claude/skills/knowledgebase-initialize/scripts/install-global-skills.sh - Set `CLAUDE_SKILLS_DIR` to override the destination. Edit the `GLOBAL_SKILLS` array in the script to expose more skills (currently: `handoff-document` and `wiki-note`). -## Phase 4: Usage Report Mode - -Use the `usage-report-setup` skill if the user asks for detailed setup. For initialization, ask which source-specific reports to enable: - -| Mode | Cron wrapper | -|---|---| -| none | no usage report job | -| OpenCode | `scripts/cron/kb-opencode-daily-report.sh` | -| Hermes | `scripts/cron/kb-hermes-daily-report.sh` | -| Claude Code | `scripts/cron/kb-claude-code-daily-report.sh` | -| multiple separate | selected wrappers only | - -Default recommendation: enable only sources the user actually runs. Do not create combined usage reports. - -## Phase 5: Propose Cron Jobs - -KnowledgeBase owns the expected job contract and portable wrapper scripts under `scripts/cron/`; it does not require a specific scheduler backend. - -Scheduler backend guidance: - -- **Tested scheduler**: Hermes cron, using scheduler-local dispatcher scripts that call the repo wrappers. -- **Compatible but not yet tested here**: OpenClaw cron, native Unix crontab, systemd timers, or any equivalent scheduler that can run the wrapper scripts on the documented schedule. -- Actual job registration state belongs to the chosen scheduler backend. Run evidence belongs in DB tables (`cron_runs`, `handoffs`, `operation_logs`, and summary `pages`); `data/raw/ops/cron/{YYYY}/{MM}/`, `data/handoffs/`, `data/log.md`, and wrap-up markdown are generated export/debug copies. - -Show the exact list before making cron changes: - -```text -KnowledgeBase jobs: -- daily memory build: 03:30 every day -- wiki promote: 04:00 every day -- weekly memory build: 04:15 every Monday -- monthly memory maintenance:04:45 on day 1 of each month -- wiki TTL sweep: 00:30 every day -- cron wrap-up: 05:00 every day - -Optional usage report jobs: -- OpenCode daily usage: 03:10 every day -- Hermes daily usage: 03:15 every day -- Claude Code daily usage: 03:20 every day - -Optional global digest job: -- morning Slack digest: 09:00 every day (optional, recommended) -``` - -If the user wants the optional global digest, read `reference/optional-global-digest.md` and show the setup separately from the required KB cron jobs. - -Wrapper prompt policy: - -- Memory wrappers import `.claude/skills/memory-report/SKILL.md`. -- Wiki promotion wrapper imports `.claude/skills/wiki-approval/SKILL.md`. -- Cron wrap-up wrapper imports `.claude/skills/cron-wrapup/SKILL.md` and writes a Slack-digest-stable `wiki/summaries/.../{date}-cron-wrapup.md` plus run handoff. -- Usage setup/import prompts use `.claude/skills/usage-report-setup/SKILL.md`. -- TTL sweep runs `uv run kb-db-ttl-sweep --days 7`, applying status changes in-process through the `kb.service` layer. - -### Register kb-mcp in agent runtimes (ask first) +## Phase 4: Register kb-mcp in Agent Runtimes LLM/agent runtimes reach the write surface by connecting to the `kb-mcp` daemon over http. Before registering anything, **ask the user two things**: @@ -225,17 +161,9 @@ Recipes (use the confirmed ``): - **Claude Code** — `claude mcp add --transport http --scope user kb-mcp ` - **Other runtimes (e.g. Hermes)** — ask the user where their MCP config lives; add an http entry pointing at ``. -Deterministic jobs (`kb-db-ttl-sweep`, daily reports, ingest) need no registration — they call `kb.service` in-process. The daemon has no auth: if it must be reachable from other hosts, publish it on the appropriate interface (compose default `0.0.0.0:8765`); on a single host, host-loopback (`127.0.0.1:8765:8765`) is safer (see #41). - -Only after approval: - -1. Create or update wrapper scripts under `scripts/cron/`. -2. Make wrappers executable. -3. Each wrapper writes its run log to `data/raw/ops/cron/{YYYY}/{MM}/{TARGET}_kb-.log` (where YYYY/MM derives from TARGET_DATE). The `kb-cron-wrapup` wrapper writes to `.cron/logs/cron-wrapup.log` during the session, then commits it to `data/raw/ops/cron/` in a follow-up commit after the session exits. -4. Keep locks in `.cron/locks/`. -5. Show scheduler entries for the selected backend, or install/register them only if the user explicitly asks. +The daemon has no auth: if it must be reachable from other hosts, publish it on the appropriate interface (compose default `0.0.0.0:8765`); on a single host, host-loopback (`127.0.0.1:8765:8765`) is safer (see #41). -## Phase 6: Initialization Handoff +## Phase 5: Initialization Handoff Write: @@ -285,7 +213,7 @@ Run: # Handoff validation runs inside the kb-mcp create_handoff tool (returns code: lint_failed on failure) ``` -## Phase 7: Log +## Phase 6: Log Write the setup note through the kb-mcp `create_operation_log` tool (the generated export may update `data/log.md`): @@ -297,8 +225,6 @@ Write the setup note through the kb-mcp `create_operation_log` tool (the generat - **directories**: created / already present - **tooling**: - **global skills**: symlinked / skipped -- **usage reports**: -- **cron**: proposed / approved / skipped - **handoff**: handoffs/YYYY/MM/kb-initialize/.md ``` @@ -309,14 +235,10 @@ Write the setup note through the kb-mcp `create_operation_log` tool (the generat - CLI smoke tests ran or blockers are documented. - Global skills are symlinked into `~/.claude/skills/` or explicitly skipped. - kb-mcp is registered in the agent runtimes the user approved (or explicitly skipped). -- Usage report mode is selected or explicitly skipped. -- Cron entries are proposed or explicitly skipped. - Initialization handoff exists and passes lint. - No existing `data/raw/` file was modified. ## Red Flags - About to write private data into the outer repo. -- About to install crontab without explicit approval. -- About to add auto-commit to daily/weekly/monthly memory wrappers. - About to rewrite existing user data to satisfy lint without instruction. diff --git a/.claude/skills/knowledgebase-initialize/reference/optional-global-digest.md b/.claude/skills/knowledgebase-initialize/reference/optional-global-digest.md deleted file mode 100644 index b07e4bf..0000000 --- a/.claude/skills/knowledgebase-initialize/reference/optional-global-digest.md +++ /dev/null @@ -1,40 +0,0 @@ -# Optional Global Digest - -The morning global digest is optional but recommended once `kb-cron-wrapup` is producing stable, DB-backed daily summaries. - -## Purpose - -Create one morning notification that reports the previous night's KnowledgeBase pipeline state without re-reading KnowledgeBase internals. This job is read-only: it reports from the `kb-cron-wrapup` artefact and does not create, edit, lint, or commit KB data. - -The digest should consume only: - -- `data/wiki/summaries/YYYY/MM/YYYY-MM-DD-cron-wrapup.md` -- The fixed H2 sections: `Status`, `Insights`, `Action Items`, `Anomalies`, `Counters`, `Links` - -It should not read: - -- `data/wiki/summaries/.../*-memory.md` -- `data/raw/` (including `data/raw/ops/cron/` — that is the wrap-up's input, not the digest's) -- individual usage report pages - -## Recommended Schedule - -```text -morning-slack-digest: 09:00 every day -``` - -Run it after the `05:00` `kb-cron-wrapup` job so the digest has a single stable, exported artifact to parse. - -## Setup Guidance - -Use whatever scheduler or agent runtime the user already uses for notifications. Keep the job global rather than KnowledgeBase-owned, because it is a delivery concern, not a KB data-generation or data-commit step. - -Recommended prompt shape: - -```text -Read the latest KnowledgeBase cron wrap-up summary for yesterday. Summarize only the fixed H2 contract sections: Status, Insights, Action Items, Anomalies, Counters, and Links. Do not inspect memory pages, raw data, usage pages, or cron logs directly. Do not create, edit, lint, commit, or push KB files. Send a concise morning status digest. -``` - -## Dependency - -Set this up only after at least one `*-cron-wrapup.md` file exists and follows the fixed H2 contract. diff --git a/.claude/skills/memory-report/SKILL.md b/.claude/skills/memory-report/SKILL.md deleted file mode 100644 index 19379eb..0000000 --- a/.claude/skills/memory-report/SKILL.md +++ /dev/null @@ -1,401 +0,0 @@ ---- -name: memory-report -description: Use when running the daily, weekly, or monthly memory workflow — discovering period sources, writing summary + worthy wiki pages through the kb-mcp tools, writing the period handoff, and lint-clean uncommitted output. Covers raw discovery, period dispatch (daily/weekly/monthly), wiki page frontmatter, lint ordering, and canonical lint failures. ---- - -# memory-report - -## DB-Canonical Override - -DB rows are the source of truth. Write summaries, pages, handoffs, and operation -logs through the kb-mcp tools (`upsert_page`, `patch_page`, `create_handoff`, -`create_operation_log`, `create_raw_source`) — these run inside the `opencode run` -cron session where the kb-mcp MCP server is registered. -**Reads** (source discovery in each Step 0) go directly to Postgres via `psql` -(or the read-only kb-mcp `query_sql` tool) — -schema + recipes in `docs/db_informations/state-db-schema-reference.md`. -Markdown under `data/` is generated export. If any older instruction below says -to write files, lint as a write gate, commit `data/`, or discover sources by -scanning `data/`, ignore that instruction and use the DB instead. - -## Overview - -One skill, three periods. The agent receives a target (`YYYY-MM-DD`, `YYYY-WNN`, or `YYYY-MM`) and produces a period summary + zero-or-more wiki pages + one handoff, all lint-clean and uncommitted. - -The shared half (Repo layout, wiki schemas, lint order, common failures) is identical across periods. The per-period half (source discovery, promotion intensity, output filename) differs and lives in three small sections below. - -This SKILL replaces the doc-load pattern where each cron agent re-read 6 markdown files on every run. - -**Bundled reference (self-contained — do not consult docs/ at runtime):** -- `reference/templates/{entity,concept,decision,question,improvement,checklist}.md` — period workflow copies remain here for backward compatibility -- `reference/templates/summaries/{weekly,monthly}.md` — weekly/monthly summary skeletons (daily summary schema is inlined below) - -For new atomic wiki pages, import `wiki-authoring` as the canonical page-authoring contract. Handoff authoring is delegated to `handoff-document` — invoke that skill for the period handoff. This skill does not duplicate handoff schema. - -If anything below contradicts the lint code (`src/kb/lint/wiki.py`), the lint code wins — file an issue and update this skill. - -## Period Dispatch — find your period BEFORE reading sections - -| Period | Cron schedule (KST) | Target arg format | Lock file | Skill section | -|---|---|---|---|---| -| daily | `30 3 * * *` | `YYYY-MM-DD` | `daily.lock` | [§Daily](#daily) | -| weekly | `15 4 * * 1` | `YYYY-WNN` (ISO week) | `weekly.lock` | [§Weekly](#weekly) | -| monthly | `45 4 1 * *` | `YYYY-MM` | `monthly.lock` | [§Monthly](#monthly) | - -If the cron prompt says "daily memory workflow for 2026-05-19", you're in §Daily. Read SHARED CONVENTIONS once, then jump to the matching period section. - -## When to Use - -- Any of the three crons fires -- Manual invocation: "Run the {daily|weekly|monthly} memory workflow for {target}" - -**Do NOT use** for: wiki promotion (separate workflow), raw ingestion (manual / external scripts), or recovery from a previously failed run without reading the prior handoff first. - ---- - -# Shared Conventions - -## Repo Layout (memorize, don't re-discover) - -``` -data/ -├── raw/ -│ ├── github/ # GitHub CLAUDE.md + issues/PRs (flat — date in filename) -│ ├── gmail/{YYYY}/{MM}/ # date-partitioned -│ ├── sessions/{YYYY}/{MM}/ # date-partitioned -│ ├── web/ # flat -│ └── ops/ -│ ├── cron/{YYYY}/{MM}/ # cron-generated usage reports -│ ├── runs/{YYYY}/{MM}/ -│ ├── decisions/{YYYY}/{MM}/ -│ ├── traces/{YYYY}/{MM}/ -│ └── ingest_state/ # state files, not source -├── handoffs/{YYYY}/{MM}// -└── wiki/ - ├── entities/{subject}/{YYYY-MM}/ - ├── improvements/{YYYY-MM}/ - ├── concepts/ # FLAT (timeless ideas) - ├── decisions/ # flat, filename prefixes date: 2026-05-18-foo.md - ├── questions/ # flat - ├── checklists/ # flat - └── summaries/{YYYY}/{MM}/ # YYYY-MM-DD-memory.md, YYYY-WNN-weekly.md, YYYY-MM-monthly.md -``` - -`sources:` frontmatter paths are **relative to `data/`** (e.g. `wiki/summaries/...`, NOT `data/wiki/summaries/...`). - -## Wiki Page Schemas (the only thing lint ERRORs you for) - -All review-tracked types (`entity`, `concept`, `decision`, `improvement`, `checklist`, `question`) inherit `review_status: not_processed` from template. Don't manually set it to anything else. - -### summary - -```yaml ---- -type: summary -subtype: daily | weekly | monthly -date: "YYYY-MM-DD" # daily only -week: "YYYY-WNN" # weekly only -month: "YYYY-MM" # monthly only -period_start: "YYYY-MM-DD" # weekly/monthly -period_end: "YYYY-MM-DD" # weekly/monthly -created: "YYYY-MM-DD" -updated: "YYYY-MM-DD" -sources: - - wiki/summaries/YYYY/MM/... - - handoffs/YYYY/MM//... -tags: [] ---- -``` -Required: `type, created, updated, sources, tags` + the period-specific date fields. **No `review_status`** — summaries are exempt from approval. - -### entity / concept / question - -```yaml ---- -type: entity # or concept, question -review_status: not_processed -created: "YYYY-MM-DD" -updated: "YYYY-MM-DD" -sources: [...] -aliases: [] # entity & concept only -tags: [] ---- -``` - -### improvement — **the lint trap** - -```yaml ---- -type: improvement -review_status: not_processed -kind: improvement # one of: improvement | issue | proposal -observed_at: "YYYY-MM-DD" # ISO date — required, lint validates format -domain: dx # one of: cost | correctness | perf | dx | security -severity: med # one of: low | med | high -issue_status: open # one of: open | acknowledged | resolved | wontfix -related: [] # list of wikilinks, lint resolves them -created: "YYYY-MM-DD" -updated: "YYYY-MM-DD" -sources: [...] -tags: [] ---- -``` - -Missing any of `kind/observed_at/domain/severity/issue_status/related` → ERROR. Invalid enum → ERROR. - -### decision - -```yaml ---- -type: decision -review_status: not_processed -created: "YYYY-MM-DD" -updated: "YYYY-MM-DD" -sources: [...] -tags: [decision] ---- -``` -Filename convention: `YYYY-MM-DD-.md` (date prefix in filename, not folder). - -### checklist - -```yaml ---- -type: checklist -review_status: not_processed -created: "YYYY-MM-DD" -updated: "YYYY-MM-DD" -sources: [...] -tags: [] ---- -``` -Body **must** contain `## Items` section with `- [ ] ...` task-list syntax. Lint enforces this. - -## DB Write Order - -The kb-mcp write tools take **structured args** — pass the `frontmatter` as an -object (not YAML text) and `body_md` as the body without the `---` fence, plus -`slug`, `type`, and `export_path`. The same applies to `create_handoff`. - -1. Submit the summary page through the kb-mcp `upsert_page` tool. -2. Submit any new or updated evidence-derived wiki pages through the kb-mcp - `upsert_page` tool (use the kb-mcp `patch_page` tool for partial updates). -3. Submit the run handoff through the kb-mcp `create_handoff` tool. -4. Submit the operation note through the kb-mcp `create_operation_log` tool. -5. Confirm each tool result reports `export.status == success`. On an - `error`/`code` result, handle it (`lint_failed` → fix and retry; - `conflict` → already exists). - -## Common Lint Failures (preemptive fixes) - -| ERROR pattern | Cause | Fix before lint | -|---|---|---| -| `source file not found: wiki/summaries/daily/...` | Cited a legacy path. Daily summaries live at `wiki/summaries/{YYYY}/{MM}/`, not `wiki/summaries/daily/` | Use the partitioned path | -| `source file not found: data/raw/...` | `sources:` paths must be relative to `data/`, not `data/data/` | Drop the `data/` prefix | -| `missing frontmatter field: kind` (improvement) | Used base template, didn't fill improvement-specific fields | All 6 extra fields required | -| `invalid domain: 'xxx'` | Domain not in enum | Use one of: cost, correctness, perf, dx, security | -| `raw file modified after creation (immutability violation)` | Someone (not you) edited a raw file | **Do not touch raw files.** Log in handoff §8; cannot be fixed by this run | -| `stub page — body N chars (< 100)` | WARN only, ignore | — | -| `orphan page — no inbound links` | WARN only, expected for new pages | — | - -## Handoff (defer to `handoff-document`) - -Every run writes one handoff. Use the `handoff-document` skill for filename grammar, frontmatter, body sections, and lint. This skill only specifies the **task folder name** and **content focus** per period (see each period section). - -## Log Format (submit through the kb-mcp `create_operation_log` tool, all periods) - -```markdown - -## YYYY-MM-DD ({period} memory build — target: {TARGET}) - -- **fill**: {TARGET} {period} synthesis - - 소스: - - 출력: wiki/summaries/YYYY/MM/.md (신규) -- **fill**: -- **handoff**: -- **db_write**: pages + handoff + operation log exported successfully -- Note: commit은 사용자가 수동으로 review 후 진행 -``` - -## Workflow Discipline (all periods) - -- **One target per run.** Never extend scope implicitly to "and also yesterday/last week". -- **Never edit raw files.** Raw sources are DB rows; add new raw evidence through the kb-mcp `create_raw_source` tool. -- **Do not auto-promote** wiki pages to `pending_for_approve` — that's the wiki-promote cron's job. -- **Do not git commit.** Markdown is generated export from DB rows. -- **If blocked**, write the handoff with `status: ready`, list the blocker in §8, and exit non-zero. - ---- - -# §Daily - -**Target**: `YYYY-MM-DD` (yesterday). -**Output summary**: `data/wiki/summaries/{YYYY}/{MM}/{TARGET}-memory.md` -**Handoff folder**: `data/handoffs/{YYYY}/{MM}/wiki-daily-build/` -**Promotion intensity**: triage > restructure. Prefer summary; create wiki pages only when source has clear future value. - -## Daily Step 0: Identify yesterday's raw — query the DB, never scan `data/` - -Postgres is the source of truth; `data/` is a human-readable export. Discover sources with `psql`, not by walking the export tree. - -```bash -TARGET="$1" # e.g. 2026-05-19 -psql_kb() { psql "${DATABASE_URL/+psycopg/}" -tAc "$1"; } - -# Raw ingested on TARGET. captured_at is stored in UTC, so range over the KST day. -FROM=$(date -u -d "$TARGET 00:00:00+09:00" +%FT%T) -TO=$(date -u -d "$TARGET 00:00:00+09:00 +1 day" +%FT%T) -psql_kb "SELECT source_key, source_type, captured_at FROM raw_sources - WHERE captured_at >= '$FROM' AND captured_at < '$TO' ORDER BY captured_at;" - -# Ready handoffs for context (prior daily build + anything else still open) -psql_kb "SELECT task_slug, handoff_id, status FROM handoffs - WHERE status='ready' ORDER BY updated_at DESC;" -``` - -Never `find`/`ls`/`grep` under `data/` to discover sources — the export can lag the DB. - -## Daily Workflow (9 steps) - -``` -1. Run Daily Step 0 → raw set + active handoffs for TARGET -2. Read prior wiki-daily-build handoff (status: ready) for context -3. Write data/wiki/summaries/{Y}/{M}/{TARGET}-memory.md -4. Create only obviously-worthy entity/improvement/concept pages - — leave uncertain items in handoff "Promotion Candidates" -5. Mark prior daily handoff status: consumed IF its open items were handled -6. Write new handoff via `handoff-document` under wiki-daily-build/ -7. Submit operation log through the kb-mcp `create_operation_log` tool -8. Confirm each kb-mcp tool result reports export.status == success; on an `error`/`code` result handle it (lint_failed → fix and retry; conflict → already exists) -9. STOP. Do NOT git commit. -``` - -## Daily promotion intensity - -Prefer summary + triage. Create atomic pages only when source has clear future value. Uncertain → handoff's §10 Promotion Candidates, not a new page. - ---- - -# §Weekly - -**Target**: `YYYY-WNN` (last ISO week, e.g. `2026-W20`). -**Output summary**: `data/wiki/summaries/{YYYY}/{MM}/{TARGET}-weekly.md` (where MM = month of week's end) -**Template**: `reference/templates/summaries/weekly.md` -**Handoff folder**: `data/handoffs/{YYYY}/{MM}/wiki-weekly-build/` -**Promotion intensity**: patterns over events. Promote signals **repeated across ≥2 days** or strongly supported by one important source. - -## Weekly Step 0: collect the week's daily summaries and handoffs - -```bash -TARGET="$1" # e.g. 2026-W20 -psql_kb() { psql "${DATABASE_URL/+psycopg/}" -tAc "$1"; } -# Resolve ISO week → Monday date and Sunday date -PERIOD_START=$(date -d "$(echo $TARGET | sed 's/-W/ /')-1" +%F 2>/dev/null \ - || date -d "${TARGET%-W*}-01-01 +$((10#${TARGET#*-W} * 7 - 7)) days" +%F) -PERIOD_END=$(date -d "$PERIOD_START +6 days" +%F) - -# Daily memory summaries falling in the week (slug starts with the date) -psql_kb "SELECT slug FROM pages WHERE type='summary' AND slug LIKE '%-memory' - AND left(slug,10) BETWEEN '$PERIOD_START' AND '$PERIOD_END' ORDER BY slug;" - -# Ready handoffs from the daily build for the period -psql_kb "SELECT handoff_id, status FROM handoffs - WHERE task_slug='wiki-daily-build' AND status='ready' ORDER BY updated_at DESC;" -``` - -Sanity check: expect 7 daily memory summary rows. Document any missing day in the handoff §8; do not synthesize phantom days. - -## Weekly Workflow (10 steps) - -``` -1. Run Weekly Step 0 → seven daily summaries + ready daily handoffs -2. Read prior wiki-weekly-build handoff (status: ready) if any -3. Write data/wiki/summaries/{Y}/{M}/{TARGET}-weekly.md using reference/templates/summaries/weekly.md -4. Promote repeated patterns into concepts, improvements, checklists, or decisions - (see Weekly promotion intensity below) -5. Mark consumed daily handoffs as status: consumed ONLY for items actually handled -6. Write new handoff via `handoff-document` under wiki-weekly-build/ -7. Note monthly candidates in handoff §10 (which patterns are monthly-worthy) -8. Submit operation log through the kb-mcp `create_operation_log` tool -9. Confirm each kb-mcp tool result reports export.status == success; on an `error`/`code` result handle it (lint_failed → fix and retry; conflict → already exists) -10. STOP. Do NOT git commit. -``` - -## Weekly promotion intensity - -- **Promote** when signal repeats across multiple days or has one strong-source support. -- Convert recurring operational mistakes → **checklist** candidates. -- Convert unresolved-but-actionable work → **improvements**. -- Convert closed architectural/workflow choices → **decisions** (only if rationale is captured). -- Don't promote single-day mentions unless source is high-value. - ---- - -# §Monthly - -**Target**: `YYYY-MM` (last month). -**Output summary**: `data/wiki/summaries/{YYYY}/{MM}/{TARGET}-monthly.md` -**Template**: `reference/templates/summaries/monthly.md` -**Handoff folder**: `data/handoffs/{YYYY}/{MM}/wiki-monthly-maintenance/` -**Promotion intensity**: cleanup > add. Consolidate duplicates, close stale, promote stable procedures. - -## Monthly Step 0: collect weekly summaries and prior monthly handoff - -```bash -TARGET="$1" # e.g. 2026-05 -YEAR="${TARGET%-*}" -MONTH="${TARGET#*-}" -psql_kb() { psql "${DATABASE_URL/+psycopg/}" -tAc "$1"; } - -# Weekly summaries exported under this month -psql_kb "SELECT slug FROM pages WHERE type='summary' AND slug LIKE '%-weekly' - AND export_path LIKE 'wiki/summaries/$YEAR/$MONTH/%' ORDER BY slug;" - -# Prior monthly maintenance handoffs (ready only) -psql_kb "SELECT handoff_id, status FROM handoffs - WHERE task_slug='wiki-monthly-maintenance' AND status='ready' ORDER BY updated_at DESC;" - -# Cleanup signal — orphan/stub pages flagged by lint -uv run kb-lint wiki -``` - -## Monthly Workflow (11 steps) - -``` -1. Run Monthly Step 0 → weekly summaries + ready monthly handoff + cleanup signals -2. Review duplicate concepts (same idea, different page), stale improvements - (issue_status: open but observed_at > 30 days ago), open decision candidates -3. Write data/wiki/summaries/{Y}/{M}/{TARGET}-monthly.md using reference/templates/summaries/monthly.md -4. Consolidate duplicated concepts ONLY when sources support the merge - (different evidence → keep both; same evidence → merge with combined sources) -5. Close or defer stale improvements: set issue_status to resolved/wontfix with rationale, - OR leave open and note the blocker -6. Promote stable repeated procedures into checklists -7. Mark consumed weekly handoffs as status: consumed for items handled -8. Write monthly handoff via `handoff-document` under wiki-monthly-maintenance/ - — flag reusable automation as promotion: skill_candidate -9. Submit operation log through the kb-mcp `create_operation_log` tool -10. Confirm each kb-mcp tool result reports export.status == success; on an `error`/`code` result handle it (lint_failed → fix and retry; conflict → already exists) -11. STOP. Do NOT git commit. -``` - -## Monthly promotion intensity - -- **Prefer cleanup** over new pages. Don't add fresh content the weekly should have caught. -- **Don't merge pages** if source evidence differs in meaning — separate facts deserve separate pages. -- **Mark reusable automation patterns** as `promotion: skill_candidate` in handoff §10. Don't write the skill in this run; flag it for the user. -- **Leave human-decision items open** rather than turning them into ADRs. Decisions are user-driven, not synthesized. - ---- - -# Red Flags — STOP and re-check - -- About to `Read data/raw/` for the 3rd time → use Step 0 find command for your period -- About to create a wiki page but haven't checked `sources:` path is `data/`-relative → check before submitting via kb-mcp -- About to write improvement page from `reference/templates/improvement.md` without filling the 6 extra fields → fill them or downgrade to entity/concept -- About to lint exported markdown files instead of using the kb-mcp tools → STOP. Submit through the kb-mcp `upsert_page`/`create_handoff`/`create_operation_log` tools. -- About to `git commit` → STOP. Markdown is generated export from DB rows. -- Lint ERRORs on raw immutability → not your run's fault, log it in handoff §8 and PASS the run -- About to mix periods in one run (e.g. "and also catch up last week's") → STOP. Run one period at a time -- Weekly run with <7 daily summaries → document missing days in handoff; do not synthesize phantom days -- Monthly run with no weekly summaries → STOP. Weekly must run first; write a handoff explaining and exit diff --git a/.claude/skills/memory-report/reference/templates/checklist.md b/.claude/skills/memory-report/reference/templates/checklist.md deleted file mode 100644 index 3c6e791..0000000 --- a/.claude/skills/memory-report/reference/templates/checklist.md +++ /dev/null @@ -1,19 +0,0 @@ ---- -type: checklist -review_status: not_processed -created: "" -updated: "" -sources: [] -tags: [] ---- - -# {{ChecklistTitle}} - -## Purpose - -## Items - -- [ ] item 1 -- [ ] item 2 - -## Notes diff --git a/.claude/skills/memory-report/reference/templates/concept.md b/.claude/skills/memory-report/reference/templates/concept.md deleted file mode 100644 index 5823623..0000000 --- a/.claude/skills/memory-report/reference/templates/concept.md +++ /dev/null @@ -1,18 +0,0 @@ ---- -type: concept -review_status: not_processed -created: "" -updated: "" -sources: [] -aliases: [] -tags: [] ---- - -# {{ConceptName}} - -## Definition - -## Context - -## Related - diff --git a/.claude/skills/memory-report/reference/templates/decision.md b/.claude/skills/memory-report/reference/templates/decision.md deleted file mode 100644 index 59aa025..0000000 --- a/.claude/skills/memory-report/reference/templates/decision.md +++ /dev/null @@ -1,17 +0,0 @@ ---- -type: decision -review_status: not_processed -created: "" -updated: "" -sources: [] -tags: [decision] ---- - -# {{DecisionTitle}} - -## Context - -## Decision - -## Consequences - diff --git a/.claude/skills/memory-report/reference/templates/entity.md b/.claude/skills/memory-report/reference/templates/entity.md deleted file mode 100644 index 1a07b84..0000000 --- a/.claude/skills/memory-report/reference/templates/entity.md +++ /dev/null @@ -1,18 +0,0 @@ ---- -type: entity -review_status: not_processed -created: "" -updated: "" -sources: [] -aliases: [] -tags: [] ---- - -# {{EntityName}} - -## Overview - -## Key Details - -## Related - diff --git a/.claude/skills/memory-report/reference/templates/improvement.md b/.claude/skills/memory-report/reference/templates/improvement.md deleted file mode 100644 index 24865d9..0000000 --- a/.claude/skills/memory-report/reference/templates/improvement.md +++ /dev/null @@ -1,24 +0,0 @@ ---- -type: improvement -review_status: not_processed -kind: improvement -observed_at: "" -domain: "" -severity: "" -issue_status: open -related: [] -created: "" -updated: "" -sources: [] -tags: [] ---- - -# {{ImprovementTitle}} - -## Observation - -## Impact - -## Proposed Action - -## Notes diff --git a/.claude/skills/memory-report/reference/templates/question.md b/.claude/skills/memory-report/reference/templates/question.md deleted file mode 100644 index 9c30f3e..0000000 --- a/.claude/skills/memory-report/reference/templates/question.md +++ /dev/null @@ -1,18 +0,0 @@ ---- -type: question -review_status: not_processed -created: "" -updated: "" -sources: [] -tags: [] ---- - -# {{QuestionTitle}} - -## Question - -## Answer - -## Context - -## Related diff --git a/.claude/skills/memory-report/reference/templates/summaries/monthly.md b/.claude/skills/memory-report/reference/templates/summaries/monthly.md deleted file mode 100644 index 707f806..0000000 --- a/.claude/skills/memory-report/reference/templates/summaries/monthly.md +++ /dev/null @@ -1,49 +0,0 @@ ---- -type: summary -subtype: monthly -month: "" -period_start: "" -period_end: "" -created: "" -updated: "" -sources: [] -tags: [] ---- - -# Monthly Report — {{YYYY-MM}} - -## Summary - -- Total sessions: N / active days: N (empty days: N) -- Total tokens: N / average daily tokens: N -- Average cache hit rate: N% -- Average TODO completion rate: N% - -## Weekly Trends - -| Week | Sessions | Tokens | Cache Hit Rate | TODO Completion | Notes | -|------|------|------|------------|------------|------| -| W01 | | | | | | -| W02 | | | | | | -| W03 | | | | | | -| W04 | | | | | | - -## Monthly Insights - -### What Went Well - - - -### Problem Patterns - - - -### Action Items - - - - -## Observations - - - diff --git a/.claude/skills/memory-report/reference/templates/summaries/weekly.md b/.claude/skills/memory-report/reference/templates/summaries/weekly.md deleted file mode 100644 index c9f3235..0000000 --- a/.claude/skills/memory-report/reference/templates/summaries/weekly.md +++ /dev/null @@ -1,66 +0,0 @@ ---- -type: summary -subtype: weekly -week: "" -period_start: "" -period_end: "" -created: "" -updated: "" -sources: [] -tags: [] ---- - -# Weekly Summary - {{YYYY-WNN}} - -## Summary - -- Period: {{YYYY-MM-DD}} to {{YYYY-MM-DD}} -- Source reports reviewed: N -- Active systems: OpenCode / Hermes / Claude Code / other -- Main pattern: - -## Daily Coverage - -| Date | Memory | OpenCode | Hermes | Claude Code | Notes | -|---|---|---|---|---|---| -| YYYY-MM-DD | present/missing | present/missing/N/A | present/missing/N/A | present/missing/N/A | | - -## Signals - -### Work Patterns - - - -### Quality / Reliability - - - -### Cost / Efficiency - - - -## Promotions - -### Concepts - - - -### Improvements - - - -### Checklists - - - -### Decisions - - - -## Open Items - - - -## Next Week - - diff --git a/.claude/skills/usage-report-setup/SKILL.md b/.claude/skills/usage-report-setup/SKILL.md deleted file mode 100644 index 6bed3fc..0000000 --- a/.claude/skills/usage-report-setup/SKILL.md +++ /dev/null @@ -1,117 +0,0 @@ ---- -name: usage-report-setup -description: Use when choosing OpenCode, Hermes, or Claude Code usage report modes — creating or auditing source-specific daily report cron jobs, configuring report output paths, handling missing telemetry sources, or wiring usage reports before daily memory builds. ---- - -# Usage Report Setup - -Use this skill as the runtime contract for source-specific usage report setup. Do not look for a workflow doc during execution; this skill is the complete operating surface. - -## Policy - -Generate separate reports by source. Do not create a combined usage report in this layer; daily memory synthesis may read multiple source reports and write one combined memory summary later. -Reports are written in-process through the `kb.service` layer (lint → DB → Markdown -export). Markdown under `data/wiki/` is generated export, not the source of truth. -Set `DATABASE_URL` (and optionally `KB_DATA_DIR`) before running any non-dry-run -report command. There is no token to set. - -| Source | Command | DB Output | -|---|---|---| -| OpenCode | `uv run kb-opencode-daily-report --date YYYY-MM-DD --lint` | CLI writes metrics + summary page in-process via the service layer | -| Hermes | `uv run kb-hermes-daily-report --date YYYY-MM-DD --lint` | CLI writes metrics + summary page in-process via the service layer | -| Claude Code | `uv run kb-claude-code-daily-report --date YYYY-MM-DD --lint` | CLI writes metrics + summary page in-process via the service layer | - -Metrics are canonical in the DB, written by the CLI through the service layer; a derived `.metrics.json` is exported under `data/ops/reports/`, not authored by hand. - -## Setup Workflow - -1. Inspect which report commands exist: - ```bash - uv run kb-opencode-daily-report --help - uv run kb-hermes-daily-report --help - uv run kb-claude-code-daily-report --help - ``` -2. Ask the user which modes to enable if not already specified: - - none - - OpenCode only - - Hermes only - - Claude Code only - - multiple separate reports -3. Verify matching wrapper scripts under `scripts/cron/`. -4. Ensure usage report jobs run before daily memory build. -5. Propose crontab entries. Do not install them unless the user explicitly approves installation. -6. Run a manual dry run only for selected sources. -7. Verify service-layer write success, generated markdown export, and metrics JSON output. - -## Cron Schedule - -Recommended KST ordering: - -```cron -10 3 * * * /scripts/cron/kb-opencode-daily-report.sh -15 3 * * * /scripts/cron/kb-hermes-daily-report.sh -20 3 * * * /scripts/cron/kb-claude-code-daily-report.sh -30 3 * * * /scripts/cron/kb-memory-daily.sh -``` - -Enable only selected sources. - -## Wrapper Contract - -Each wrapper must: - -1. Resolve `KB_ROOT` from the script location. -2. Create `.cron/locks`. -3. Use a source-specific `flock` lock. -4. Compute target date in KST as yesterday. -5. Compute `LOG_FILE="$KB_ROOT/data/raw/ops/cron/$(TZ=Asia/Seoul date -d "$TARGET_DATE" +%Y/%m)/${TARGET_DATE}_kb--daily-report.log"` and `mkdir -p` its parent. -6. Run the source-specific report command with `--lint`, redirecting stdout/stderr to `$LOG_FILE`. -7. Submit the completed wrapper log through `uv run kb-submit-cron-run`. -8. Exit non-zero if the report command or DB log submission fails. - -Example command inside a wrapper: - -```bash -uv run kb-opencode-daily-report --date "$TARGET_DATE" --lint -``` - -## Missing Source Handling - -If a selected source is not installed or has no database/telemetry: - -- do not fail unrelated source report jobs -- do not generate a misleading empty success report unless the user requested empty-day reports -- write a clear wrapper log entry -- if an agent is involved, write a handoff or operation log through the service layer - -## Output Rules - -Daily usage report markdown must: - -- use `type: summary` -- use `subtype: daily` -- set `date`, `created`, and `updated` -- use `sources: []` for local DB/telemetry inputs -- avoid dollar signs; write `N USD` -- be accepted by the service-layer write (the same path the report CLI uses); Markdown is exported after the DB write - -## Validation - -For each enabled source: - -```bash -uv run kb--daily-report --date YYYY-MM-DD --lint -# DB is canonical and write-only (no read endpoints). The command prints -# `export.status: success` on a good write; confirm the derived exports landed: -ls data/wiki/summaries/YYYY/MM/*--usage.md -ls data/ops/reports/YYYY/MM/*-usage.metrics.json -``` - -Do not commit `data/`; it is generated export. - -## Red Flags - -- About to create a combined usage report command in this layer. -- About to treat a missing optional source as failure for all reports. -- About to run daily memory before selected usage reports. -- About to install crontab entries without explicit user approval. diff --git a/.claude/skills/wiki-approval/SKILL.md b/.claude/skills/wiki-approval/SKILL.md deleted file mode 100644 index 9921d88..0000000 --- a/.claude/skills/wiki-approval/SKILL.md +++ /dev/null @@ -1,202 +0,0 @@ ---- -name: wiki-approval -description: Use when promoting DB-backed wiki pages from not_processed to pending_for_approve — approving or rejecting pages, running TTL sweep, updating approved pages, and writing wiki-promote handoffs/logs through kb-mcp tools. ---- - -# Wiki Approval - -## DB-Canonical Override - -Approval state lives in DB pages. Use the kb-mcp tools (`promote_page`, -`approve_page`, `reject_page`, `ttl_sweep_pages`, `create_handoff`, -`create_operation_log`) for promote/approve/reject/TTL sweep, handoffs, and -operation logs. Markdown under `data/` is generated export. If any older -instruction below says to lint as a write gate or commit `data/`, prefer the -kb-mcp tools. - -Use this skill as the runtime contract for the KnowledgeBase review lifecycle. Do not look for a workflow doc during execution; this skill is the complete operating surface. - -## Status Model - -Applies only to wiki page types `entity`, `concept`, `decision`, `improvement`, `checklist`, and `question`. - -| Status | Meaning | Next | -|---|---|---| -| `not_processed` | AI wrote or semantically changed the page | promote or TTL reject | -| `pending_for_approve` | Waiting for human review | approve or reject | -| `approved` | Official wiki content | semantic edit self-resets to `not_processed` | -| `rejected` | Preserved outside wiki under `data/rejected/` | terminal | - -`summary` and `index` pages are outside the approval lifecycle. - -## Commands - -**Reads** go directly to Postgres (set `DATABASE_URL`; schema + recipes in -`docs/db_informations/state-db-schema-reference.md`). The read-only kb-mcp -`query_sql` tool is an equivalent alternative: - -```bash -psql "${DATABASE_URL/+psycopg/}" -tAc \ - "SELECT slug, type, created_at FROM pages WHERE review_status='not_processed' ORDER BY updated_at DESC;" -``` - -**Writes** go through the kb-mcp tools: - -- call the kb-mcp `promote_page` tool with `slug=`, `feedback=""`, `source="agent"` -- call the kb-mcp `approve_page` tool with `slug=`, `feedback=""`, `source="user"` -- call the kb-mcp `reject_page` tool with `slug=`, `feedback=""`, `source="user"` -- call the kb-mcp `ttl_sweep_pages` tool with `days=7` - -State machine: `promote_page` requires the page be in `not_processed`, `approve_page` -requires `pending_for_approve`, and `reject_page` works from `pending_for_approve` -or `not_processed`. A tool result with `code: conflict` means the page was not in -the expected `review_status` for that transition. - -`` is the filename without `.md`. - -## Reserved Body Section - -`## User Feedback` is CLI-owned. Never use that exact heading as normal wiki content. Use `## Feedback`, `## Reviewer Notes`, or another heading for authored content. - -## Promotion Workflow - -Use this for the daily `wiki-promote` cron or a manual promotion run. - -1. List the not_processed queue (direct Postgres read): - ```bash - psql "${DATABASE_URL/+psycopg/}" -tAc \ - "SELECT slug, type, created_at FROM pages WHERE review_status='not_processed' ORDER BY updated_at DESC;" - ``` -2. Prioritize recent `not_processed` pages. -3. Promote only when all are true: - - source paths are clear, real, and verifiable - - future lookup value exists - - page is durable knowledge, not a raw event dump -4. Promote worthy pages: call the kb-mcp `promote_page` tool with `slug=`, - `feedback=""`, `source="agent"`. The page must be in `not_processed`; a - `code: conflict` result means it was not. -5. Leave borderline pages untouched. Do not reject manually from an agent promotion run. -6. Confirm each tool result reports `export.status == success`; on an `error`/`code` - result handle it (conflict → page not in the expected `review_status`; - not_found → wrong slug). -7. Write a promotion handoff via the kb-mcp `create_handoff` tool. -8. Append the operation note via the kb-mcp `create_operation_log` tool. -9. Do not commit `data/`; it is generated export. - -## Human Approval / Rejection - -For user-requested page review: - -1. List pending pages (direct Postgres read): - ```bash - psql "${DATABASE_URL/+psycopg/}" -tAc \ - "SELECT slug, type FROM pages WHERE review_status='pending_for_approve';" - ``` -2. Read the page and its `sources:` files. -3. Approve when the page is correct and useful: call the kb-mcp `approve_page` - tool with `slug=`, `feedback="..."`, `source="user"`. The page must be - in `pending_for_approve`; a `code: conflict` result means it was not. -4. Reject only when the user has decided it should not enter the wiki: call the - kb-mcp `reject_page` tool with `slug=`, `feedback="..."`, `source="user"`. - This works from `pending_for_approve` or `not_processed`. -5. Confirm the tool result reports `export.status == success`; on an `error`/`code` - result handle it (conflict → page not in the expected `review_status`; - not_found → wrong slug). - -Rejected pages are marked in DB with `review_status: rejected`, `rejected_at`, -and `rejected_by`; export writes them under `data/rejected/...`. - -## TTL Sweep - -Cron-safe deterministic cleanup: call the kb-mcp `ttl_sweep_pages` tool with `days=7`. - -It rejects `not_processed` pages older than the threshold with `rejected_by: auto_ttl`. The wrapper may run this directly without an agent because no judgment is needed. - -## Editing Approved Pages - -When an agent edits an `approved` page: - -- typo, formatting, link-only cleanup: keep `review_status: approved` -- factual change, new information, changed conclusion, changed scope: set `review_status: not_processed` - -There is no deterministic drift detector. The editing agent must make this call. - -## Subject Hub Rule - -`INDEX.md` is generated. Subject `_index.md` hub files are not auto-synchronized. After approval, a user or companion agent may add a manual `- [[]]` line where useful. Missing hub entries are warnings, not blockers. - -## Handoff Quick Contract - -Promotion runs write under: - -```text -data/handoffs/YYYY/MM/wiki-promote/ -``` - -Use a filename that passes handoff validation (the kb-mcp `create_handoff` tool validates on submission), for example: - -```text -wiki-promote_opencode_handoff_01.md -``` - -Required frontmatter: - -```yaml ---- -handoff_id: "wiki-promote:wiki-promote:opencode:01" -task_slug: "wiki-promote" -subject: "wiki-promote" -role: opencode -handoff_seq: 1 -created: "YYYY-MM-DD" -updated: "YYYY-MM-DD" -status: ready -security: - contains_secrets: false - redaction_status: unchecked -promotion: null ---- -``` - -Include these body sections: - -```markdown -## 1. Assignment -## 2. Context received -## 3. Work performed -## 4. Tool trace -## 5. Findings / decisions -## 6. Outputs -## 7. Verification -## 8. Risks / uncertainties -## 9. Next handoff instructions -## 10. Promotion candidates -``` - -`## 4. Tool trace` table must have 7 columns: - -```markdown -| # | Tool | Purpose | Input summary | Output summary | Status | Side effect | -| --- | --- | --- | --- | --- | --- | --- | -``` - -## Log Format - -Submit through the kb-mcp `create_operation_log` tool (export may update `data/log.md`, but the DB row is canonical): - -```markdown - -## YYYY-MM-DD (wiki promotion) - -- **promoted**: -- **left**: -- **handoff**: handoffs/YYYY/MM/wiki-promote/.md -- **db_write**: promoted pages + handoff + operation log exported successfully -``` - -## Red Flags - -- About to reject during promotion run: stop. Agents promote or leave; users reject. -- About to use `## User Feedback` as authored content: rename the heading. -- About to commit from the outer repo: stop. Do not commit data/; DB is canonical. -- About to approve without reading sources: read the cited files first. diff --git a/.claude/skills/wiki-authoring/SKILL.md b/.claude/skills/wiki-authoring/SKILL.md index 7dfb93b..f2ca8a8 100644 --- a/.claude/skills/wiki-authoring/SKILL.md +++ b/.claude/skills/wiki-authoring/SKILL.md @@ -9,7 +9,7 @@ Use this skill as the runtime contract for raw-to-wiki authoring. DB rows are th source of truth; Markdown under `data/` is generated export. Do not look for a workflow doc during execution; this skill is the complete operating surface. -> **Evidence-derived only.** This skill is for pages grounded in a `data/raw` source — including all LLM/cron authoring, which MUST cite real sources. For a **first-party human note** with no external source, use the `wiki-note` skill (`origin: authored`, `sources: []`); do not fake a citation here. +> **Evidence-derived only.** This skill is for pages grounded in a `data/raw` source — including all LLM/automated authoring, which MUST cite real sources. For a **first-party human note** with no external source, use the `wiki-note` skill (`origin: authored`, `sources: []`); do not fake a citation here. ## Rules @@ -18,8 +18,6 @@ workflow doc during execution; this skill is the complete operating surface. - Never use `data/raw/...` or absolute paths in `sources:`. - Use wikilinks only to existing file stems. `aliases:` do not satisfy lint. - Do not use the reserved `## User Feedback` heading in authored content. -- New `entity`, `concept`, `decision`, `improvement`, `checklist`, and `question` pages start with `review_status: not_processed`. -- If semantically editing an `approved` page, reset `review_status: not_processed`; keep `approved` only for typo/format-only edits. ## Bundled Templates @@ -71,7 +69,6 @@ Use stable ASCII slugs unless an existing local convention requires otherwise. ```yaml --- type: entity -review_status: not_processed created: "YYYY-MM-DD" updated: "YYYY-MM-DD" sources: @@ -88,7 +85,6 @@ Use `type: concept` or `type: question` as appropriate. `aliases:` is required f ```yaml --- type: decision -review_status: not_processed created: "YYYY-MM-DD" updated: "YYYY-MM-DD" sources: @@ -104,7 +100,6 @@ Filename must carry the decision date: `YYYY-MM-DD-.md`. ```yaml --- type: improvement -review_status: not_processed kind: improvement observed_at: "YYYY-MM-DD" domain: dx @@ -131,7 +126,6 @@ Enums: ```yaml --- type: checklist -review_status: not_processed created: "YYYY-MM-DD" updated: "YYYY-MM-DD" sources: @@ -163,7 +157,7 @@ tags: [] --- ``` -Weekly/monthly summaries use `period_start` and `period_end`; weekly also has `week: "YYYY-WNN"` and monthly `month: "YYYY-MM"`. Summaries do not use `review_status`. +Weekly/monthly summaries use `period_start` and `period_end`; weekly also has `week: "YYYY-WNN"` and monthly `month: "YYYY-MM"`. ## Promotion Criteria @@ -186,7 +180,7 @@ Writes go through kb-mcp tools (registered in the `opencode run` session). For a - `body_md` — the body **without** the `---` fence - `slug`, `type`, `export_path` -Optional: `title`, `category`, `review_status`, `origin`, `created_at`, `updated_at`, `source`. For a partial update to an existing page, call the kb-mcp `patch_page` tool with `slug` plus only the fields to change (`title`, `body_md`, `frontmatter`, `category`, `review_status`, `source`, `note`). +Optional: `title`, `category`, `origin`, `created_at`, `updated_at`, `source`. For a partial update to an existing page, call the kb-mcp `patch_page` tool with `slug` plus only the fields to change (`title`, `body_md`, `frontmatter`, `category`, `source`, `note`). The tool appends a revision and exports Markdown immediately. Do not manually regenerate `INDEX.md`; it is generated export from the DB. @@ -214,7 +208,6 @@ Call the kb-mcp `create_operation_log` tool with this `body_md`: - **sources**: raw/... - **created/updated**: wiki/... -- **review_status**: not_processed - **db_write**: page export succeeded ``` diff --git a/.claude/skills/wiki-authoring/reference/templates/checklist.md b/.claude/skills/wiki-authoring/reference/templates/checklist.md index 3c6e791..422ddb0 100644 --- a/.claude/skills/wiki-authoring/reference/templates/checklist.md +++ b/.claude/skills/wiki-authoring/reference/templates/checklist.md @@ -1,6 +1,5 @@ --- type: checklist -review_status: not_processed created: "" updated: "" sources: [] diff --git a/.claude/skills/wiki-authoring/reference/templates/concept.md b/.claude/skills/wiki-authoring/reference/templates/concept.md index 5823623..8f63a26 100644 --- a/.claude/skills/wiki-authoring/reference/templates/concept.md +++ b/.claude/skills/wiki-authoring/reference/templates/concept.md @@ -1,6 +1,5 @@ --- type: concept -review_status: not_processed created: "" updated: "" sources: [] diff --git a/.claude/skills/wiki-authoring/reference/templates/decision.md b/.claude/skills/wiki-authoring/reference/templates/decision.md index 59aa025..93e65eb 100644 --- a/.claude/skills/wiki-authoring/reference/templates/decision.md +++ b/.claude/skills/wiki-authoring/reference/templates/decision.md @@ -1,6 +1,5 @@ --- type: decision -review_status: not_processed created: "" updated: "" sources: [] diff --git a/.claude/skills/wiki-authoring/reference/templates/entity.md b/.claude/skills/wiki-authoring/reference/templates/entity.md index 1a07b84..f24afca 100644 --- a/.claude/skills/wiki-authoring/reference/templates/entity.md +++ b/.claude/skills/wiki-authoring/reference/templates/entity.md @@ -1,6 +1,5 @@ --- type: entity -review_status: not_processed created: "" updated: "" sources: [] diff --git a/.claude/skills/wiki-authoring/reference/templates/improvement.md b/.claude/skills/wiki-authoring/reference/templates/improvement.md index 24865d9..9b62b5b 100644 --- a/.claude/skills/wiki-authoring/reference/templates/improvement.md +++ b/.claude/skills/wiki-authoring/reference/templates/improvement.md @@ -1,6 +1,5 @@ --- type: improvement -review_status: not_processed kind: improvement observed_at: "" domain: "" diff --git a/.claude/skills/wiki-authoring/reference/templates/question.md b/.claude/skills/wiki-authoring/reference/templates/question.md index 9c30f3e..9dd1f2a 100644 --- a/.claude/skills/wiki-authoring/reference/templates/question.md +++ b/.claude/skills/wiki-authoring/reference/templates/question.md @@ -1,6 +1,5 @@ --- type: question -review_status: not_processed created: "" updated: "" sources: [] diff --git a/.claude/skills/wiki-authoring/reference/templates/summaries/monthly.md b/.claude/skills/wiki-authoring/reference/templates/summaries/monthly.md index 707f806..e3912ae 100644 --- a/.claude/skills/wiki-authoring/reference/templates/summaries/monthly.md +++ b/.claude/skills/wiki-authoring/reference/templates/summaries/monthly.md @@ -10,40 +10,26 @@ sources: [] tags: [] --- -# Monthly Report — {{YYYY-MM}} +# Monthly Summary — {{YYYY-MM}} ## Summary -- Total sessions: N / active days: N (empty days: N) -- Total tokens: N / average daily tokens: N -- Average cache hit rate: N% -- Average TODO completion rate: N% +- Period: {{YYYY-MM}} +- Takeaway: -## Weekly Trends +## Highlights -| Week | Sessions | Tokens | Cache Hit Rate | TODO Completion | Notes | -|------|------|------|------------|------------|------| -| W01 | | | | | | -| W02 | | | | | | -| W03 | | | | | | -| W04 | | | | | | + -## Monthly Insights +## Signals / Patterns -### What Went Well + - +## Open Items -### Problem Patterns - - - -### Action Items - - + -## Observations +## Next Period - - + diff --git a/.claude/skills/wiki-authoring/reference/templates/summaries/weekly.md b/.claude/skills/wiki-authoring/reference/templates/summaries/weekly.md index c9f3235..890e71a 100644 --- a/.claude/skills/wiki-authoring/reference/templates/summaries/weekly.md +++ b/.claude/skills/wiki-authoring/reference/templates/summaries/weekly.md @@ -15,51 +15,19 @@ tags: [] ## Summary - Period: {{YYYY-MM-DD}} to {{YYYY-MM-DD}} -- Source reports reviewed: N -- Active systems: OpenCode / Hermes / Claude Code / other -- Main pattern: +- Takeaway: -## Daily Coverage +## Highlights -| Date | Memory | OpenCode | Hermes | Claude Code | Notes | -|---|---|---|---|---|---| -| YYYY-MM-DD | present/missing | present/missing/N/A | present/missing/N/A | present/missing/N/A | | + -## Signals +## Signals / Patterns -### Work Patterns - - - -### Quality / Reliability - - - -### Cost / Efficiency - - - -## Promotions - -### Concepts - - - -### Improvements - - - -### Checklists - - - -### Decisions - - + ## Open Items - + ## Next Week diff --git a/.claude/skills/wiki-note/SKILL.md b/.claude/skills/wiki-note/SKILL.md index 9da2297..99f93e6 100644 --- a/.claude/skills/wiki-note/SKILL.md +++ b/.claude/skills/wiki-note/SKILL.md @@ -1,6 +1,6 @@ --- name: wiki-note -description: Use when you (a human) want to capture a first-party note, insight, or decision directly into the KnowledgeBase wiki — original thinking that does NOT come from a data/raw source. Works from any repo. Not for LLM/cron authoring and not for evidence-derived pages. +description: Use when you (a human) want to capture a first-party note, insight, or decision directly into the KnowledgeBase wiki — original thinking that does NOT come from a data/raw source. Works from any repo. Not for LLM/automated authoring and not for evidence-derived pages. --- # wiki-note @@ -22,12 +22,12 @@ external `data/raw` evidence behind it — into the DB-backed KnowledgeBase wiki Normal wiki pages must cite `sources:`. That rule exists to **ground LLM-generated content in evidence**. A note you author yourself has different, valid provenance: *you wrote it*. This skill is the sanctioned exception — pages are marked -`origin: authored` with `sources: []` and may be born `approved`. +`origin: authored` with `sources: []`. **Core split — never blur it:** - **You, a human, thinking** → `origin: authored`, no source. **This skill.** -- **An LLM / cron summarizing captured evidence** → MUST cite real `data/raw` sources. +- **An LLM / automated process summarizing captured evidence** → MUST cite real `data/raw` sources. Use `wiki-authoring`. Automation must NEVER set `origin: authored`. ## When to Use @@ -45,7 +45,7 @@ content in evidence**. A note you author yourself has different, valid provenanc Supported types here: **`concept`, `decision`, `entity`, `checklist`, `improvement`** — one bundled template per type in `reference/templates/`, each pre-set with the -first-party frontmatter (`origin: authored`, `review_status: approved`, `sources: []`). +first-party frontmatter (`origin: authored`, `sources: []`). ## Step 1 — Resolve the KB root (works from any repo) @@ -90,7 +90,6 @@ keep the rest: --- type: concept # matches the template you copied origin: authored # first-party authorship — keep this; it's why sources can be empty -review_status: approved # born approved (you author and approve); not_processed also valid created: "YYYY-MM-DD" # today updated: "YYYY-MM-DD" sources: [] # empty is allowed BECAUSE origin: authored @@ -125,11 +124,6 @@ object/dict (not YAML text) and the body **without** the `---` fence: `tags`, and any type-specific fields) - `body_md` — the page body with no frontmatter fence - `origin` — `"authored"` (first-party authorship; keep it — it's why `sources` can be empty) -- `review_status` — `"approved"` if born approved, else `"not_processed"` - -If you author the page born approved, you may instead pass `review_status="not_processed"` -and then call the kb-mcp `approve_page` tool with the same `slug` — keep whichever -lifecycle you chose in Step 3. On success the tool returns the page plus `"export": {"status": "success", "written": N}`, confirming Markdown was exported immediately. On failure it returns @@ -147,17 +141,15 @@ Write through the kb-mcp `create_operation_log` tool: - **db_write**: page export succeeded ``` -## Commit & approval +## Commit - Do not commit `data/`; it is generated export. -- `wiki-promote` (cron) will not touch an already-`approved` page, so first-party notes - bypass the promotion ladder by design. ## Common Mistakes | Mistake | Fix | |---|---| -| `origin: authored` on an LLM/cron page | Automation must cite real sources — use `wiki-authoring` | +| `origin: authored` on an LLM/automated page | Automation must cite real sources — use `wiki-authoring` | | `sources: []` but you actually have a source | Cite it via `wiki-authoring`; don't mark authored | | `improvement` enum left blank/invalid | Fill `kind`/`domain`/`severity`/`issue_status` per template comments | | links scattered inline instead of `## Related` | Collect them as `- [[stem]] — why` in `## Related` | diff --git a/.claude/skills/wiki-note/reference/templates/checklist.md b/.claude/skills/wiki-note/reference/templates/checklist.md index 40bed0d..53a2121 100644 --- a/.claude/skills/wiki-note/reference/templates/checklist.md +++ b/.claude/skills/wiki-note/reference/templates/checklist.md @@ -1,7 +1,6 @@ --- type: checklist origin: authored -review_status: approved created: "YYYY-MM-DD" updated: "YYYY-MM-DD" sources: [] diff --git a/.claude/skills/wiki-note/reference/templates/concept.md b/.claude/skills/wiki-note/reference/templates/concept.md index 3f4383f..43b4cee 100644 --- a/.claude/skills/wiki-note/reference/templates/concept.md +++ b/.claude/skills/wiki-note/reference/templates/concept.md @@ -1,7 +1,6 @@ --- type: concept origin: authored -review_status: approved created: "YYYY-MM-DD" updated: "YYYY-MM-DD" sources: [] diff --git a/.claude/skills/wiki-note/reference/templates/decision.md b/.claude/skills/wiki-note/reference/templates/decision.md index 4fdcca2..8b480d4 100644 --- a/.claude/skills/wiki-note/reference/templates/decision.md +++ b/.claude/skills/wiki-note/reference/templates/decision.md @@ -1,7 +1,6 @@ --- type: decision origin: authored -review_status: approved created: "YYYY-MM-DD" updated: "YYYY-MM-DD" sources: [] diff --git a/.claude/skills/wiki-note/reference/templates/entity.md b/.claude/skills/wiki-note/reference/templates/entity.md index 0ad19c7..5c01074 100644 --- a/.claude/skills/wiki-note/reference/templates/entity.md +++ b/.claude/skills/wiki-note/reference/templates/entity.md @@ -1,7 +1,6 @@ --- type: entity origin: authored -review_status: approved created: "YYYY-MM-DD" updated: "YYYY-MM-DD" sources: [] diff --git a/.claude/skills/wiki-note/reference/templates/improvement.md b/.claude/skills/wiki-note/reference/templates/improvement.md index 165ded9..97ea251 100644 --- a/.claude/skills/wiki-note/reference/templates/improvement.md +++ b/.claude/skills/wiki-note/reference/templates/improvement.md @@ -1,7 +1,6 @@ --- type: improvement origin: authored -review_status: approved kind: improvement # improvement | issue | proposal observed_at: "YYYY-MM-DD" domain: dx # cost | correctness | perf | dx | security diff --git a/CHANGELOG.md b/CHANGELOG.md index 6f90f6b..4e92f8f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -17,6 +17,10 @@ Keep entries concise and user/operator-facing. Avoid tool traces, lint output, h ## Unreleased +### Removed + +- Removed the cron pipeline, usage/daily reports (metrics), and the wiki promote/review_status lifecycle — including related CLIs, MCP tools, skills, docs, and a migration dropping the review_status column and cron_runs/metrics tables. + ### Added - Added the MCP read tools `query_sql` and `get_schema` (`src/kb/mcp/tools_read.py`), registered on the `kb-mcp` server alongside the write tools. `query_sql` is **strictly read-only** with three layers of defense: a statement guard (rejects multi-statement input and any body not starting with `SELECT`/`WITH`), a `SET TRANSACTION READ ONLY` Postgres transaction issued as the first statement of the autobegun transaction (the real control — rejects every write, including data-modifying CTEs that pass the prefix guard), and a `fetchmany(limit)` row cap (default 100, `truncated` flag). DB errors (e.g. a write blocked by the read-only transaction) surface as `{"error", "code": "query_error"}` dicts rather than exceptions, so an agent loop continues. `get_schema` introspects the static ORM metadata (no DB connection) and returns every table's columns plus example queries; agents call it before `query_sql`. The 12-write-tool registration smoke test was relaxed from exact-set to subset so it stays correct as tools are added to the shared `mcp` instance. Read-only is verified against real Postgres in `test/test_mcp_read_tools.py` (10 tests: SELECT, real-data read, prefix-guard rejection of INSERT/UPDATE/DELETE/DROP, multi-statement rejection, the writing-CTE transaction-level block with a no-row assertion, row cap, and schema shape). @@ -34,6 +38,10 @@ Keep entries concise and user/operator-facing. Avoid tool traces, lint output, h - Added optional global digest guidance under `knowledgebase-initialize/reference/optional-global-digest.md`. - Added Phase 1 backend for Improvement → Kanban Dispatch: `GET /api/kanban/boards` (with a 30s in-memory TTL cache, invalidated on successful dispatch) and `POST /api/pages/{stem}/send-to-kanban` for sending a `pending_for_approve` improvement page to a Hermes kanban board. Helpers live in `src/kb_mcp/cli/wiki_review/_kanban.py` (`list_boards`, `create_card`, `archive_card`, `append_dispatch`); the route layer translates them per the spec's error taxonomy and writes the dispatch entry back onto the page's `kanban_dispatches` frontmatter list. Unit and route tests added under `test/test_kanban_helpers.py` and `test/test_kanban_route.py`. +### Fixed + +- `upsert_page` and `create_handoff` now strip a stray leading `data/` from `export_path` before storing it (`export_path` is relative to `KB_DATA_DIR`). Callers that passed the path including the `data/` prefix were producing rows that exported to `data/data/wiki/...` and `data/data/handoffs/...` because export computes `data_dir / export_path`. Backfilled the 23 affected rows (10 pages, 13 handoffs) and removed the stale `data/data/` export tree. + ### Changed (API→MCP refactor, 2026-06-12) - Replaced the FastAPI HTTP write API (`kb-web`) with the `kb-mcp` FastMCP server (streamable-http, 127.0.0.1:8765, no auth) plus an in-process `src/kb/service/` layer that is the single write path (lint→DB→Markdown export). Writes are exposed as 12 MCP tools (`create_raw_source`, `upsert_page`, `patch_page`, `promote_page`, `approve_page`, `reject_page`, `ttl_sweep_pages`, `create_handoff`, `create_operation_log`, `create_cron_run`, `upsert_metrics`, `export_markdown`); reads via the read-only `query_sql` tool + `get_schema`, or `psql`. The deterministic cron CLIs (daily reports, `kb-db-ttl-sweep`, `kb-submit-cron-run`, ingest-papers) now call the service layer in-process and no longer require a running server. Env: `KB_WEB_HOST/PORT` → `KB_MCP_HOST/PORT`; run the server with `uv run kb-mcp --transport streamable-http`. diff --git a/CLAUDE.md b/CLAUDE.md index c261ff1..49d5042 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -14,21 +14,17 @@ KnowledgeBase is a memory-workflow system (v0) that captures knowledge from mult KnowledgeBase/ # Outer repo: code, lint, templates, docs ├── src/kb/ # CLI tools + FastMCP server + service layer │ ├── cli/ -│ │ ├── lint.py # kb-lint command (wiki + handoff validation) -│ │ ├── db_ttl_sweep.py # kb-db-ttl-sweep command -│ │ ├── opencode_daily_report.py # kb-opencode-daily-report command -│ │ ├── hermes_daily_report.py # kb-hermes-daily-report command -│ │ └── claude_code_daily_report.py # kb-claude-code-daily-report command +│ │ └── lint.py # kb-lint command (wiki + handoff validation) │ ├── service/ # DB-canonical write path (lint → DB → export) │ │ ├── pages.py # Wiki page operations │ │ ├── sources.py # Raw source ingest │ │ ├── handoffs.py # Handoff document operations -│ │ ├── ops.py # Cron runs, metrics, operation logs, export +│ │ ├── ops.py # Operation logs, export │ │ ├── export.py # Markdown export │ │ └── session.py # SQLAlchemy session helpers │ └── mcp/ # FastMCP server (kb-mcp) │ ├── server.py # FastMCP app + lifespan -│ ├── tools_write.py # 12 write tools +│ ├── tools_write.py # 6 write tools │ ├── tools_read.py # query_sql + get_schema read tools │ └── validators.py # Input validators ├── src/CLAUDE.md # CLAUDE.md file for src/ @@ -76,14 +72,14 @@ data/ # Generated Markdown export tree (KB_DATA_DIR) - Never modify files in `data/raw/`. They are immutable after creation. Use `kb-lint` to enforce (via the `kb.service` layer): git-status must not show modifications, `captured_at` must be ≤ file mtime (60s tolerance), and required raw frontmatter fields (`source_url`, `type`, `captured_at`, `contributor`) are always validated. - `data/wiki/` pages must always list their `sources:` in frontmatter. -- Keep `data/log.md` updated for `data/` repo operations only: raw ingest, wiki page creation/update, handoff creation/update, promotion/rejection, cron/report outputs, and data lint results. +- Keep `data/log.md` updated for `data/` repo operations only: raw ingest, wiki page creation/update, handoff creation/update, and data lint results. - Keep outer repo changes in `CHANGELOG.md`: source code, scripts, docs, skills, templates, config, schemas, and workflow contract changes. - Do not write outer repo implementation details to `data/log.md`; reference the data artefact or handoff only if the operation created one. - Use uv for package management ## Change Records -- `CHANGELOG.md` is for the outer repo. Update it when a change affects maintainers/operators or changes runtime behavior, CLI behavior, cron wrappers, skills, docs contracts, schemas, templates, or setup instructions. +- `CHANGELOG.md` is for the outer repo. Update it when a change affects maintainers/operators or changes runtime behavior, CLI behavior, skills, docs contracts, schemas, templates, or setup instructions. - Do not write outer repo implementation details to `data/log.md`; reference the data artefact or handoff only if the operation created one. - If outer-repo work produces or adjusts `data/` artefacts for verification, leave those `data/` changes uncommitted unless the user specifically asks to handle them. @@ -101,65 +97,17 @@ Never commit `data/` contents to the outer repository. Project skills live under `.claude/skills/`, auto-load by description match, and are the source of truth for workflow behavior. -- `knowledgebase-initialize` — bring up Postgres (compose `db`), run migrations, verify tooling, and propose cron jobs for approval. -- `wiki-approval` — promote, approve, reject, or TTL-sweep wiki pages through the `review_status` lifecycle; runtime contract for wiki-promote cron. +- `knowledgebase-initialize` — bring up Postgres (compose `db`), run migrations, and verify tooling. - `wiki-authoring` — create or update source-backed `data/wiki/` pages with valid schemas, paths, wikilinks, templates, and lint order. -- `usage-report-setup` — select and wire source-specific OpenCode/Hermes/Claude Code usage report jobs. - `handoff-document` — write or update handoff documents under `data/handoffs/` with lintable frontmatter, filename grammar, and canonical body sections. -- `memory-report` — daily, weekly, or monthly memory workflow (period dispatch inside the skill). Imports `wiki-authoring` for page edits and `handoff-document` for run handoffs. -- `cron-wrapup` — nightly KB cron wrap-up. Aggregates the previous day's usage reports, memory page, wiki-promote/TTL outcomes, and per-run cron evidence from `data/raw/ops/cron/{YYYY}/{MM}/{date}_kb-*.log` into a single Slack-digest-stable `wiki/summaries/.../{date}-cron-wrapup.md` plus run handoff. Runtime contract for the 05:00 cron job. - -## Cron Jobs - -The cron job runs KB cron jobs directly. The script delegates LLM work to `opencode run` where reasoning is needed, or calls deterministic `uv run` CLIs for pure data plumbing. - -### Execution patterns - -| Pattern | Used by | Mechanism | -|---------|---------|-----------| -| **LLM-Driven** | memory-daily/weekly/monthly, wiki-promote, cron-wrapup | skill-driven + opencode run `opencode run --model anthropic/claude-sonnet-4-6` | -| **Deterministic** (no LLM) | opencode/hermes/claude-code daily reports, wiki-ttl-sweep, ingest-papers | `uv run kb-*-daily-report --date --lint`, `uv run kb-db-ttl-sweep --days 7`, `uv run python scripts/ingest-daily-papers.py` | - -The deterministic CLIs call the `kb.service` layer **in-process** — no server needed. The LLM-driven cron jobs connect to the `kb-mcp` http daemon (compose `mcp` service, `127.0.0.1:8765`); see `knowledgebase-initialize` for registering `kb-mcp` in each agent runtime. - -### Pipeline schedule (KST) - -``` -00:30 kb-db-ttl-sweep deterministic -03:10 kb-opencode-daily-report deterministic -03:15 kb-hermes-daily-report deterministic -03:20 kb-claude-code-daily-report deterministic -03:30 kb-memory-daily LLM-Driven -04:00 kb-wiki-promote LLM-Driven -04:15 kb-memory-weekly (Mon) LLM-Driven -04:45 kb-memory-monthly (1st) LLM-Driven -05:00 kb-cron-wrapup LLM-Driven -10:05 kb-ingest-daily-papers deterministic -``` - -`morning-slack-digest` (09:00, Hermes agent) reads the wrapup artefact and delivers to Slack — it is **not** a KB cron job and runs with the agent. - -### Proxy scripts - -Cron jobs reference short script names (e.g. `kb-memory-daily.sh`) that resolve to `~/.hermes/scripts/`. Each is a thin proxy that `cd`s to KB root and `exec`s the real script under `scripts/cron/`. When adding a new cron script, create both the real script and the `~/.hermes/scripts/` proxy. - -### Runtime contracts - -OpenCode-powered jobs load their behavior from `.claude/skills//SKILL.md`: -- memory-* → `memory-report` (+ `wiki-authoring`, `handoff-document`) -- wiki-promote → `wiki-approval` (+ `wiki-authoring` if fixes needed) -- cron-wrapup → `cron-wrapup` (+ `handoff-document`) - -All run logs go to `data/raw/ops/cron/{YYYY}/{MM}/{date}_{job}.log`. ## Documentation - [Documentation Index](docs/README.md) — design/reference document map - [Architecture](docs/architecture.md) — repository layout and memory layers -- [Workflows](docs/workflows.md) — at-a-glance diagram map (nightly pipeline, review lifecycle); skills own the execution detail - [Frontmatter Conventions](docs/reference/frontmatter.md) — schema reference; `wiki-authoring` and `handoff-document` carry runtime rules - [Wiki Categories](docs/reference/wiki-categories.md) — category reference; use `wiki-authoring` at runtime -- [Commands](docs/reference/commands.md) — kb-lint, kb-db-ttl-sweep, kb-submit-cron-run, daily report CLIs, kb-mcp +- [Commands](docs/reference/commands.md) — kb-lint, kb-mcp ## Linting diff --git a/DESIGN.md b/DESIGN.md deleted file mode 100644 index 170866b..0000000 --- a/DESIGN.md +++ /dev/null @@ -1,129 +0,0 @@ - ---- -name: KnowledgeBase Review Console -description: Calm, precise review-and-insights surface for an LLM-fed wiki, handoffs, and rejection patterns. ---- - -# Design System: KnowledgeBase Review Console - -## 1. Overview - -**Creative North Star: "The Workshop Bench"** - -This is a single-operator review console for a personal LLM-fed wiki. The metaphor is a well-lit workshop bench: tools laid out exactly where the hand expects them, surfaces clean, nothing decorative. Wiki reading happens elsewhere (Antigravity / Obsidian); this surface exists strictly to **decide** — approve, reject, inspect a pattern — and to surface the *shape* of those decisions over time. - -The aesthetic family is the restrained-operations cluster: Linear, Plausible, Stripe Dashboard, Raycast. Tinted slate-blue neutrals carry almost the whole UI. One accent appears on ≤5% of any screen, and only for live state (focus, current selection, the row a destructive action is about to land on). Density is deliberate; whitespace is structural, not decorative. Status is conveyed by position, glyph, and label — never by color alone. - -The system explicitly rejects: the four-up KPI strip, the giant-number-with-trend-arrow card, gradient accents, "Welcome back!" greetings, decorative sparklines, soft pastels, rounded-everything, illustration-heavy empty states, and the Filament / Django-Admin / Retool look. If a screen could be confused with a generic SaaS dashboard at a glance, it has failed. - -**Key Characteristics:** -- Restrained color strategy: tinted slate-blue neutrals + a single signal accent ≤5% -- Single technical/geometric sans across the entire UI -- Responsive motion language: feedback + short transitions; no choreography -- Operator-grade density; whitespace is structural, not breathing room -- Lint-grade honesty: state is truthful, never decorative - -## 2. Colors - -A near-monochrome cool-slate palette. Neutrals do almost all the work. The accent earns its place by appearing rarely. - -### Primary -- **Signal** *(`[to be resolved during implementation]`)*: the one accent. Used only for live state — focus rings, the currently-selected review row, the active filter chip, the destructive-action confirmation. Target use: ≤5% of any screen. Hue direction: a single saturated step away from the neutral ramp's hue, leaning slightly cool. Not violet (Linear's lane), not red (Raycast's lane); pick a neighboring tone during implementation. - -### Neutral -The whole surface and text system lives here. All neutrals are **tinted toward a single cool-slate hue** (small chroma, ~0.005–0.012); never pure greys, never `#000` or `#fff`. - -- **Bench** *(deepest surface)*: the page background. Slightly warmer than panel surfaces so panels feel raised without shadow. -- **Panel** *(primary surface)*: cards, the review pane, the dashboard panels. -- **Rail** *(secondary surface)*: nav rail, secondary toolbars, condensed table rows. -- **Hairline** *(divider)*: the project's primary delineator. 1px, low-contrast, used liberally instead of shadows. -- **Ink** *(body text)*: ≥4.5:1 against Panel; the main reading color. -- **Ink-Muted** *(secondary text)*: timestamps, paths, frontmatter keys, byline-style metadata. ≥4.5:1 in the contexts it appears. -- **Ink-Dim** *(tertiary text)*: shortcut hints, "n items" counts, disabled labels. ≥3:1 minimum; never used for primary content. - -*Hex / OKLCH values to be resolved during implementation. Generate the ramp in OKLCH around a single cool-slate hue.* - -### Status (semantic, not decorative) -- **Accepted**: not a color. A solid check glyph and a position change (the row leaves the queue). If a tint is needed, a quiet positive-leaning neutral; never a saturated green. -- **Rejected**: a small ring or hollow glyph, again not pure red. Pair every rejected pill with its TTL countdown so red never means "alarm" — it means "scheduled deletion". -- **Pending / Awaiting**: no color at all. Position in the queue is the status. -- **Expiring soon (TTL ≤ 24h)**: the Signal accent, used here as a deadline cue, not decoration. - -### Named Rules - -**The One Accent Rule.** Across any single screen, Signal occupies ≤5% of the pixels. If it covers more, something has been treated as decoration. Remove it. - -**The Tinted Neutrals Rule.** Every neutral is tinted toward the cool-slate hue. `#000`, `#fff`, and untinted greys are forbidden. The tint is the brand. - -**The Color-Plus-Glyph Rule.** No status — accepted, rejected, pending, expiring — is communicated by color alone. Color is paired with a glyph, a label, or position. Color-blind users see the same state as anyone else. - -## 3. Typography - -**Display Font:** a single technical/geometric sans across the entire UI *(family to be chosen at implementation — Inter, Geist, IBM Plex Sans, and Söhne are all in-register; pick one with a mono companion or excellent tabular figures)*. -**Body Font:** same family. -**Mono companion:** *(to be chosen at implementation; should pair with the display/body family)*. Used for paths, slugs, hashes, frontmatter keys, lint codes, TTL countdowns. - -**Character:** One typeface across the UI gives the console a single, neutral voice. The mono companion is used sparingly and only for content that is *literally a string of code or path* — never for emphasis. Tabular figures are mandatory for any column of numbers (counts, percentages, TTLs). - -### Hierarchy -- **Display** *(weight 500, ~28–32px, line-height 1.15)*: page headings (e.g. "Review Queue", "Dashboard"). Single instance per screen. -- **Headline** *(weight 500, ~20px, line-height 1.25)*: section headings inside a screen. -- **Title** *(weight 500, ~15px, line-height 1.3)*: card / row titles (wiki page title in the review queue, handoff doc title, dashboard panel header). -- **Body** *(weight 400, 14–15px, line-height 1.5, max width 65–75ch)*: descriptive text, page summaries, dashboard prose. Body never goes below 14px effective. -- **Body-Mono** *(weight 400, ~13–14px)*: paths, slugs, lint codes, TTL strings. Tabular figures on. -- **Label** *(weight 500, 11–12px, letter-spacing +0.04em, uppercase)*: small metadata labels — "type", "source", "captured\_at". Used sparingly; never for body content. - -### Named Rules - -**The One Family Rule.** A single typographic family carries the entire UI. Pairing a display serif with a body sans is in-register for editorial brand work, not for this operations console; it is prohibited here. - -**The Tabular Figures Rule.** Every column of numbers — counts, percentages, TTL countdowns, IDs — uses tabular figures. Numbers must align vertically. Proportional figures in a stat column are a defect. - -**The Mono-for-Strings Rule.** Mono is reserved for content that *is* a literal string: paths, slugs, hashes, lint codes, frontmatter keys. Mono used for "technical feel" or emphasis is prohibited. - -## 4. Elevation - -Flat by default. Depth is conveyed by **tonal layering** between Bench (page), Panel (primary surface), and Rail (secondary surface) — never by shadow on resting elements. Hairline dividers do almost all the structural work that shadows would do in a more decorative system. - -The only acceptable shadows are **state shadows**, applied transiently: -- A subtle focus halo on the currently-focused interactive element (paired with the Signal accent ring). -- A faint lift on a row when it's the target of a hover-pending destructive action. -- Modals, when they exist, sit on a low-opacity scrim, not a glassy blur. Glassmorphism is forbidden anywhere in this system. - -### Named Rules - -**The Flat-By-Default Rule.** Surfaces are flat at rest. Shadow appears only as a response to state (focus, destructive-target hover). Persistent decorative shadows on cards, panels, or rails are prohibited. - -**The Hairline-Not-Shadow Rule.** Where another design would reach for a soft shadow to separate surfaces, this system reaches for a 1px hairline. Hairlines are the project's primary tool for structure. - -## 5. Components - -*Component specification deferred until implementation. The console has no components yet; the next pass of `/impeccable document` will run in scan mode and capture the real primitives — review row, dashboard panel, handoff card, TTL countdown, filter chip, command palette — directly from the codebase.* - -## 6. Do's and Don'ts - -### Do: -- **Do** tint every neutral toward the cool-slate hue (chroma ~0.005–0.012). Pure greys, `#000`, and `#fff` are forbidden. -- **Do** keep the Signal accent on ≤5% of any screen, reserved for live state and the next destructive action. -- **Do** pair every status (accepted / rejected / pending / expiring) with a glyph or label, never color alone. -- **Do** use tabular figures in every column of numbers — counts, percentages, TTL strings, IDs. -- **Do** prefer 1px hairlines over shadows to separate surfaces. -- **Do** keep body text ≥14px effective and body line length within 65–75ch. -- **Do** honor `prefers-reduced-motion: reduce`: remove transitions, fall back to opacity / instant state changes only. -- **Do** put real values on screen — real counts, real timestamps, real lint codes. Fake skeletons or rounded-down numbers that hide errors are forbidden. - -### Don't: -- **Don't** build the four-up KPI strip (Total / Active / Pending / +12% MoM). It is the SaaS-dashboard cliché the PRODUCT.md explicitly rejects. -- **Don't** use the giant-number-with-trend-arrow card. A correctly chosen number in its own context beats a hero metric every time. -- **Don't** use decorative sparklines. Charts must explain a rejection or acceptance pattern; if a chart doesn't answer a question, delete it. -- **Don't** apply gradients anywhere — on text (`background-clip: text` is banned), on backgrounds, on accents. -- **Don't** use side-stripe borders (`border-left` or `border-right` > 1px as a colored accent). On rows, status pills, callouts: forbidden. -- **Don't** apply glassmorphism — backdrop-blur as decoration, "glassy" panels, frosted nav. Forbidden anywhere in this system. -- **Don't** wrap content in identical card grids. Cards are the lazy answer; nested cards are always wrong. -- **Don't** write "Welcome back!", "Awesome!", or any encouragement copy. The voice is short, factual, second-person only when necessary; never marketing. -- **Don't** show illustration-heavy empty states. Empty states are short factual sentences (e.g. "No pages awaiting review. Last cron completed at 03:17.") with the next action, if any, in plain text. -- **Don't** drift toward Filament / Django-Admin / Retool aesthetics — uniform tables, generic action menus, framework-default forms. The PRODUCT.md anti-reference is the line. -- **Don't** drift toward consumer-warm notebook aesthetics — soft pastels, rounded-everything, friendly serifs, mascots. Wrong register for this tool. -- **Don't** add Notion/WYSIWYG editing affordances — slash menus, drag handles, block editors. This is a review surface, not an editing surface. -- **Don't** animate CSS layout properties. Animate transform and opacity only. No bounce, no elastic; ease out with quart/quint/expo curves. -- **Don't** rely on color alone to communicate any status, ever. diff --git a/PRODUCT.md b/PRODUCT.md deleted file mode 100644 index 8c92b29..0000000 --- a/PRODUCT.md +++ /dev/null @@ -1,76 +0,0 @@ -# Product - -## Register - -product - -## Users - -A single owner-operator (the repo author) running KnowledgeBase as a personal LLM-fed wiki and operations system. The user already lives in the CLI (`kb-lint-wiki`, `kb-wiki-review`, daily report scripts) and reads finished wiki pages inside Antigravity or Obsidian — those tools own the reading experience. - -The user comes to this frontend in short, focused sessions, usually after the daily ingest/fill cron has produced new candidate wiki pages, rejections, and handoff updates. The job is to: - -1. Review pending AI-written wiki pages and approve, reject, or send back for revision. -2. Track handoff documents in flight — what's open, what's been promoted, what's stalled. -3. Read the dashboard for rejection and acceptance patterns: which wiki types get rejected most, which sources produce noisy raw data, which rejected pages are about to be deleted by the TTL sweep. - -Context of use: desktop browser on a local network, alongside an editor and terminal. Sessions are minutes long, not hours. No phone use, no on-the-go review. - -## Product Purpose - -A precise, calm review-and-insights surface for an LLM-fed wiki the user has already built around `kb-wiki-review`, handoff documents, and lint. The frontend exists to: - -- Replace the parts of `kb-wiki-review` that are slow or blind in a terminal: side-by-side review of pending pages with their raw sources, batch decisions, and the "what's about to expire" view. -- Make handoff status legible at a glance instead of grepping `data/handoffs/`. -- Surface the patterns in accept / reject decisions that no individual page review can show: rejection rate by wiki type, by source, by author-agent, by time-of-week; TTL countdown for rejected pages; what's about to be permanently deleted. - -Out of scope for now: wiki page reading (Antigravity/Obsidian do this), editing wiki bodies, raw source browsing, content authoring, multi-user collaboration, mobile. - -Success looks like: a one-screen weekly read tells the user *"the AI is over-producing concept pages from conversation sources and 60% are getting rejected for thin sourcing"* — a finding the CLI cannot produce, and that changes upstream prompts. - -## Brand Personality - -Calm, precise, expert-tool. Three words: **considered, accurate, quiet**. - -The reference cluster is Linear, Raycast, Stripe Dashboard, Plausible — interfaces that respect the user's existing expertise, do not explain themselves, and earn trust through restraint and accuracy rather than personality. The voice in the UI is short, factual, second-person only when necessary. No mascots, no encouragement, no exclamation marks, no "Awesome!" empty states. - -Emotionally the frontend should feel like a well-organized workshop bench: tools laid out, surfaces clean, nothing decorative. The user should never feel marketed to by their own tool. - -## Anti-references - -What this must explicitly NOT look or feel like: - -- **Heavy admin frameworks** — Django Admin, Filament, Retool, generic CRUD UIs. Tell-tale signs: dense uniform tables, generic action menus, framework-default forms, no opinion about what matters on a screen. KnowledgeBase has strong opinions about wiki types and handoff lifecycle; the UI must reflect that, not flatten it into rows. -- **Generic SaaS dashboard clichés** — big hero metric cards (Total / Active / +12% MoM), three identical KPI tiles in a row, gradient accents, "Welcome back, user!" greetings, illustration-heavy empty states, Vercel/Tailwind-template hero blocks. Specifically banned shapes: the four-up KPI strip, the giant-number-with-trend-arrow card, decorative sparklines. -- **Notion / WYSIWYG editor energy** — slash commands, drag handles, block menus. This is a *review* surface, not an editing surface. -- **Consumer-warm notebook aesthetics** — soft pastels, rounded-everything, friendly illustrations, cozy serif headings paired with playful sans body. Wrong register for an operations tool. - -## Design Principles - -1. **Review, don't read.** Wiki reading lives in Antigravity and Obsidian. Every screen here is for *deciding* (approve / reject / inspect a pattern), not for consuming long-form content. If a screen starts looking like a reader, it has drifted out of scope. - -2. **Patterns over piles.** A list of rejected pages is the floor, not the ceiling. The dashboard's real job is to make the *shape* of decisions legible — rejection rate by type, by source, by time — so the user can fix upstream prompts and ingestion, not just clear a queue. - -3. **Precision over polish.** A correctly chosen number, a sharp diff, a real TTL countdown are worth more than animated charts and gradient cards. Borrow the Linear/Stripe rule: if you can't justify a visual element to the operator-user, remove it. - -4. **One user, no shims.** This is a single-operator tool. No role badges, no "your team" copy, no shared-cursor presence, no permission UI. Designing for a hypothetical future team would inject SaaS clichés the user explicitly rejected. - -5. **Lint-grade honesty.** The wiki is enforced by lint; the UI must match that ethos. No fake totals, no rounded-down counts that hide errors, no skeleton placeholders that imply data exists when it doesn't, no "Everything looks great!" empty states when the cron actually failed at 03:17. - -## Implementation Stack - -- **Frontend:** Vite + React + TypeScript. Single-page application loaded locally; no SSR, no SEO. -- **Backend:** small FastAPI service inside the existing `src/kb/` Python tree. Read-mostly endpoints over the markdown content of `data/wiki/`, `data/handoffs/`, `data/log.md`, and over `kb-wiki-review` actions for approve / reject / TTL queries. -- **Data:** the markdown files in `data/` remain the source of truth. The API reads them with frontmatter parsing on demand; no separate database. Approve / reject mutations go through the existing `kb-wiki-review` machinery, not a parallel write path. -- **Runtime posture:** local-only, single user, single machine. Same privacy stance as the rest of the repo: never deployed, never exposed beyond localhost. -- **Why this stack over HTMX:** the review console's keyboard-first interactions (command palette, multi-select queue actions, optimistic approve / reject with TTL hint motion) and tabular data with live filters are native in React and possible-but-grafted in HTMX. The two-runtime cost (uv + npm) is accepted in exchange for that fit. - -This is the only place this decision is recorded. No separate ADR. - -## Accessibility & Inclusion - -- WCAG 2.1 AA as the floor: 4.5:1 contrast on body text, 3:1 on UI components and large text, visible focus rings on every interactive element, semantic landmarks, labelled form controls, error messages tied to inputs with `aria-describedby`. -- `prefers-reduced-motion: reduce` honored throughout. Default motion is already minimal (no parallax, no hero animations); the reduced variant removes the few remaining transitions and uses opacity/instant state changes only. -- Keyboard navigable across all primary flows (review queue, handoff list, dashboard filters). Standard tab order, no keyboard traps. -- No reliance on color alone to convey status — accepted / rejected / pending / expiring also carry a glyph, label, or position. -- Body text never below 14px effective; line length capped at 65–75ch. diff --git a/README.md b/README.md index ce513ad..bc541a7 100644 --- a/README.md +++ b/README.md @@ -16,9 +16,7 @@ KnowledgeBase/ │ └── ingest-github.sh GitHub source collection ├── .claude/skills/ Runtime workflow contracts + bundled templates │ ├── wiki-authoring/ Wiki page templates and authoring rules -│ ├── wiki-approval/ review_status lifecycle workflow -│ ├── knowledgebase-initialize/ Setup workflow -│ └── usage-report-setup/ Usage report mode workflow +│ └── knowledgebase-initialize/ Setup workflow ├── docs/raw/ Raw source frontmatter templates ├── CLAUDE.md LLM entry point and project skill map ├── README.md This file @@ -40,13 +38,12 @@ data/ Generated Markdown export tree (Postgres is ca │ ├── improvements/ Open-ended improvements │ ├── checklists/ Operational checklists │ └── summaries/ Time/subject rollups -├── rejected/ Rejected wiki pages (created on reject via kb-mcp/service layer) └── log.md Operation record ``` ## Workflows -Project workflows live in `.claude/skills/`. Use `wiki-authoring` for source-backed wiki edits, `wiki-approval` for review lifecycle work, `memory-report` for daily/weekly/monthly synthesis, and `handoff-document` for handoffs. +Project workflows live in `.claude/skills/`. Use `wiki-authoring` for source-backed wiki edits and `handoff-document` for handoffs. ## Privacy @@ -83,7 +80,7 @@ kb-lint ### Commit -Writes go through the `kb-mcp` MCP server tools (or the in-process `kb.service` layer used by cron CLIs) — Markdown files under `data/` are generated exports. +Writes go through the `kb-mcp` MCP server tools (or the in-process `kb.service` layer) — Markdown files under `data/` are generated exports. Changes to the outer repo (code, docs, skills) are committed as usual. ## Files @@ -93,7 +90,6 @@ Changes to the outer repo (code, docs, skills) are committed as usual. | `CLAUDE.md` | LLM entry point and project skill map | | `scripts/ingest-github.sh` | GitHub source collection | | `src/kb/cli/lint.py` | Wiki + handoff validation | -| `src/kb/cli/db_ttl_sweep.py` | Auto-reject stale unprocessed pages | | `src/kb/mcp/` | DB-canonical MCP server (`kb-mcp`) | | `src/kb/service/` | In-process write path (lint → DB → export) | | `data/log.md` | Operation record | diff --git a/alembic/versions/a1b2c3d4e5f6_drop_cron_metrics_review_status.py b/alembic/versions/a1b2c3d4e5f6_drop_cron_metrics_review_status.py new file mode 100644 index 0000000..43137f6 --- /dev/null +++ b/alembic/versions/a1b2c3d4e5f6_drop_cron_metrics_review_status.py @@ -0,0 +1,75 @@ +"""drop review_status column and cron_runs/metrics tables + +The ORM models and service layer no longer read or write +``pages.review_status``, the ``cron_runs`` table, or the ``metrics`` table +(references were removed in a prior change). Drop them from the schema. + +Revision ID: a1b2c3d4e5f6 +Revises: c2d3e4f5a6b7 +Create Date: 2026-07-10 00:00:00.000000 + +""" + +from __future__ import annotations + +from typing import Sequence, Union + +from alembic import op +import sqlalchemy as sa + +revision: str = "a1b2c3d4e5f6" +down_revision: Union[str, Sequence[str], None] = "c2d3e4f5a6b7" +branch_labels: Union[str, Sequence[str], None] = None +depends_on: Union[str, Sequence[str], None] = None + + +def upgrade() -> None: + """Drop pages.review_status and the cron_runs/metrics tables.""" + op.drop_index("ix_pages_review_status_updated_at", table_name="pages") + op.drop_column("pages", "review_status") + op.drop_table("cron_runs") + op.drop_table("metrics") + + +def downgrade() -> None: + """Recreate pages.review_status and the cron_runs/metrics tables.""" + op.add_column("pages", sa.Column("review_status", sa.Text(), nullable=True)) + op.create_index( + "ix_pages_review_status_updated_at", + "pages", + ["review_status", sa.text("updated_at DESC")], + ) + + op.create_table( + "cron_runs", + sa.Column("id", sa.Integer(), primary_key=True), + sa.Column("job_name", sa.Text(), nullable=False), + sa.Column("target", sa.Text(), nullable=False), + sa.Column("status", sa.Text(), nullable=False), + sa.Column("exit_code", sa.Integer(), nullable=True), + sa.Column("log_body", sa.Text(), nullable=False), + sa.Column("log_path", sa.Text(), nullable=True), + sa.Column("started_at", sa.Text(), nullable=True), + sa.Column("finished_at", sa.Text(), nullable=True), + sa.Column("created_at", sa.Text(), nullable=False), + ) + op.create_index("ix_cron_runs_target_job", "cron_runs", ["target", "job_name"]) + + op.create_table( + "metrics", + sa.Column("id", sa.Integer(), primary_key=True), + sa.Column("report_date", sa.Text(), nullable=False), + sa.Column("report_type", sa.Text(), nullable=False), + sa.Column("session_count", sa.Integer(), nullable=True), + sa.Column("token_total", sa.Integer(), nullable=True), + sa.Column("cost_usd", sa.Float(), nullable=True), + sa.Column("tool_error_count", sa.Integer(), nullable=True), + sa.Column("metrics_json", sa.JSON(), nullable=False), + sa.Column("created_at", sa.Text(), nullable=False), + ) + op.create_index( + "uq_metrics_date_type", + "metrics", + ["report_date", "report_type"], + unique=True, + ) diff --git a/docs/README.md b/docs/README.md index 89a6715..572ab2f 100644 --- a/docs/README.md +++ b/docs/README.md @@ -18,9 +18,6 @@ Agents should execute core workflows from `.claude/skills/`, not by loading work | Initialize a clone/profile | `.claude/skills/knowledgebase-initialize/SKILL.md` | | Write or fix wiki pages (evidence-derived) | `.claude/skills/wiki-authoring/SKILL.md` | | Capture a first-party note (human, no source) | `.claude/skills/wiki-note/SKILL.md` | -| Promote/approve/reject wiki pages | `.claude/skills/wiki-approval/SKILL.md` | -| Configure usage reports | `.claude/skills/usage-report-setup/SKILL.md` | -| Daily/weekly/monthly memory build | `.claude/skills/memory-report/SKILL.md` | | Write handoff documents | `.claude/skills/handoff-document/SKILL.md` | For human onboarding, read: @@ -35,10 +32,9 @@ For human onboarding, read: |---|---| | `docs/architecture.md` | System layout and data boundaries | | `docs/reference/` | Human-readable schemas, categories, commands, and lookup tables | -| `docs/db_informations/` | Database and reporting references | +| `docs/db_informations/` | Database references | | `docs/db_informations/state-db-schema-reference.md` | Postgres state DB schema + `psql` read recipes (reads go direct to the DB) | | `docs/db-canonical.md` | DB-as-source-of-truth memory architecture decision | -| `docs/workflows.md` | At-a-glance diagram map: nightly pipeline, review lifecycle (overview only — skills own execution) | | `docs/CLAUDE.md` | Document authoring rules | ### Document Types @@ -50,19 +46,14 @@ Use project skills for ordered execution steps. Use `docs/reference/` for human- | Need | Read | |---|---| | Understand the repo | `docs/architecture.md` | -| See the whole picture at a glance | `docs/workflows.md` | | Run raw-to-wiki pipeline | `.claude/skills/wiki-authoring/SKILL.md` | | Jot a first-party note (no source, from any repo) | `.claude/skills/wiki-note/SKILL.md` | -| Configure cron jobs | `.claude/skills/knowledgebase-initialize/SKILL.md` | -| Configure usage reports | `.claude/skills/usage-report-setup/SKILL.md` | -| Run daily/weekly/monthly memory build | `.claude/skills/memory-report/SKILL.md` | +| Initialize a clone/profile | `.claude/skills/knowledgebase-initialize/SKILL.md` | | Handle handoff lifecycle | `.claude/skills/handoff-document/SKILL.md` | | Write valid frontmatter | `.claude/skills/wiki-authoring/SKILL.md`; reference: `docs/reference/frontmatter.md` | | Choose wiki category/path | `.claude/skills/wiki-authoring/SKILL.md`; reference: `docs/reference/wiki-categories.md` | -| Approve/reject wiki pages | `.claude/skills/wiki-approval/SKILL.md` | | Run CLI commands | `docs/reference/commands.md` | | Read the DB directly (queries) | `docs/db_informations/state-db-schema-reference.md` | -| Start review console (web UI) | `README.md` → "Review console" section; `scripts/dev-web.sh` | | Back up / move the DB across machines | `docs/db-canonical.md` | | Understand the DB-canonical migration | `docs/db-canonical.md` | @@ -72,11 +63,11 @@ Use project skills for ordered execution steps. Use `docs/reference/` for human- ### A. PatchNote -- 2026-06-04: Removed the legacy `data-sync` skill and `docs/data-sync.md` (git-based two-repo sync); DB-canonical replaces it. Dropped the sync flow from `docs/workflows.md` and its document-map/routing rows here. +- 2026-06-04: Removed the legacy `data-sync` skill and `docs/data-sync.md` (git-based two-repo sync); DB-canonical replaces it. Dropped the sync flow from the at-a-glance workflow overview doc (since removed) and its document-map/routing rows here. - 2026-06-04: Postgres became the sole source of truth (SQLite removed); added `docs/db_informations/state-db-schema-reference.md` for direct `psql` reads. - 2026-06-04: Added `docs/db-canonical.md` to the document map and usage table. - 2026-06-02: Added the `wiki-note` skill (first-party human-authored pages, `origin: authored`) to runtime routing + usage tables; it is exposed globally like `handoff-document`. -- 2026-06-02: Added `docs/workflows.md` — at-a-glance Mermaid map of the nightly pipeline, review lifecycle, and two-repo sync. +- 2026-06-02: Added an at-a-glance Mermaid workflow overview doc (since removed) mapping the nightly pipeline, review lifecycle, and two-repo sync. - 2026-05-29: Updated data-sync sync entry to point at the `data-sync` skill. - 2026-05-28: Added `docs/data-sync.md` for private remote sync of the nested `data/` repo. - 2026-05-20: Removed `docs/workflows/`; project skills are now the sole workflow surface. diff --git a/docs/architecture.md b/docs/architecture.md index c44897a..8c1f414 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -27,7 +27,6 @@ Never commit `data/` contents to the outer repository. The outer `.gitignore` ex | Raw | `data/raw/` | Captured source evidence | Create only; never edit existing files | | Handoffs | `data/handoffs/` | Operational state and agent handoff | Update during tasks and periodic runs | | Wiki | `data/wiki/` | Durable long-term knowledge | Update only with source-backed frontmatter | -| Rejected | `data/rejected/` | Wiki pages rejected during review (audit trail) | Populated through DB API reject endpoint; mirrors `wiki/` tree | | Log | `data/log.md` | Append-only operation history | Append every operation | | Skill templates | `.claude/skills/*/reference/templates` | Runtime file skeletons bundled with skills | Update in outer repo | | Raw templates | `docs/raw/` | Raw source frontmatter skeletons | Update in outer repo | @@ -48,7 +47,7 @@ data/wiki/ summaries/ # daily, weekly, monthly, migration rollups ``` -Six of the seven types (`entity`, `concept`, `decision`, `improvement`, `checklist`, `question`) carry a `review_status` field (`not_processed` → `pending_for_approve` → `approved`) managed through DB API (promote/approve/reject endpoints). Summaries are exempt. Only `approved` pages appear in `INDEX.md`. Runtime approval work uses `.claude/skills/wiki-approval/SKILL.md`. +`INDEX.md` is generated export from the DB. ### Operating Flow @@ -58,7 +57,7 @@ The DB-canonical flow is: data/raw/ -> kb-mcp tools / service layer -> Postgres write -> data/wiki/ (Markdown export) -> data/log.md -> lint ``` -Writes go through the `kb-mcp` MCP server tools or the in-process `kb.service` layer (used by cron CLIs); Markdown files under `data/wiki/` are generated exports +Writes go through the `kb-mcp` MCP server tools or the in-process `kb.service` layer; Markdown files under `data/wiki/` are generated exports exported from Postgres. Handoffs sit beside the flow as the operational state board. They record what was processed, what is blocked, and what the next agent should do. @@ -76,7 +75,7 @@ is a generated export, not a read surface. | Component | Path | Role | |---|---|---| | `kb-mcp` FastMCP server | `src/kb/mcp/` | 12 write tools + `query_sql`/`get_schema` read tools over Postgres | -| Service layer | `src/kb/service/` | In-process write path (lint → DB → export); called by cron CLIs without a running server | +| Service layer | `src/kb/service/` | In-process write path (lint → DB → export); no running server needed | ## 3. Usage diff --git a/docs/db-canonical.md b/docs/db-canonical.md index da0c14d..0ae416d 100644 --- a/docs/db-canonical.md +++ b/docs/db-canonical.md @@ -19,13 +19,12 @@ The DB owns durable memory: | Table | Role | |---|---| -| `pages` | Canonical wiki page record: slug, title, body markdown, frontmatter JSON, review state, type/category/tags payload. | +| `pages` | Canonical wiki page record: slug, title, body markdown, frontmatter JSON, type/category/tags payload. | | `raw_sources` | Immutable captured evidence, replacing `data/raw/` as the canonical source store. | | `page_sources` | Many-to-many citation links between pages and raw sources. | | `page_revisions` | Append-only page revision ledger for body/frontmatter changes. | | `dispatches` | Existing external workflow dispatch state. | | `wiki_edits` | Former legacy frontmatter edit audit, now removed. `page_revisions` is the sole audit table. | -| `metrics` | Operational metrics (promotions, rejections, TTL sweeps, page counts) for dashboard and reporting. | Markdown exports are generated from DB rows. They may still be committed, searched, or opened in external tools, but a Markdown file no longer wins over a @@ -39,7 +38,7 @@ DB row when the two disagree. 3. ✅ Reads go directly against Postgres (`psql`), not Markdown scans or read endpoints — the web app is write-only. See `docs/db_informations/state-db-schema-reference.md`. -4. ✅ Move mutation endpoints (create/approve/reject/frontmatter edits) into one +4. ✅ Move mutation endpoints (create/update/frontmatter edits) into one DB transaction that also appends `page_revisions`, with write-time lint. 5. ✅ Add Markdown export as an explicit step. Export is allowed to overwrite generated files because it is derived output. diff --git a/docs/db_informations/hermes-monitoring-guide.md b/docs/db_informations/hermes-monitoring-guide.md deleted file mode 100644 index 77f6d9b..0000000 --- a/docs/db_informations/hermes-monitoring-guide.md +++ /dev/null @@ -1,106 +0,0 @@ -# Hermes Monitoring Guide — Daily & Weekly Report - -Updated: 2026-05-11 - -## 1. Synopsis - -- **Purpose**: Hermes DB에서 어떤 데이터를 왜 모니터링해야 하는지 정의. 쿼리는 에이전트가 이 문서를 보고 생성. -- **DB**: `~/.hermes/state.db` + `~/.hermes/profiles/*/state.db` - ---- - -## 2. 모니터링 지표 정의 - -### Layer 1: 비용·효율 (Cost & Efficiency) - -**목적**: 얼마나 썼나, 낭비는 없나. - -| 지표 | 소스 | 의미 | -|------|------|------| -| 일일 총 비용 | `COALESCE(actual_cost_usd, estimated_cost_usd)` | 전날 대비 20%+ 증가 시 드릴다운 | -| 모델별 비용 점유율 | `sessions.model` + 비용 | 비싼 모델이 단순 task에 쓰이는지 확인 | -| 캐시 히트율 | `cache_read_tokens / (input_tokens + cache_read_tokens) × 100` | 낮으면 system prompt 구조 재검토 | -| reasoning 토큰 비율 | `reasoning_tokens / output_tokens` | thinking 모델에서 단순 task에 낭비되는지 확인 | -| API 호출 횟수 | `sessions.api_call_count` | 세션당 과도하면 루프 탈출 실패 의심 | - -### Layer 2: 작업 품질 (Task Quality) - -**목적**: 작업이 완료됐나, 어디서 막혔나. - -| 지표 | 소스 | 의미 | -|------|------|------| -| 완료율 | `end_reason = 'cron_complete', 'cli_close', 'api_complete' 등` / 전체 | 실측: `cli_close` 31건, NULL(비정상) 24건 | -| end_reason 분포 | `sessions.end_reason` | `context_limit` 多 → compaction 설정 조정, `error` 多 → tool 실패 분석 | -| 진행 중 세션 수 | `ended_at IS NULL` | 비정상적으로 많으면 좀비 세션 의심 | -| 도구 호출 횟수 | `sessions.tool_call_count` | 세션당 평균 대비 3배+ = 루프 또는 반복 재시도 | -| context compression 발생 | `end_reason = 'compression'` 또는 `parent_session_id IS NOT NULL` | 세션 분할 또는 컨텍스트 정리 전략 수립 | - -### Layer 3: 행동 패턴 (Behavioral Patterns) - -**목적**: 언제, 어떻게, 어디서 일하나. - -| 지표 | 소스 | 의미 | -|------|------|------| -| source별 세션 분포 | `sessions.source` | 실측: cli(37) > api_server(18) > telegram(12) > cron(11) > tui(10) | -| 시간대별 세션 분포 | `strftime('%H', started_at, 'unixepoch')` | 피크 시간대 파악, cron 스케줄 최적화 | -| 평균 세션 시간 | `AVG(ended_at - started_at)` | 길어지면 context 폭발 또는 tool 루프 의심 | -| 도구 사용 Top N | `messages.tool_calls` JSON 파싱 | 가장 많이 쓰는 도구 패턴 파악 | -| 세션당 도구 호출 빈도 | `tool_call_count / COUNT(sessions)` | 특정 도구가 평소의 3배면 루프 징후 | - -### Layer 4: 프로파일별 분포 (Multi-Agent Focus) - -**목적**: 어느 에이전트가 얼마나 일하는가. - -| 지표 | 소스 | 의미 | -|------|------|------| -| 프로파일별 세션·비용 | 각 `profiles/*/state.db` | 에이전트별 부하 분산 확인 | -| 프로파일별 모델 사용 | `sessions.model` per profile | 에이전트별 모델 라우팅 정책 검증 | -| root vs subagent 비율 | `parent_session_id IS NULL` | 실측: root 85개 vs subagent 4개 | - ---- - -## 3. 신호 → 액션 매핑 - -| 신호 | 임계값 | 액션 | -|------|--------|------| -| 일일 비용 급증 | 전날 대비 +20% | 해당일 세션 Top 5 드릴다운 | -| 캐시 히트율 저하 | < 30% | system prompt 구조 재검토 | -| NULL end_reason 비율 | > 20% | 비정상 종료 원인 분석 | -| 세션당 API 호출 | 평균 대비 3배+ | 루프 탈출 실패 여부 확인 | -| reasoning 토큰 비율 | > 30% | `max_thinking_tokens` 조정 검토 | -| context compression | 주간 3회+ | 세션 분할 전략 수립 | -| 특정 도구 호출 급증 | 평소 대비 3배+ | 루프 또는 반복 재시도 징후 | - ---- - -## 4. Daily vs Weekly 분리 기준 - -``` -Daily Report (어제 기준) Weekly Report (지난 7일) -───────────────────────── ────────────────────────────── -• 총 비용 / 세션 수 / 완료율 • 일별 비용·토큰 트렌드 -• source별 사용 분포 • 캐시 히트율 추이 (개선/악화) -• end_reason 분포 • 모델별 비용 점유율 -• 도구 사용 Top 10 • 도구별 calls_per_session 트렌드 -• 비정상 종료 세션 목록 • 피크 시간대 패턴 -• 진행 중(좀비) 세션 수 • 프로파일별 부하 분산 -``` - ---- - -## Appendix - -### A. OpenCode와의 차이점 - -| 항목 | Hermes | OpenCode | -|------|--------|----------| -| 타임스탬프 단위 | **초** (float) | **밀리초** (integer) | -| 토큰 저장 위치 | `sessions` 테이블 컬럼 | `part` 테이블 JSON | -| 비용 컬럼 | `actual_cost_usd` / `estimated_cost_usd` 분리 | `part.step-finish.cost` | -| 도구 호출 기록 | `messages.tool_calls` JSON 배열 | `part.tool` JSON | -| 멀티 DB | 프로파일별 독립 DB | 단일 DB | -| FTS 지원 | ✅ (FTS5 + trigram) | ❌ | - -### B. 관련 문서 - -- `hermes-schema-reference.md` — 테이블·컬럼·JSON 구조 상세 diff --git a/docs/db_informations/hermes-schema-reference.md b/docs/db_informations/hermes-schema-reference.md deleted file mode 100644 index b5d982d..0000000 --- a/docs/db_informations/hermes-schema-reference.md +++ /dev/null @@ -1,169 +0,0 @@ -# Hermes SQLite Schema Reference - -Updated: 2026-05-11 - -## 1. Synopsis - -- **Purpose**: Hermes DB 스키마 레퍼런스. 에이전트가 쿼리 작성 전 참조하는 원천 문서. -- **DB**: `~/.hermes/state.db` (SQLite, WAL 모드, FTS5, schema v11) - ---- - -## 2. DB 위치 - -| 경로 | 설명 | -|------|------| -| `~/.hermes/state.db` | **메인 DB** — 모든 세션·메시지 기록 | -| `~/.hermes/profiles/*/state.db` | 프로파일별 독립 DB (멀티-에이전트) | -| `~/.hermes/kanban.db` | 칸반 태스크 DB | - -**프로파일 목록**: `main-gateway`, `research-agent`, `execution-agent`, `structuring-agent`, `verification-agent` - -> 프로파일 DB는 메인 DB와 **완전히 별개**. 합산 시 개별 조회 후 UNION 필요. - ---- - -## 3. 테이블 관계도 - -``` -sessions ──< messages - └──< messages_fts (FTS5 전문 검색) - └──< messages_fts_trigram (CJK/부분 문자열 검색) -state_meta (키-값 메타데이터) -schema_version (마이그레이션 버전) -``` - -- `sessions.id` → `messages.session_id` -- `sessions.parent_session_id` → `sessions.id` (self-referential, context compression 분기) - ---- - -## 4. 테이블 스키마 - -### sessions - -| 컬럼 | 타입 | 설명 | -|------|------|------| -| `id` | TEXT PK | 세션 식별자 (`YYYYMMDD_HHMMSS_hex`) | -| `source` | TEXT | `cli` \| `tui` \| `telegram` \| `slack` \| `cron` \| `api_server` | -| `user_id` | TEXT | 사용자 식별자 (NULL 가능) | -| `model` | TEXT | 사용 모델명 (예: `claude-opus-4-7`, `Qwen3.6-35B-A3B-FP8`) | -| `parent_session_id` | TEXT FK | NULL = root 세션, 값 있음 = context compression 분기 | -| `started_at` | REAL | Unix epoch **초** (float) | -| `ended_at` | REAL | Unix epoch 초. NULL = 진행 중 | -| `end_reason` | TEXT | 종료 이유 (아래 참조) | -| `message_count` | INTEGER | 세션 내 메시지 수 | -| `tool_call_count` | INTEGER | 도구 호출 횟수 | -| `api_call_count` | INTEGER | API 호출 횟수 | -| `input_tokens` | INTEGER | 입력 토큰 합계 | -| `output_tokens` | INTEGER | 출력 토큰 합계 | -| `cache_read_tokens` | INTEGER | 캐시 읽기 토큰 | -| `cache_write_tokens` | INTEGER | 캐시 쓰기 토큰 | -| `reasoning_tokens` | INTEGER | Extended thinking 토큰 | -| `billing_provider` | TEXT | 청구 제공자 | -| `billing_mode` | TEXT | 청구 방식 | -| `estimated_cost_usd` | REAL | 추정 비용 | -| `actual_cost_usd` | REAL | 실제 청구 비용 | -| `cost_status` | TEXT | 비용 계산 상태 (`unknown` 등) | -| `title` | TEXT | 세션 제목 (UNIQUE, NULL 가능) | - -> **비용 컬럼 우선순위**: `COALESCE(actual_cost_usd, estimated_cost_usd)` -> **캐시 히트율**: `cache_read_tokens / (input_tokens + cache_read_tokens) × 100` - -**`end_reason` 값 목록** - -| 값 | 설명 | -|----|------| -| `cli_close` | CLI 세션 종료 | -| `tui_close` / `tui_shutdown` | TUI 세션 종료 | -| `cron_complete` | 크론 작업 완료 | -| `api_complete` | API 서버 요청 정상 완료 | -| `api_error` | API 서버 요청 중 에러/크래시 | -| `api_cancelled` | API 서버 요청 서버 취소 (shutdown 등) | -| `api_client_disconnect` | API 서버 SSE 클라이언트 연결 끊김 | -| `session_reset` | 세션 리셋 | -| `new_session` | 새 세션으로 전환 | -| `compression` | 컨텍스트 압축 | -| NULL | 비정상 종료 또는 진행 중 | - -### messages - -| 컬럼 | 타입 | 설명 | -|------|------|------| -| `id` | INTEGER PK | 자동 증가 | -| `session_id` | TEXT FK | → `sessions.id` | -| `role` | TEXT | `user` \| `assistant` \| `tool` \| `session_meta` | -| `content` | TEXT | 메시지 본문 | -| `tool_calls` | TEXT (JSON) | 도구 호출 배열 (role=assistant일 때) | -| `tool_name` | TEXT | 호출된 도구 이름 (role=tool일 때) | -| `timestamp` | REAL | Unix epoch 초 | -| `finish_reason` | TEXT | `stop` \| `tool_calls` | -| `reasoning` | TEXT | Extended thinking 원문 | -| `token_count` | INTEGER | 해당 메시지 토큰 수 | - ---- - -## 5. messages.tool_calls JSON 구조 - -```json -[ - { - "id": "call_function_xxx", - "call_id": "call_function_xxx", - "type": "function", - "function": { - "name": "terminal", - "arguments": "{\"command\": \"ls -la\"}" - } - } -] -``` - -> 도구명 추출: `json_extract(value, '$.function.name')` + `json_each(tool_calls)` -> `tool_name` 컬럼은 role=tool 행에만 채워짐. role=assistant의 도구 호출은 `tool_calls` JSON에서 추출. - ---- - -## Appendix - -### A. 공통 제약 - -- 타임스탬프는 **초 단위 float** → `date(col, 'unixepoch')` (OpenCode의 밀리초와 다름) -- 비용: `COALESCE(actual_cost_usd, estimated_cost_usd)` 패턴 사용 -- 루트 세션만 집계: `WHERE parent_session_id IS NULL` (중복 집계 방지) -- 진행 중 세션 제외: `WHERE ended_at IS NOT NULL` -- 스키마 버전 확인: `SELECT version FROM schema_version;` (현재: 11) - -### B. 인덱스 (성능 참고) - -``` -idx_sessions_source ON sessions(source) -idx_sessions_parent ON sessions(parent_session_id) -idx_sessions_started ON sessions(started_at DESC) -idx_messages_session ON messages(session_id, timestamp) -``` - -### C. FTS5 검색 - -```sql --- 전문 검색 (영문) -SELECT rowid FROM messages_fts WHERE messages_fts MATCH 'keyword'; - --- 부분 문자열 / CJK 검색 -SELECT rowid FROM messages_fts_trigram WHERE messages_fts_trigram MATCH '검색어'; -``` - -> FTS5 인덱스는 `content + tool_name + tool_calls` 를 합쳐서 색인. - -### D. 프로파일 DB 통합 조회 패턴 - -```bash -for db in ~/.hermes/state.db ~/.hermes/profiles/*/state.db; do - [ -f "$db" ] || continue - sqlite3 "$db" "SELECT ..." -done -``` - -### E. 관련 문서 - -- `hermes-monitoring-guide.md` — 무엇을 왜 모니터링할지 지표 정의 diff --git a/docs/db_informations/opencode-monitoring-guide.md b/docs/db_informations/opencode-monitoring-guide.md deleted file mode 100644 index c00a1d6..0000000 --- a/docs/db_informations/opencode-monitoring-guide.md +++ /dev/null @@ -1,84 +0,0 @@ -# OpenCode Monitoring Guide — Daily & Weekly Report - -Updated: 2026-05-11 - -## 1. Synopsis - -- **Purpose**: OpenCode DB에서 어떤 데이터를 왜 모니터링해야 하는지 정의. 쿼리는 에이전트가 이 문서를 보고 생성. -- **DB**: `~/.local/share/opencode/opencode.db` — 스키마 상세는 `opencode-schema-reference.md` 참조. - ---- - -## 2. 모니터링 지표 정의 - -### Layer 1: 비용·효율 (Cost & Efficiency) - -**목적**: 얼마나 썼나, 낭비는 없나. - -> **모델별 토큰·비용은 `message` 테이블이 유일한 소스.** -> `session.model`은 subagent 세션에서 NULL이지만, `message.data.modelID`에는 root/subagent 구분 없이 실제 모델명이 정확히 기록된다. - -| 지표 | 소스 | 의미 | -|------|------|------| -| 모델별 일일 토큰 | `message.data` (modelID, tokens.*) | root+subagent 포함 실제 모델 분포 | -| 기록 비용 | `message.data.cost` | OpenCode가 모든 모델에 대해 자동 집계. vllm(자체호스팅)은 0 | -| 캐시 히트율 | `cache_read / (input_cache_miss + cache_read) × 100` | 높을수록 비용 효율 좋음. `input`은 캐시 미스분만이므로 반드시 cache_read와 합산해서 분모 계산 | -| API 호출 횟수 | SQLite `part.type=step-finish` count per session | 51회+ = 루프 탈출 실패 의심 | -| reasoning 토큰 비율 | `message.tokens.reasoning` / 전체 | Extended thinking 남용 여부 | - -### Layer 2: 작업 품질 (Task Quality) - -**목적**: 작업이 완료됐나, 어디서 막혔나. - -| 지표 | 소스 | 의미 | -|------|------|------| -| Todo 완료율 | `todo.status` | 전체 평균 80.9%. 세션 완료율 0% = 중단된 작업 | -| 도구별 에러율 | `tool.state.status = error` | webfetch 12.8%, write 5.1%, edit 3.9% | -| 연속 에러 세션 | 같은 도구가 동일 세션에서 2회+ 실패 | 에이전트가 막혀서 반복 실패하는 구간 | -| compaction 발생 | `part.type = compaction` | 세션이 너무 길어짐 → 분할 검토 | -| step-finish reason=error | `step-finish.reason = error` | API 레벨 실패 (드묾, 실측 2건) | - -### Layer 3: 행동 패턴 (Behavioral Patterns) - -**목적**: 언제, 어떻게 일하나 — 습관 최적화. - -| 지표 | 소스 | 의미 | -|------|------|------| -| 시간대별 세션 분포 | `session.time_created` hour | 실측: 새벽 1~3시 집중(150개), 낮 저활동 | -| 요일별 활동량 | `session.time_created` weekday | 실측: 월요일 136개로 압도적 1위 | -| root vs subagent 비율 | `session.parent_id IS NULL` | 실측: root 86개 vs subagent 249개 (위임율 74%) | -| 도구 사용 믹스 | `tool.tool` 분포 | read > bash > grep 순. 탐색 vs 실행 비율 | -| 세션 길이 분포 | `step-finish` count per session | short(35%) / medium(40%) / long(18%) / very long(6%) | - -### Layer 4: 집중도·안정성 (Focus & Stability) - -**목적**: 어디에 집중했나, 무엇이 불안정한가. - -| 지표 | 소스 | 의미 | -|------|------|------| -| 일일 프로젝트 전환 수 | SQLite `session.project_id` distinct per day | 3개+ = 컨텍스트 스위칭 과다 | -| 핫 파일 (주간 수정 빈도) | SQLite `patch.files[]` | 반복 수정 = 설계 불안정. 실측: mail_server.py 46회 | -| 세션당 수정 파일 수 | SQLite `patch.files[]` distinct per session | 과도하게 많으면 범위 초과 작업 | -| 모델별 세션 분포 (전체) | `message.data.modelID` GROUP BY | root+subagent 포함 실제 모델 분포. session.model 사용 금지 | - ---- - -## 4. 신호 → 액션 매핑 - -| 신호 | 임계값 | 액션 | -|------|--------|------| -| API 호출 횟수 | 세션 > 50회 | 루프 탈출 실패 여부 확인, 세션 분할 검토 | -| 도구 에러율 | 도구별 > 5% | retry 로직 추가 또는 대체 도구 검토 | -| Todo 완료율 | 세션 < 50% | 중단 원인 분석 (계획 과다? 실행 실패?) | -| compaction 발생 | 1회 이상 | 세션 분할 또는 컨텍스트 정리 전략 수립 | -| 일일 프로젝트 전환 | > 3개 | 집중도 저하 경고 | -| 핫 파일 수정 횟수 | 주간 > 10회 | 설계 불안정 → 리팩토링 우선순위 상향 | -| reasoning 토큰 비율 | > 30% | Extended thinking 사용 타당성 검토 | - ---- - -## Appendix - -### A. 관련 문서 - -- `opencode-schema-reference.md` — 테이블·컬럼·JSON 구조 상세 diff --git a/docs/db_informations/opencode-schema-reference.md b/docs/db_informations/opencode-schema-reference.md deleted file mode 100644 index fe0f0ec..0000000 --- a/docs/db_informations/opencode-schema-reference.md +++ /dev/null @@ -1,215 +0,0 @@ -# OpenCode SQLite Schema Reference - -Updated: 2026-05-11 - -## 1. Synopsis - -- **Purpose**: OpenCode DB 스키마 레퍼런스. 에이전트가 쿼리 작성 전 참조하는 원천 문서. -- **DB**: `~/.local/share/opencode/opencode.db` (SQLite 3.51.2, Drizzle ORM, WAL 모드) - ---- - -## 2. 테이블 관계도 - -``` -project ──< session ──< part - └──< message ──< part - └──< todo - └──< session_share -``` - -- `project.id` → `session.project_id` -- `session.id` → `part.session_id`, `message.session_id`, `todo.session_id` -- `message.id` → `part.message_id` - ---- - -## 3. 테이블 스키마 - -### project - -| 컬럼 | 타입 | 설명 | -|------|------|------| -| `id` | TEXT PK | 프로젝트 식별자 | -| `worktree` | TEXT | 작업 디렉토리 경로 | -| `name` | TEXT | 프로젝트 이름 (NULL 가능) | -| `time_created` | INTEGER | Unix epoch 밀리초 | -| `time_updated` | INTEGER | Unix epoch 밀리초 | - -### session - -| 컬럼 | 타입 | 설명 | -|------|------|------| -| `id` | TEXT PK | 세션 식별자 (`ses_...`) | -| `project_id` | TEXT FK | → `project.id` | -| `parent_id` | TEXT | NULL = root 세션, 값 있음 = subagent 세션 | -| `title` | TEXT | 세션 제목 | -| `model` | TEXT (JSON) | root 세션만 기록. **subagent 세션은 NULL** (upstream 미기록) | -| `agent` | TEXT | 에이전트 이름 (예: `CodeRabbit-Dev`) | -| `directory` | TEXT | 작업 디렉토리 경로 | -| `time_created` | INTEGER | Unix epoch 밀리초 | -| `time_updated` | INTEGER | Unix epoch 밀리초 | - -> ⚠️ **`session.model`은 모델 집계에 사용하지 말 것.** subagent 세션은 NULL이므로 불완전하다. -> 실제 모델명은 `message.data.modelID`에 root/subagent 구분 없이 정확히 기록된다. - -### part - -| 컬럼 | 타입 | 설명 | -|------|------|------| -| `id` | TEXT PK | 파트 식별자 (`prt_...`) | -| `message_id` | TEXT FK | → `message.id` | -| `session_id` | TEXT FK | → `session.id` | -| `time_created` | INTEGER | Unix epoch 밀리초 | -| `data` | TEXT (JSON) | `data.type`으로 분기 (아래 참조) | - -**`data.type` 값 목록** - -| type | 비율 | 설명 | -|------|------|------| -| `tool` | 34% | 도구 호출 1건 | -| `step-start` | 19% | API 호출 시작 | -| `step-finish` | 19% | API 호출 완료 — 토큰·비용의 유일한 소스 | -| `reasoning` | 14% | Extended thinking 내용 | -| `text` | 13% | 텍스트 응답 | -| `patch` | 2% | 파일 변경 스냅샷 | -| `file` | <1% | 첨부 파일 | -| `compaction` | <1% | 컨텍스트 압축 발생 | - -### todo - -| 컬럼 | 타입 | 설명 | -|------|------|------| -| `session_id` | TEXT FK | → `session.id` | -| `content` | TEXT | 할 일 내용 | -| `status` | TEXT | `pending` \| `in_progress` \| `completed` \| `cancelled` | -| `priority` | TEXT | `high` \| `medium` \| `low` | -| `position` | INTEGER | 순서 (PK 구성요소) | -| `time_created` | INTEGER | Unix epoch 밀리초 | - ---- - -## 4. part.data JSON 구조 (타입별) - -### step-finish -```json -{ - "type": "step-finish", - "reason": "tool-calls", - "snapshot": "a2ff2a1b...", - "tokens": { - "total": 42537, - "input": 1592, - "output": 113, - "reasoning": 0, - "cache": { "write": 0, "read": 40832 } - }, - "cost": 0 -} -``` -> `total = input + cache.read + output` -> **`input` = 캐시 미스 토큰만** (새로 읽힌 것). `cache.read` = 캐시 히트 토큰. -> 실제 LLM 입력량 = `input + cache.read`. 캐시 히트율 = `cache.read / (input + cache.read)` -> `cost`는 OpenCode가 모든 모델에 대해 자동 집계해서 기록. - -### tool -```json -{ - "type": "tool", - "tool": "bash", - "callID": "toolu_...", - "state": { - "status": "completed", - "input": { "command": "ls -la" }, - "metadata": { "output": "..." } - } -} -``` -> `state.status`: `completed` | `error` | `running` | `pending` - -### patch -```json -{ - "type": "patch", - "hash": "a2ff2a1b...", - "files": ["/absolute/path/to/file.py"] -} -``` - -### reasoning -```json -{ - "type": "reasoning", - "thinking": "..." -} -``` - -### compaction -```json -{ - "type": "compaction" -} -``` -> 세션이 너무 길어져 컨텍스트 압축이 발생했음을 나타냄. - ---- - -## Appendix - -### A. 공통 제약 - -- 모든 `time_*` 컬럼은 **밀리초 epoch** → `datetime(col/1000,'unixepoch')` -- `session.model`은 root 세션만 기록, subagent는 NULL → **모델 집계에 사용 금지** -- **모델별 토큰·비용은 `message` 테이블에서 집계** (아래 §B 참조) -- `patch.files`는 JSON 배열 → `json_each(json_extract(data,'$.files'))`로 펼치기 - -### B. message 테이블 — 토큰·비용 집계 (핵심) - -`message.data` (JSON)에 root/subagent 구분 없이 실제 모델명과 토큰이 기록된다. - -```sql --- 날짜별 모델별 토큰·비용 집계 (KST 기준) -SELECT - json_extract(m.data, '$.modelID') AS modelID, - json_extract(m.data, '$.providerID') AS providerID, - SUM(json_extract(m.data, '$.tokens.input')) AS input_cache_miss, -- 캐시 미스 토큰만 - SUM(json_extract(m.data, '$.tokens.cache.read')) AS cache_read, -- 캐시 히트 토큰 - SUM(json_extract(m.data, '$.tokens.output')) AS output, - SUM(json_extract(m.data, '$.tokens.cache.write')) AS cache_write, - SUM(json_extract(m.data, '$.tokens.reasoning')) AS reasoning, - SUM(json_extract(m.data, '$.tokens.input') - + json_extract(m.data, '$.tokens.cache.read')) AS total_input, -- 실제 LLM 입력량 - SUM(json_extract(m.data, '$.cost')) AS actual_cost -FROM message m -JOIN session s ON m.session_id = s.id -WHERE date(datetime(s.time_created/1000, 'unixepoch', '+9 hours')) = 'YYYY-MM-DD' - AND json_extract(m.data, '$.role') = 'assistant' -GROUP BY modelID, providerID -ORDER BY total_input DESC; -``` - -> **캐시 히트율** = `cache_read / (input_cache_miss + cache_read) × 100` - -**providerID별 비용 처리 규칙:** - -| providerID | 비용 소스 | -|------------|----------| -| `anthropic` | `message.data.cost` (OpenCode 자동 집계) | -| `openai` | `message.data.cost` (OpenCode 자동 집계) | -| `opencode-go` | `message.data.cost` (실청구) | -| `google` | `message.data.cost` (OpenCode 자동 집계) | -| `vllm` | 0 USD (자체호스팅, 집계 제외) | - -### B. 인덱스 (성능 참고) - -``` -part_session_idx ON part(session_id) -part_message_id_id_idx ON part(message_id, id) -session_project_idx ON session(project_id) -session_parent_idx ON session(parent_id) -todo_session_idx ON todo(session_id) -``` - -### C. 관련 문서 - -- `opencode-monitoring-guide.md` — 무엇을 왜 모니터링할지 지표 정의 diff --git a/docs/db_informations/report-pipeline.md b/docs/db_informations/report-pipeline.md deleted file mode 100644 index 68222a2..0000000 --- a/docs/db_informations/report-pipeline.md +++ /dev/null @@ -1,109 +0,0 @@ -# Report Pipeline — Source-Specific Daily Reports - -Updated: 2026-05-18 - -## 1. Synopsis - -- **Purpose**: Define the source-specific daily report pipeline for OpenCode, Hermes, and Claude Code. -- **Format**: All reports are Markdown. Storage root: `data/wiki/summaries/`. - ---- - -## 2. Pipeline Structure - -``` -OpenCode DB -> data/wiki/summaries/YYYY/MM/YYYY-MM-DD-opencode-usage.md -Hermes DB -> data/wiki/summaries/YYYY/MM/YYYY-MM-DD-hermes-usage.md -Claude telemetry -> data/wiki/summaries/YYYY/MM/YYYY-MM-DD-claude-code-usage.md - -Daily memory cron reads source-specific reports and writes: -data/wiki/summaries/YYYY/MM/YYYY-MM-DD-memory.md -``` - -### Source Responsibilities - -| Source | Metrics | Reason | -|------|----------|------| -| `message.data` (modelID, tokens, cost) | Model tokens, recorded cost, shadow cost inputs | Records actual model names across root and subagent sessions. | -| `session` (parent_id, time_created) | Session count, root/subagent split, hourly distribution, project distribution | Session structure source. | -| `part` (type=tool, patch, compaction) | Tool error rate, hot files, compaction | In-session behavior data. | -| `todo` | TODO completion rate | Work quality proxy. | -| SQLite `hermes/state.db` | Hermes metrics | Source of truth for Hermes source-specific reports. | - ---- - -## 3. Common Rules - -### Date Boundary -- Default boundary: one local day in KST (UTC+9) unless configured otherwise. -- Session attribution: `session.time_created` or source-equivalent session start time. -- Query range: local day start <= session start < next local day start. -- OpenCode: `time_created` is Unix epoch milliseconds. -- Hermes: `started_at` is Unix epoch seconds. - -### Missing Source Handling -- If a selected source DB/telemetry is missing, skip only that source report. -- Continue generating other selected source reports. -- Record the skip reason in the wrapper log or handoff. - -### Filename Rules -- OpenCode: `YYYY-MM-DD-opencode-usage.md` -- Hermes: `YYYY-MM-DD-hermes-usage.md` -- Claude Code: `YYYY-MM-DD-claude-code-usage.md` -- Daily memory synthesis: `YYYY-MM-DD-memory.md` -- Weekly synthesis: `YYYY-WNN-weekly.md` -- Monthly synthesis: `YYYY-MM-monthly.md` - -### Weekly Synthesis Method -- Do not re-query DBs. -- Read source-specific daily reports and daily memory summaries. -- Mark missing sources as `not configured` or `missing`. - ---- - -## 4. Templates - -- Source-specific daily reports are rendered by CLI commands. -- Memory synthesis summaries follow `.claude/skills/memory-report/SKILL.md`. - -**Empty-day handling**: If a configured source has no activity, write `no activity`. If the source itself is missing, do not create that report. - -## 5. Cost Calculation - -### Cost Source - -- **Recorded cost**: `SUM(message.data.cost)`. OpenCode records this across models. - -### providerID Handling - -| providerID | Cost Source | -|------------|----------| -| `anthropic` | `message.data.cost` (OpenCode recorded cost) | -| `openai` | `message.data.cost` (OpenCode recorded cost) | -| `google` | `message.data.cost` (OpenCode recorded cost) | -| `opencode-go` | `message.data.cost` (actual recorded cost) | -| `vllm` | 0 USD (self-hosted, excluded from cost aggregation) | - ---- - -## Appendix - -### A. Full Storage Layout - -``` -data/wiki/summaries/ -└── 2026/ - └── 05/ - ├── 2026-05-10-opencode-usage.md - ├── 2026-05-10-hermes-usage.md - ├── 2026-05-10-claude-code-usage.md - ├── 2026-05-10-memory.md - ├── 2026-W19-weekly.md - └── 2026-05-monthly.md -``` - -### B. Signal-To-Action References - -Use these thresholds when interpreting daily/weekly signals: -- `opencode-monitoring-guide.md` §4 -- `hermes-monitoring-guide.md` §3 diff --git a/docs/db_informations/state-db-schema-reference.md b/docs/db_informations/state-db-schema-reference.md index 7fc8f30..c75bba3 100644 --- a/docs/db_informations/state-db-schema-reference.md +++ b/docs/db_informations/state-db-schema-reference.md @@ -6,7 +6,7 @@ Updated: 2026-06-04 - **Purpose**: Let agents/skills **read** the canonical KnowledgeBase state directly. Postgres is the sole source of truth; `data/` is a generated human-readable export. - **I/O**: `psql "$DATABASE_URL" -tAc " fetched from GET /api/kanban/boards │ -│ ├── direction