feat(sanitizer): configurable secret policy, PEM/JWT detection, honest docs (#186)

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

Author: chrisleekr

docs/architecture.md

@@ -228,7 +228,7 @@ Daemon installation paths per OS, log locations, and the configuration table liv
 Three places own the security contract:
 
 - **Encryptor**: the only path that generates age identities, derives recipients, and encrypts content. Plaintext never leaves this layer for any artefact going to disk or to the network.
-- **Sanitiser**: the only place that decides what is safe to encrypt. Never-sync paths and literal-secret detection are hard-coded rules, not opt-in policy. Loosening either requires a documented reason.
+- **Sanitiser**: the only place that decides what is safe to encrypt. Never-sync paths are hard-coded rules. Literal-secret detection is a **known-credential-format** guard (vendor key prefixes, AWS/GitHub/GitLab/Slack/Google tokens, age identities, PEM private keys, and JWTs in strict mode) — not a general secret scanner: a plain password or bespoke token with no recognised shape is not caught, so encryption, not the scan, is the real protection. The scan's job is to keep well-known credentials out of git history. Its breadth and the base64 redactor are tuned through the `[security]` config, resolved by `securityToPolicy`; the never-sync rules are not configurable.
 - **Tar bundler**: exists because some agent assets are directory-shaped. The tar is built in memory before encryption so an intermediate plaintext archive never lands on disk.
 
 Private keys stay on disk in the local runtime directory (`~/.config/agentsync/key.txt` by default on Unix, with restrictive permissions). They are never committed and never logged.

docs/commands.md

@@ -346,18 +346,23 @@ agentsync config set security.allowSecretValues '["AKIA-not-a-real-key"]'
 | `sync.debounceMs` | integer 50–10000 | Daemon quiet-window before an auto-push. |
 | `sync.autoPush` | boolean | Whether the daemon auto-pushes on change. |
 | `claudePlugins.syncPlugins` | boolean | Record the Claude plugin reinstall manifest on push. |
-| `security.secretScan` | `standard`\|`strict`\|`off` | Push-time secret-scan mode (see note). |
-| `security.allowSecretValues` | string[] (JSON) | Literal values exempt from secret detection and base64 redaction (see note). |
-| `security.redactBase64Values` | boolean | Replace long base64-looking JSON values with a redaction placeholder (see note). |
-
-> **`security.*` are recorded but not yet enforced.** This release stores the
-> policy in `agentsync.toml`; the push-time secret scanner starts honouring
-> `secretScan`, `allowSecretValues`, and `redactBase64Values` in a follow-up
-> change. Until then the scan runs with its built-in defaults regardless of
-> these values. Note `agentsync.toml` is committed in **plaintext** (only
-> artefacts are encrypted), so `allowSecretValues` is for exempting legitimate
+| `security.secretScan` | `standard`\|`strict`\|`off` | Push-time secret-scan mode. `standard` = built-in credential patterns; `strict` also flags JWTs; `off` disables the artefact-body scan. |
+| `security.allowSecretValues` | string[] (JSON) | Literal values exempt from secret detection and base64 redaction. |
+| `security.redactBase64Values` | boolean | When `true` (default), redact long base64-looking JSON values; set `false` if a config legitimately stores base64 that must round-trip. |
+
+> **What the secret scan is — and is not.** It matches a fixed set of
+> high-precision **credential formats** (vendor API-key prefixes, AWS/GitHub/GitLab/Slack/Google
+> tokens, age identities, PEM private-key headers; `strict` adds JWTs). It is
+> **not** a general secret scanner — a plain password, a bespoke token, or a
+> connection string with no recognised shape passes through. Encryption is the
+> real protection; the scan only stops well-known credentials from entering git
+> history. `off` disables the artefact-body scan, but **skill-bundle interiors
+> are always scanned at `standard`** as a fail-safe. `agentsync.toml` itself is
+> committed in **plaintext**, so `allowSecretValues` is for exempting legitimate
 > high-entropy *non-secret* values — never paste a real credential there.
-> `config set` refuses a recognised credential in any other value.
+> `config set` refuses to store a recognised credential in any key other than
+> `security.allowSecretValues`.
+> See [Push aborts because secrets were detected](operations.md#push-aborts-because-secrets-were-detected).
 
 **Outcome**: `list` and `get` are read-only. `set` validates the new value against the full config schema (so an out-of-range debounce or an invalid enum is rejected before anything is written), then — because `agentsync.toml` is shared across machines — reconciles fast-forward, commits, and pushes the change, exactly like `key add`.
 

docs/operations.md

@@ -310,7 +310,7 @@ and behaviour reference.
 
 ### Push aborts because secrets were detected
 
-The sanitiser found literal tokens or credentials in content that would otherwise be encrypted and committed. Sanitiser hits are intentionally a hard stop — they prevent the agent from leaking a secret into the vault, where it would persist even after subsequent pushes.
+The sanitiser found a literal token or credential in content that would otherwise be encrypted and committed. Sanitiser hits are intentionally a hard stop — a secret in git history persists even after later pushes and is one key-compromise away from retroactive exposure.
 
 Fix:
 
@@ -320,6 +320,27 @@ Fix:
 
 Do not bypass this by editing the vault manually.
 
+#### What the scan actually covers
+
+Be precise about the guarantee. The scan is a **known-credential-format** detector, not a general secret scanner. It matches a fixed set of high-precision patterns:
+
+- vendor API-key prefixes (`sk-ant-…`, `sk-proj-…`), GitHub (`ghp_…`, `github_pat_…`), GitLab (`glpat-…`), AWS access keys (`AKIA…`), Google (`AIza…`), Slack (`xox[abprs]-…`);
+- AgentSync's own age identity (`AGE-SECRET-KEY-1…`);
+- PEM private-key headers (`-----BEGIN … PRIVATE KEY-----`);
+- JWTs (`eyJ….eyJ….…`) — only when `security.secretScan = "strict"`.
+
+What it does **not** catch: a plain password, a bespoke or internal API token, a database connection string, or any credential with no recognised shape. Those flow into the (encrypted) vault unflagged. **Encryption is the real protection** — the scan exists only to keep well-known credentials out of git history. Treat a clean push as "no recognised credential format found", not "no secrets present".
+
+#### Tuning the scan
+
+`agentsync config set security.secretScan <mode>`:
+
+- `standard` (default) — the built-in credential patterns above, minus JWTs.
+- `strict` — adds JWT detection. Use when no legitimate JWT appears in your config.
+- `off` — disables the artefact-body scan. **Skill-bundle interiors are still scanned at `standard`** as a fail-safe, and encryption still applies.
+
+`agentsync config set security.allowSecretValues '["<literal>"]'` exempts a specific value the scanner false-positives on (and exempts it from base64 redaction). `agentsync config set security.redactBase64Values false` stops AgentSync replacing long base64-looking JSON values with a placeholder, for configs that legitimately store such values. See [config](commands.md#config).
+
 ### Daemon is not running
 
 Check, in order:

src/agents/claude/__tests__/sanitize.test.ts

@@ -57,4 +57,25 @@ describe("claude-sanitize", () => {
     };
     expect(out.hooks.PreToolUse[0]?.command).toBe(`${home}/runner`);
   });
+
+  // ─── Secret policy threading ──────────────────────────────────────────────
+  // Proves the adapter honours the SecretPolicy it is handed, not just the
+  // default — guards against an adapter silently dropping the policy argument.
+
+  test("sanitizeClaudeMcp redacts a secret value under the default policy", () => {
+    const raw = JSON.stringify({ mcpServers: { x: { env: { TOKEN: `ghp_${"a".repeat(36)}` } } } });
+    const out = JSON.parse(sanitizeClaudeMcp(raw, "").value) as {
+      mcpServers: { x: { env: { TOKEN: string } } };
+    };
+    expect(out.mcpServers.x.env.TOKEN).toContain("REDACTED");
+  });
+
+  test("sanitizeClaudeMcp leaves the secret unredacted when policy mode is off", () => {
+    const token = `ghp_${"a".repeat(36)}`;
+    const raw = JSON.stringify({ mcpServers: { x: { env: { TOKEN: token } } } });
+    const out = JSON.parse(
+      sanitizeClaudeMcp(raw, "", { mode: "off", allow: [], redactBase64: true }).value,
+    ) as { mcpServers: { x: { env: { TOKEN: string } } } };
+    expect(out.mcpServers.x.env.TOKEN).toBe(token);
+  });
 });

src/agents/claude/index.ts

@@ -2,6 +2,7 @@ import { homedir } from "node:os";
 import { AgentPaths } from "../../config/paths";
 import type { AgentSyncConfig } from "../../config/schema";
 import { denormalizeFromVault } from "../../core/path-portability";
+import { securityToPolicy } from "../../core/sanitizer";
 import {
   type ApplyPlan,
   defineFileArtifact,
@@ -28,6 +29,7 @@ export type ClaudeSnapshotResult = SnapshotResult;
 /** Collect Claude files that are safe to store in the encrypted vault. */
 export async function snapshotClaude(config: AgentSyncConfig): Promise<SnapshotResult> {
   const syncPlugins = config.claudePlugins?.syncPlugins ?? false;
+  const policy = securityToPolicy(config.security);
   const artifacts: SnapshotArtifact[] = [];
   const warnings: string[] = [];
 
@@ -40,7 +42,7 @@ export async function snapshotClaude(config: AgentSyncConfig): Promise<SnapshotR
 
   const settingsJson = await readIfExists(AgentPaths.claude.settingsJson);
   if (settingsJson !== null) {
-    const hooks = sanitizeClaudeHooks(settingsJson, homedir());
+    const hooks = sanitizeClaudeHooks(settingsJson, homedir(), policy);
     artifacts.push(
       collect(hooks, AgentPaths.claude.settingsJson, "claude/settings.hooks.json.age"),
     );
@@ -49,7 +51,7 @@ export async function snapshotClaude(config: AgentSyncConfig): Promise<SnapshotR
 
   const mcpJson = await readIfExists(AgentPaths.claude.mcpJson);
   if (mcpJson !== null) {
-    const mcp = sanitizeClaudeMcp(mcpJson, homedir());
+    const mcp = sanitizeClaudeMcp(mcpJson, homedir(), policy);
     artifacts.push(collect(mcp, AgentPaths.claude.mcpJson, "claude/claude.json.age"));
     warnings.push(...mcp.warnings);
   }

src/agents/claude/sanitize.ts

@@ -13,7 +13,12 @@
 
 import { homedir } from "node:os";
 import { normalizeForVault } from "../../core/path-portability";
-import { type RedactionResult, redactSecretLiterals } from "../../core/sanitizer";
+import {
+  DEFAULT_SECRET_POLICY,
+  type RedactionResult,
+  redactSecretLiterals,
+  type SecretPolicy,
+} from "../../core/sanitizer";
 
 /**
  * Keep only Claude hook settings and redact any embedded literal secrets.
@@ -24,11 +29,12 @@ import { type RedactionResult, redactSecretLiterals } from "../../core/sanitizer
 export function sanitizeClaudeHooks(
   rawSettingsJson: string,
   home: string = homedir(),
+  policy: SecretPolicy = DEFAULT_SECRET_POLICY,
 ): RedactionResult<string> {
   const parsed = JSON.parse(rawSettingsJson) as Record<string, unknown>;
   const hooksOnly = { hooks: parsed.hooks ?? {} };
   const normalized = normalizeForVault(hooksOnly, home);
-  const redacted = redactSecretLiterals(normalized, "hooks");
+  const redacted = redactSecretLiterals(normalized, "hooks", policy);
   return {
     value: `${JSON.stringify(redacted.value, null, 2)}\n`,
     warnings: redacted.warnings,
@@ -42,11 +48,12 @@ export function sanitizeClaudeHooks(
 export function sanitizeClaudeMcp(
   rawClaudeJson: string,
   home: string = homedir(),
+  policy: SecretPolicy = DEFAULT_SECRET_POLICY,
 ): RedactionResult<string> {
   const parsed = JSON.parse(rawClaudeJson) as Record<string, unknown>;
   const mcpOnly = { mcpServers: parsed.mcpServers ?? {} };
   const normalized = normalizeForVault(mcpOnly, home);
-  const redacted = redactSecretLiterals(normalized, "mcpServers");
+  const redacted = redactSecretLiterals(normalized, "mcpServers", policy);
   return {
     value: `${JSON.stringify(redacted.value, null, 2)}\n`,
     warnings: redacted.warnings,

src/agents/codex/index.ts

@@ -3,7 +3,13 @@ import * as TOML from "@iarna/toml";
 import { AgentPaths } from "../../config/paths";
 import type { AgentSyncConfig } from "../../config/schema";
 import { denormalizeFromVault, normalizeForVault } from "../../core/path-portability";
-import { type RedactionResult, redactSecretLiterals } from "../../core/sanitizer";
+import {
+  DEFAULT_SECRET_POLICY,
+  type RedactionResult,
+  redactSecretLiterals,
+  type SecretPolicy,
+  securityToPolicy,
+} from "../../core/sanitizer";
 import {
   type ApplyPlan,
   defineFileArtifact,
@@ -30,7 +36,11 @@ export type CodexSnapshotResult = SnapshotResult;
  * Using TOML parse → redact → stringify avoids the line-level regex approach which
  * misses multi-line values and nested tables.
  */
-function sanitizeCodexConfig(raw: string, home: string = homedir()): RedactionResult<string> {
+function sanitizeCodexConfig(
+  raw: string,
+  home: string = homedir(),
+  policy: SecretPolicy = DEFAULT_SECRET_POLICY,
+): RedactionResult<string> {
   const warnings: string[] = [];
   let parsed: TOML.JsonMap;
   try {
@@ -41,7 +51,7 @@ function sanitizeCodexConfig(raw: string, home: string = homedir()): RedactionRe
   }
 
   const normalized = normalizeForVault(parsed as unknown, home);
-  const redacted = redactSecretLiterals(normalized, "codex_config");
+  const redacted = redactSecretLiterals(normalized, "codex_config", policy);
   warnings.push(...redacted.warnings);
   return {
     value: TOML.stringify(redacted.value as TOML.JsonMap),
@@ -50,7 +60,8 @@ function sanitizeCodexConfig(raw: string, home: string = homedir()): RedactionRe
 }
 
 /** Collect Codex instructions, rules, and config that are safe to sync. */
-export async function snapshotCodex(_config?: AgentSyncConfig): Promise<SnapshotResult> {
+export async function snapshotCodex(config?: AgentSyncConfig): Promise<SnapshotResult> {
+  const policy = securityToPolicy(config?.security);
   const artifacts: SnapshotArtifact[] = [];
   const warnings: string[] = [];
 
@@ -69,7 +80,7 @@ export async function snapshotCodex(_config?: AgentSyncConfig): Promise<Snapshot
 
   const configToml = await readIfExists(AgentPaths.codex.configToml);
   if (configToml !== null) {
-    const sanitized = sanitizeCodexConfig(configToml, homedir());
+    const sanitized = sanitizeCodexConfig(configToml, homedir(), policy);
     artifacts.push(collect(sanitized, AgentPaths.codex.configToml, "codex/config.toml.age"));
     warnings.push(...sanitized.warnings);
   }

src/agents/cursor/index.ts

@@ -4,7 +4,7 @@ import { type ParseError, parse as parseJsonc } from "jsonc-parser";
 import { AgentPaths } from "../../config/paths";
 import type { AgentSyncConfig } from "../../config/schema";
 import { denormalizeStringFromVault, normalizeStringForVault } from "../../core/path-portability";
-import { sanitizeAndNormalizeJson } from "../../core/sanitizer";
+import { sanitizeAndNormalizeJson, securityToPolicy } from "../../core/sanitizer";
 import {
   type ApplyPlan,
   defineFileArtifact,
@@ -70,7 +70,8 @@ function validateCursorRuleName(ruleName: string): void {
 }
 
 /** Collect Cursor rules, MCP config, and commands that are safe to sync. */
-export async function snapshotCursor(_config?: AgentSyncConfig): Promise<SnapshotResult> {
+export async function snapshotCursor(config?: AgentSyncConfig): Promise<SnapshotResult> {
+  const policy = securityToPolicy(config?.security);
   const artifacts: SnapshotArtifact[] = [];
   const warnings: string[] = [];
 
@@ -86,7 +87,7 @@ export async function snapshotCursor(_config?: AgentSyncConfig): Promise<Snapsho
 
   const mcpRaw = await readIfExists(AgentPaths.cursor.mcpGlobal);
   if (mcpRaw !== null) {
-    const sanitized = sanitizeAndNormalizeJson(mcpRaw, "cursor_mcp");
+    const sanitized = sanitizeAndNormalizeJson(mcpRaw, "cursor_mcp", homedir(), policy);
     artifacts.push(collect(sanitized, AgentPaths.cursor.mcpGlobal, "cursor/mcp.json.age"));
     warnings.push(...sanitized.warnings);
   }

src/agents/vscode/index.ts

@@ -2,7 +2,7 @@ import { homedir } from "node:os";
 import { AgentPaths } from "../../config/paths";
 import type { AgentSyncConfig } from "../../config/schema";
 import { denormalizeStringFromVault } from "../../core/path-portability";
-import { sanitizeAndNormalizeJson } from "../../core/sanitizer";
+import { sanitizeAndNormalizeJson, securityToPolicy } from "../../core/sanitizer";
 import { type ApplyPlan, defineFileArtifact, makeApplyVault } from "../_apply";
 import {
   atomicWrite,
@@ -16,13 +16,14 @@ import {
 export type VsCodeSnapshotResult = SnapshotResult;
 
 /** Collect the VS Code MCP configuration that AgentSync manages. */
-export async function snapshotVsCode(_config?: AgentSyncConfig): Promise<SnapshotResult> {
+export async function snapshotVsCode(config?: AgentSyncConfig): Promise<SnapshotResult> {
+  const policy = securityToPolicy(config?.security);
   const artifacts: SnapshotArtifact[] = [];
   const warnings: string[] = [];
 
   const mcpRaw = await readIfExists(AgentPaths.vscode.mcpJson);
   if (mcpRaw !== null) {
-    const sanitized = sanitizeAndNormalizeJson(mcpRaw, "vscode_mcp");
+    const sanitized = sanitizeAndNormalizeJson(mcpRaw, "vscode_mcp", homedir(), policy);
     const artifact = collect(sanitized, AgentPaths.vscode.mcpJson, "vscode/mcp.json.age");
     artifacts.push(artifact);
     warnings.push(...sanitized.warnings);