refine(bench): complete Codex agentic benchmark checkpoint

- unify Claude and Codex structural/agentic contracts with fail-closed runtime, provenance, and generated-manifest checks
- publish the validated structural and 48-cell agentic evidence, including synchronized result tables and historical caveats
- release codemap-py 0.28.5 with deterministic packaging and cross-platform test isolation

---

Co-authored-by: Codex <codex@openai.com>

Author: Borda

.gitattributes

@@ -12,6 +12,12 @@
 *.yml  text eol=lf
 *.toml text eol=lf
 
+# Codemap launchers include extensionless Python, POSIX-shell, and Windows-batch
+# files whose exact bytes are part of generated benchmark manifest identities.
+plugins/codemap-py/bin/* text eol=lf
+plugins/codemap-py/LICENSE text eol=lf
+plugins/codemap-py/NOTICE  text eol=lf
+
 # Extensionless Codex Rig package text participates in exact hash identity.
 plugins/codex-rig/LICENSE text eol=lf
 plugins/codex-rig/NOTICE  text eol=lf

.github/workflows/ci-tests.yml

@@ -10,6 +10,10 @@ on:
       - "pyproject.toml"
       - "sync.sh"
 
+defaults:
+  run:
+    shell: bash
+
 jobs:
   tests:
     runs-on: ${{ matrix.os }}
@@ -37,8 +41,11 @@ jobs:
       - name: 📦 Install uv
         uses: astral-sh/setup-uv@v5
 
-      - name: 🧪 Run JS hook smoke tests
-        run: node tests/hooks/tmp-paths.test.js
+      - name: 🔒 Verify deterministic benchmark manifests
+        run: |
+          uv run --only-group test --python ${{ matrix.python-version }} python benchmarks/build-provider-parity-methodology-manifest.py --check
+          uv run --only-group test --python ${{ matrix.python-version }} python benchmarks/build-codex-integration-manifest.py --check
+          uv run --only-group test --python ${{ matrix.python-version }} python benchmarks/build-codex-agentic-manifest.py --check
 
       - name: 🧪 Run plugin tests (Python bin/ + JS hooks)
         run: uv run --only-group test --python ${{ matrix.python-version }} pytest -W error::DeprecationWarning

.pre-commit-config.yaml

@@ -23,8 +23,10 @@ repos:
     hooks:
       - id: end-of-file-fixer
         name: 📄 end-of-file-fixer
+        exclude: ^benchmarks/(manifests|suites|tests/fixtures)/.*\.json$
       - id: trailing-whitespace
         name: ✂️ trailing-whitespace
+        exclude: ^benchmarks/(manifests|suites|tests/fixtures)/.*\.json$
       - id: check-json
         name: 🔍 check-json
       - id: check-yaml
@@ -45,12 +47,13 @@ repos:
       - id: mixed-line-ending
         name: ↩️ mixed-line-ending
         args: ["--fix=lf"]
+        exclude: ^benchmarks/(manifests|suites|tests/fixtures)/.*\.json$
       - id: pretty-format-json
         name: ✨ pretty-format-json
         args: ["--autofix"]
-        # Benchmark manifests and suites are identity-locked inputs/outputs. Their
-        # dedicated builders own deterministic serialization and hash validation.
-        exclude: ^benchmarks/(manifests|suites)/.*\.json$
+        # Benchmark manifests, suites, and historical fixtures are identity-locked
+        # inputs/outputs. Their dedicated builders/tests own byte and hash validation.
+        exclude: ^benchmarks/(manifests|suites|tests/fixtures)/.*\.json$
 
   - repo: https://github.com/JoC0de/pre-commit-prettier
     rev: v3.9.4 # using tag; previously pinned SHA when tags were not persistent
@@ -237,6 +240,18 @@ repos:
         # runtime dirs are gitignored and excluded from the pattern by not matching.
         files: ^plugins/[^/]+/(agents|skills|rules|hooks)/.*\.(md|js)$|^plugins/[^/]+/\.claude-plugin/[^/]+\.json$|^\.claude/(agents|skills|rules)/.*\.md$|^\.claude/hooks/[^/]+\.js$|^\.claude/settings\.json$|^\.codex/(agents|skills|rules)/.*\.md$|^\.codex/hooks/[^/]+\.js$|^\.codex/settings\.json$
 
+      - id: no-literal-tmp-in-hooks
+        name: 🌡️ No literal /tmp/ in hook source
+        language: pygrep
+        # Hooks must derive their temp base (os.tmpdir(), ${TMPDIR:-/tmp}, tempfile) rather than
+        # hardcode /tmp/ — see cc_foundry/rules/claude-config.md §TMPDIR Sentinel Scoping. Only a
+        # quoted or template-literal /tmp/ is a real path constant; prose mentions inside comments
+        # (commit-guard.js documents its sentinel path) carry no quote and stay unflagged.
+        entry: '["''`]/tmp/'
+        # Replaces tests/hooks/tmp-paths.test.js, which checked a hardcoded 6-file cc_foundry list
+        # and missed the other 16 plugin hooks; this glob picks up new hooks automatically.
+        files: ^plugins/[^/]+/hooks/.*\.(js|py)$|^\.claude/hooks/.*\.(js|py)$|^\.codex/hooks/.*\.(js|py)$
+
   - repo: https://github.com/jorisroovers/gitlint
     rev: v0.19.1 # run `pre-commit autoupdate --repo https://github.com/jorisroovers/gitlint` to set
     hooks:

README.md

@@ -383,36 +383,28 @@ Without the plugin: pre-pass review is skipped gracefully (skills check with `cl
 
 → Install instructions: [rtk-ai/rtk](https://github.com/rtk-ai/rtk)
 
+> [!NOTE]
+>
+> RTK only compresses **Bash tool output** — shell commands like `git`, `cargo`, `pytest`, etc. It does not affect Claude Code's native tools (Read, Grep, Glob, Edit, Write), which run inside Claude's own engine and are already token-efficient by design.
+
 ### Codex CLI plugin
 
 [openai/codex-plugin-cc](https://github.com/openai/codex-plugin-cc) connects the Codex CLI to Claude Code as a local plugin — enabling the cross-validation, mechanical delegation, and diff pre-pass described in [Claude + Codex Integration](#-claude--codex-integration).
 
 → Install: `/plugin marketplace add openai/codex-plugin-cc` → `/plugin install codex@openai-codex` → `/reload-plugins`
 
-> [!NOTE]
->
-> RTK only compresses **Bash tool output** — shell commands like `git`, `cargo`, `pytest`, etc. It does not affect Claude Code's native tools (Read, Grep, Glob, Edit, Write), which run inside Claude's own engine and are already token-efficient by design.
-
 ### cc-Lens
 
 [cc-Lens](https://github.com/Arindam200/cc-lens) is a local analytics dashboard for Claude Code — token/cost trends, tool usage breakdowns, session replay. Reads `~/.claude/` directly, no cloud, no data leaves the machine.
 
 → Run: `npx cc-lens` — no install required
 
-### Colab-MCP
+### Colab-MCP (active testing)
 
 [colab-mcp](https://github.com/googlecolab/colab-mcp) connects Google Colab as a remote GPU executor. Pre-configured in `.mcp.json` (disabled by default) — used by `/research:run --colab` to offload metric-improvement iterations to a cloud GPU without a local CUDA setup. Supports hardware selection: `--colab=H100`, `--colab=L4`, `--colab=T4`, `--colab=A100`.
 
 → Enable: add `"colab-mcp"` to `enabledMcpjsonServers` in `settings.local.json`
 
-### Semble (semantic code search)
-
-[semble](https://github.com/MinishLab/semble) runs a local MCP server that adds hybrid semantic + lexical search across any repo. When available, the `develop` and `oss` skills automatically expose `mcp__semble__search` to agents as a gap-fill tool — used when the codemap index is non-exhaustive. No cloud, no API key; runs fully local via `uvx`.
-
-→ Install (global, all projects): `claude mcp add semble -s user -- uvx --from "semble[mcp]" semble`
-
-→ Install (this project only): `claude mcp add semble -s project -- uvx --from "semble[mcp]" semble`
-
 ### Caveman
 
 [caveman](https://github.com/JuliusBrussee/caveman) makes Claude respond in compressed "caveman speak" — cutting ~75% of output tokens while retaining full technical accuracy. Adjustable intensity levels (lite → full → ultra → 文言文) and a compression tool that also cuts ~46% of input tokens per session.