docs: sync public docs to post-July harness state and capture value-proof gaps plan - #41
docs: sync public docs to post-July harness state and capture value-proof gaps plan#41noodlemind wants to merge 2 commits into
Conversation
|
@coderabbitai review |
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
📝 WalkthroughWalkthroughThe PR updates project and architecture documentation for the July 2026 hardening wave. It documents memory, context budgets, token telemetry, deterministic evaluations, harness commands, and a plan for remaining value-proof gaps. ChangesHardening documentation
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
✅ Action performedReview finished.
|
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/plans/2026-08-02-feat-harness-value-proof-gaps-plan.md`:
- Line 53: Update the AC4 baseline wording in the plan to reflect the actual
enforcement mode: use “enforcement: enforce” if the baseline is fail-closed, or
explicitly identify the separate boundary meant by “fail-closed” while retaining
“enforcement: warn.” Keep the documented adopt-or-decline decision and rationale
requirement unchanged.
In `@README.md`:
- Around line 98-101: Update the opening fenced code block surrounding the
directory listing in README.md to use the text language identifier, changing the
bare fence to a text fence while preserving the listing contents and closing
fence.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: e1e3238b-bc68-4ce2-846e-844324f6400a
📒 Files selected for processing (5)
AGENTS.mdREADME.mddocs/architecture/engineer-harness.mddocs/architecture/skill-driven-prompt-library.mddocs/plans/2026-08-02-feat-harness-value-proof-gaps-plan.md
08f289d to
610b774
Compare
610b774 to
59ca6a3
Compare
Why
An external review of the harness evaluated the public docs surface (README, architecture docs, AGENTS.md) as it stood before the July hardening wave (PRs #33–#37) and concluded the system was token-heavy, unmeasured, and Copilot-bound — none of which holds at HEAD. The docs lagged reality; this PR closes that lag and captures the four gaps that survived the reconciliation.
What
Commit 1 — docs sync (4 files):
README.md: memory-model headline updated to the episodic/semantic/behavioral tiers with a link todocs/MEMORY-MODEL.md; new "Measurement, budgets, and evals" section (CI-enforced 2 KB orient pack + ~900-token engineer budget,harness reporthost token metering,evals/suite); knowledge-compounding table gains the consolidated-learnings tier; directory structure addspackages/harness/,evals/,docs/MEMORY-MODEL.md.docs/architecture/engineer-harness.md: newharness reporttelemetry paragraph; concrete budget number on the VS Code evidence row; freshness marker.docs/architecture/skill-driven-prompt-library.md: "Review Summary" rewritten from proposal voice to shipped-state "Standardization Summary";@dev-kit/harnesscommand list updated to the full shipped catalog;## Memory Cardsadded to the plan-section list (aligns with_plan-template.md); freshness marker.AGENTS.md: budgets/report/evals/memory-model line; directory structure addspackages/,evals/,docs/MEMORY-MODEL.md; freshness marker.Commit 2 — plan capture:
docs/plans/2026-08-02-feat-harness-value-proof-gaps-plan.md: unlocked capture (status: open,phase: 0) of the four adopted gaps — knowledge net-benefit measurement (corpus-gated), session-level compaction, correction-effort metric, verify-tier decide-and-document — plus the reviewer's standing offer to pre-review the release-eval methodology before Terminal-Bench trials run. Each gap locks via its own planning pass before implementation.Verification
harness validate-planon the new plan: pass, exit 0 (schema v1, criterion mappings, configured checks all green)npm --prefix packages/harness test: 655/655 passSummary by CodeRabbit