feat: K5/K6 computed from a real workbook, not read by the model - #99
Merged
Conversation
Point 1 of the trz-expert evaluation flagged that the model recomputes all 86 checks by prose reasoning on every audit, and that its measured accuracy is worst on group K - a hand-typed total (K5) and unrounded accruals (K6) are exactly the multi-step arithmetic a language model miscounts. tools/k_checker.py is a small pilot, not the full move suggested in that review: it automates only K5 and K6, because those two ask nothing about any column but the one being checked. The other six K checks (K1, K3, K4, K7, K8) each need either the full accrual-column universe, a number outside the workbook, or the neighbouring sheet - a generic tool cannot promise those without risking a false positive, which this project treats as no better than a miss. It reuses tools/preflight.py's column resolution (Mapping, analyse()) rather than a second one, and stays outside skills/trz-expert on the same grounds preflight.py already does: SKILL.md promises prose only, and a plugin install copies the skill directory whole. test/k_checker_test.py proves it the way the shape suite does: a clean workbook is silent, and each of K5 and K6 is found exactly once with nothing else raised. Confirmed it has teeth by breaking ROUND_EPS and watching the K6 case go red before restoring it. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XvbzEbQwxwxxHAtXfPMrno
svedbg
force-pushed
the
feat/k-checker-pilot
branch
from
September 6, 2026 06:35
9c2bb9c to
8f2a434
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Small pilot for point 1 of the trz-expert evaluation ("the model rewrites 86 checks as prose on every audit; measured accuracy is worst on the multi-step arithmetic in group K"). Rather than the full move to
scripts/+stavki.yamlthat review sketched, this automates only the two group-K checks that are safe for a generic tool to claim:Both ask nothing about any column but the one being checked, so they carry no false-positive risk regardless of what else the file contains. The other six K checks (K1, K3, K4, K7, K8, K9/K10) each need either the full accrual-column universe, a number outside the workbook, or a second sheet/file — a generic tool can't promise those safely, and a confident false positive is treated the same as a miss in this repo (
CONTRIBUTING.md).tools/k_checker.pysits outsideskills/trz-experton purpose, for the same reasontools/preflight.pydoes:SKILL.mdpromises prose only, and a plugin install copies the skill directory whole. It reusespreflight.py'sMapping/analyse()column resolution instead of a second copy.Test plan
python test/k_checker_test.py— clean workbook silent, K5 and K6 each found exactly once, nothing else raisedROUND_EPS, watched the K6 case go red, restored itpython test/skill_test.py,python test/run_tests.py— unaffected, still greentest/vedomost_05_2026.xlsx) — correctly silent🤖 Generated with Claude Code
https://claude.ai/code/session_01XvbzEbQwxwxxHAtXfPMrno