feat: audit.py computes I1, I5 (narrow), B1, B4, B5 and F5 from a real workbook - #113
Merged
Conversation
…l workbook Point 1 of the trz-expert evaluation: the model rewrites all 86 checks as prose every audit, and its measured accuracy is worst on the multi-step arithmetic in groups B, F and I - exactly the checks this script covers, mechanically, the same way k_checker.py already covers K5/K6. Two new files: - scripts/rates.py reads every rate audit.py needs fresh from references/stavki.md + references/stavki/*.md at call time, never hardcoded - the same "no rate from memory" rule applied to a script instead of the model's prose. Period-scoped lookups (minimum wage, maximum insurable income) match a table row's own date range rather than assuming a specific year, so a period the reference has no row for returns None instead of reusing the nearest one. - scripts/audit.py: I1 (БРУТО minus contributions minus tax minus deductions chains to НЕТО), I5 narrow (sick pay accrued with zero sick days - the fuller "day columns sum to the month's calendar norm" needs a public-holiday calendar this script doesn't have), B1/B5 (main wage / insurable income against the minimum wage), B4 (insurable income against the maximum, both over the correct cap and copied from the neighbouring half-year's), F5 (ТЗПБ extracted algebraically from the employer's total contributions, compared against the mapping's declared rate). Left to the model, for the same reason K1/K3/K4/K7 are left to k_checker.py: B2/B3/B6 need a company-specific number mapping.yaml doesn't carry; F1/F2/F3/F4/F6/F7/F9's composition/relief method needs a judgment call this script isn't positioned to make safely yet; F8/F10 need information one workbook doesn't hold; K1/K3/K4/K7/K8 are k_checker.py's own closed-vocabulary risk, unchanged. Testing this against test/generate_wide.py at scale, not only a hand-built fixture, found three real defects before anyone relied on any of them: 1. preflight.py's CONCEPTS already collapsed "НЕТО преди удръжки" with "НЕТО за изплащане", and "Вноски работодател ДОО+ТЗПБ" with "Вноски работодател общо" - raising the blocking DUPLICATE_CONCEPT signal on every realistic generated fixture and stopping every check in this file cold. Split now, the same way "Клас %"/"Клас сума" already were. 2. B1/B5 compared against the full minimum wage with no notion of part-time work - test/vedomost_05_2026.xlsx's own documented part-time row (Стефка Ангелова, 4 hours against 8) caught this directly, flagged as a violation when expected_findings.md says explicitly it must raise nothing. 3. Fixing that exposed the same gap for partial-month attendance: a row that took leave or sick days has a legitimately prorated "основна", and generate_wide.py's own random leave/sick days produced 159 false B1 findings at 200 seeds once checked at scale. Neither script has a public-holiday calendar to compute the true month norm, so both gates use the highest hours/days count declared on the sheet as the least-wrong stand-in for full-time/full-month - tried the modal (most common) value first and rejected it: in a small office where nobody that month has full attendance, the mode is itself a partial value. Nine new preflight.py CONCEPTS entries from the previous commit made this testable at all - the composition vocabulary they were added for is not yet used by anything in this file. Wired into CI (both the `validate` job at 300 seeds and `deep-suites` at 3000), the pre-commit hook (which now also runs it whenever skills/trz-expert/scripts/ changes), and CLAUDE.md. Verified: I1 106/106, B4 70/70 (both shapes), F5 94/94 across 200 seeds, zero false positives on B1/B5 across 300 seeds, exact match against expected_findings.md's suite-1 fixture including the part-time row. Confirmed the test has teeth by breaking TOL to 50.0 and watching 24 checks fail before restoring it. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XvbzEbQwxwxxHAtXfPMrno
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Point 1 of the trz-expert evaluation: the model rewrites all 86 checks as prose every audit, measured weakest on the multi-step arithmetic in groups B, F and I. This covers the subset that's mechanical and safe for a generic tool — the same discipline
k_checker.pyalready applies to K5/K6.scripts/rates.py— every rate read fresh fromreferences/stavki.md+references/stavki/*.mdat call time, never hardcoded. Period lookups match a table row's own date range, so a period the reference has no row for returnsNonerather than reusing the nearest one.scripts/audit.py— I1 (vertical net reconciliation), I5 narrow (sick pay with zero sick days), B1/B5 (minimum wage), B4 (maximum insurable, both over-cap and copied-from-wrong-half-year), F5 (ТЗПБ extracted algebraically, compared against the mapping's declared rate).Left to the model, same reasoning as
k_checker.py's K1/K3/K4/K7: B2/B3/B6 need a company-specific numbermapping.yamldoesn't carry; F1/F2/F3/F4/F6/F7/F9's composition method needs a judgment call not safe yet; F8/F10 need information one workbook can't hold.Real defects found by testing against
generate_wide.pyat scale (not just a hand fixture)preflight.pyvocabulary bug: "НЕТО преди удръжки"/"НЕТО за изплащане" and "Вноски работодател ДОО+ТЗПБ"/"Вноски работодател общо" each collapsed into one concept, raising the blockingDUPLICATE_CONCEPTsignal on every realistic fixture — stopping every check in this file cold before it could even run. Split now.test/vedomost_05_2026.xlsx's own documented part-time row (4h vs 8h, explicitly "must produce no finding" inexpected_findings.md) was flagged as a violation.Test plan
generate_wide.pyseedsexpected_findings.md's suite-1 fixture, including the part-time row staying silentTOLto 50.0, watched 24 checks fail, restored itvalidateat 300 seeds,deep-suitesat 3000), pre-commit hook,CLAUDE.md🤖 Generated with Claude Code
https://claude.ai/code/session_01XvbzEbQwxwxxHAtXfPMrno