Repository navigation
test(security): cross-check the TS lens against eslint-plugin-security - #72
Merged
Merged
Conversation
Closes the validation asymmetry M4 left behind. The Python lens was measured against Bandit at 27/27; TypeScript had only planted fixtures. This measures it against eslint-plugin-security — the JS counterpart to Bandit, and again a tool this project does not control, so the check cannot flatter itself. Scope is restricted the same way: eslint.config.mjs enables exactly the three rules inside the lens's claims — detect-eval-with-expression, detect-child-process and detect-non-literal-require. The plugin's other rules (unsafe regex, object injection, timing attacks, fs filenames) are left off, because the lens does not claim them and counting them would measure a promise never made. Measured across shelljs, cross-spawn, rechoir, liftoff, open and resolve: 10/10 in-scope recall, no eslint-only findings. The expected asymmetries are documented rather than hidden: eslint flags eval only with a computed argument while the lens flags every eval (matching Bandit), and eslint has no rule for new Function, the vm module or string-bodied timers, which the lens catalogues. Worth recording: the harness initially reported 6/6, and it was wrong. Reading eslint's output with the platform locale hit a cp1252 decode failure on Windows that silently dropped findings, turning four real agreements on `resolve` into apparent lens-only divergences. Decoding UTF-8 explicitly fixed it and the true figure is 10/10. A validation harness that fails quietly is worse than none, so the cause is noted in the README. node_modules/ is gitignored; package.json and package-lock.json are committed so the toolchain and scan targets are reproducible. Suite 25/25; benchmark 9/20. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
MODScan extension-point diffNo breaking changes to extension points |
MODScan attack-surface diff —
|
Rinkia
added a commit
that referenced
this pull request
Jul 23, 2026
Bump to 0.1.7 and move the [Unreleased] entries under [0.1.7]. Ships the Java front-end, the security lens over MCP and for TypeScript/JavaScript, the process sinks the Bandit cross-check exposed, and two TypeScript front-end fixes. Also adds the two CHANGELOG entries that were missing: recognising CommonJS exports (#73) and no longer scanning .d.ts declaration files (#76). Both change what `--language typescript` returns, so both belong in a user-facing changelog. The cross-check harnesses (#70, #72) and benchmark targets (#74, #78) stay out: they are repository artifacts, not shipped in the wheel. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Closes the validation asymmetry M4 (#71) left behind. Python was measured against Bandit at 27/27; TypeScript had only planted fixtures. This measures it against eslint-plugin-security — the JS counterpart to Bandit, and again a tool this project doesn't control.
Result
Measured across shelljs, cross-spawn, rechoir, liftoff, open, resolve:
Same scoping discipline as the Bandit check
js/eslint.config.mjsenables exactly the three rules inside the lens's claims —detect-eval-with-expression(code_exec),detect-child-process(process),detect-non-literal-require(dynamic_load). The plugin's other rules (unsafe regex, object injection, timing attacks, fs filenames, pseudo-random bytes) are off: the lens doesn't claim them, so counting them would measure a promise never made.Documented asymmetries (not defects):
evalonly with a computed argument; the lens flags everyeval, matching Bandit's Python stance.new Function,vm.*, or string-bodied timers — lens-only coverage.detect-child-processcoversexecbut notspawn/execFileSync; the lens covers the family, split shell vs no-shell.A bug the harness found in itself — worth reading
The first run reported 6/6. It was wrong. Reading eslint's stdout with the platform locale hit a cp1252 decode failure on Windows that silently dropped findings — turning four real agreements on
resolveinto apparent "lens-only" divergences, and inflating the recall ratio by shrinking the denominator.Fixed by decoding UTF-8 explicitly; true figure is 10/10. Recorded in the README, because a validation harness that fails quietly is worse than no harness at all.
Test plan
node_modules/gitignored and confirmed not staged;package.json+package-lock.jsoncommitted for reproducibility..py; printed output is ASCII.Both lenses now have an external authority: Python 27/27 vs Bandit, TS 10/10 vs eslint-plugin-security.
🤖 Generated with Claude Code