Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -358,6 +358,14 @@ one run per adapter against them. Wave M1's three rows are complete:
cases are `score_tier: "modeling"`, are never in a core denominator, and are
never averaged with a kernel score.

The [tool-native model profile](docs/native-profile.md) shares that tier and
inverts the question: not what an engine can be told, but what each tool
*ships*. Six templates over real platform APIs, per language, scored on their
own profile and never pooled with the benchmark-controlled matrix — a native
miss is a coverage fact about a product, and a controlled miss is an accuracy
fact about an engine. Wave N1's first row is
[the JavaScript tool-native probe set](docs/javascript-native.md).

The [Java kernel contract](docs/java-kernel.md) records the origin population
and the first landed wave of the [challenge-tier
expansion](docs/challenge-tier.md), which grows the Java core to 29 templates
Expand Down
28 changes: 28 additions & 0 deletions adapters/bifrost/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -653,3 +653,31 @@ verbatim with the document's own rationale.
(`reports/raw/load-bearing-java-modeling/bifrost-declared-source-{with,without}-model.json`).

See [the Java modeling matrix](../../docs/java-modeling.md).

## JavaScript tool-native probe set

Zero of six, decided before the CLI is invoked — and the twelve assertions were
produced without launching Bifrost at all.

The tool-native activation contract is built-in policy packs only:
`--policy-pack` / `--policy-category` / `--policy-id` over the catalog
`--list-policies` prints. A native run may not pass `--policy-file`, which is
how every benchmark-controlled Bifrost run in this repository supplies its
models, and the no-benchmark-models gate refuses one.

The standalone policy CLI ships no taint policy and no source or sink endpoint
set. BrokkAi/bifrost-dev **#2620** is the open issue under which shipped
endpoints — its own candidate inventory names `System.getenv`, `Runtime.exec`,
`ProcessBuilder` — would first exist at all, and **#2691** is the standalone-CLI
activation surface for external procedure summaries that this profile would
need. **#1871** (closed) supplies summaries, not endpoints. Without a source and
a sink, no cell in this profile can produce a finding, which is why every
template reads the same way, and the sanitizer and summary categories are
additionally declined for the surface gaps this README already records above.

Stating that position in a preregistration published by Bifrost's own vendor,
before a run, with the vendor's open issues named, is the point. It is capability
coverage: the twelve `unsupported` assertions are never negatives and reduce no
denominator.

See [the JavaScript tool-native probe set](../../docs/javascript-native.md).
41 changes: 41 additions & 0 deletions adapters/codeql/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -961,3 +961,44 @@ CodeQL does not follow `Opaque.class.getMethod(…).invoke(…)` on its own —
on the same fixture, [Joern does](../joern/README.md#taint-modeling-matrix).

See [the Java modeling matrix](../../docs/java-modeling.md).

## JavaScript tool-native probe set

A different profile, not a different population of the same one. Everything
above supplies CodeQL its models; this run supplies **none**. The activation is
the shipped query suite and one documented CLI option, and the
no-benchmark-models gate reads the invocation shape before the binary is touched
and refuses a run that names any benchmark-authored artifact.

- Activation:
`codeql/javascript-queries@2.4.4:codeql-suites/javascript-security-extended.qls`
with `--threat-model=local`. No adapter query, no data extension, no
`--additional-packs` model of ours. 103 rules resolved for the JavaScript
fixtures.
- `--threat-model=local` configures shipped models rather than adding any: it
enables the vendor's `local` group, which
`codeql/threat-models@1.0.55` defines as containing `environment` and
`commandargs`. Without it, templates 1, 5, and 6 would be decided by the
default `remote`-only threat model and would miss for a reason unrelated to
coverage.
- **The library resolution deliberately differs from this adapter's.** The
pinned *query* pack bundles `codeql/javascript-all@2.10.0`; every
benchmark-controlled run above pins `javascript-all@2.9.0`. A native run must
measure the shipped product as shipped, which is one more reason the two
profiles are never pooled.
- Database creation is byte-for-byte the benchmark-controlled path —
`codeql_database_create_args` under `CodeqlLanguage::Javascript` — because
extraction is a property of the language, not of the model profile. Only the
analyze step differs.
- Reconciliation anchors sit on the platform callsite rather than on a declared
entity, because this profile has no declared entity. A shipped suite answers
many questions at once, so a finding away from the anchor is retained as a
diagnostic and never becomes evidence of a flow; it did not arise in this run.
- Result: **10 of 12**, with a false positive on the sanitizer negative
(`encodeURIComponent` is a barrier for XSS and request forgery but a
taint-preserving step for command injection) and both cells wrong on the
persistence template (the `process.env` write/read pair is unlinked, while the
plain environment source fires on the distinct-key read). Both were
preregistered as expectations before the run.

See [the JavaScript tool-native probe set](../../docs/javascript-native.md).
29 changes: 29 additions & 0 deletions adapters/joern/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1029,3 +1029,32 @@ its commentary lives in [the Java modeling matrix](../../docs/java-modeling.md).
rather than a scored cell decided by a missing model.

See [the Java modeling matrix](../../docs/java-modeling.md).

## JavaScript tool-native probe set

Zero of six, decided before the distribution is invoked — and the twelve
assertions were produced without launching Joern at all.

The tool-native activation contract is whatever the distribution activates
without a user-authored query or semantics file: `DefaultSemantics`, and nothing
else. No benchmark `.semantics` file may load, and the no-benchmark-models gate
refuses `adapters/joern/semantics/model-*.semantics` and the shared
`adapters/joern/queries/modeling.sc`.

`DefaultSemantics` is a table of **flow constraints** — operator semantics, a
list of C standard-library functions, and a short list of JVM-ecosystem method
full names. It contains no source catalog and no sink catalog, and flow
semantics never say where taint starts or where its arrival is a finding. In
every other Joern population here the endpoints come from this adapter's own
query parameters, which is precisely what the native activation rule forbids.
`joern-scan` exists in the distribution but its query database does not: it
downloads `querydb.zip` from a floating `latest` release asset, unpinnable at
run time.

That is a statement about the OSS distribution's *product packaging*, not about
its engine — the benchmark-controlled matrix scores this same engine on sixteen
JavaScript assertions. Each of the twelve retained decisions carries the
preregistration's rationale verbatim with
`"evidence_kind": "retained-capability-decision"`.

See [the JavaScript tool-native probe set](../../docs/javascript-native.md).
37 changes: 37 additions & 0 deletions adapters/semgrep/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -752,3 +752,40 @@ Ten of the twenty-four cells are scored.
(`reports/raw/load-bearing-java-modeling/semgrep-declared-sink-{with,without}-model.json`).

See [the Java modeling matrix](../../docs/java-modeling.md).

## JavaScript tool-native probe set

This adapter's native activation is a **vendored snapshot**, not a registry
configuration: `--config p/…` is network-fetched and version-unpinnable at run
time, so two runs a week apart are two different rulesets under one name.

- Snapshot: `adapters/semgrep/native/javascript/` — thirty `.yaml` rule files
copied byte for byte from `javascript/lang/security/` at
`semgrep/semgrep-rules@40b8c63f75dc7c22c8a77482d73bfb864b146f7e`, with
`provenance.json` recording `kind: derived`, the upstream repository, the
source commit and its date, the paths, the license, and the retrieval date.
The runner refuses a native run whose activation directory lacks that file,
and it refused this one until the snapshot landed.
- **`taint_assume_safe_functions` is deliberately NOT set**, unlike every
benchmark-controlled Semgrep run here. There, a permissive default would
decide a cell the supplied model was meant to decide. Here the default *is*
the product: setting it would be editing the vendor's rules, which the
activation rule forbids. `--oss-only` still applies — the CE engine is the
product under test.
- Result: **0 of 6 scored, all six retained unsupported** by
[Amendment N1](../../docs/native-profile.md#n1--2026-08-27--semgrep-ces-javascript-cells-evaluated-against-the-vendored-snapshot).
Fifteen of the thirty rules are `mode: taint` and every one of them roots its
`pattern-sources` in a function parameter or a framework request object.
`detect-child-process.yaml` and `audit/dangerous-spawn-shell.yaml` bind
exactly this profile's `child_process.execSync` sink and take an enclosing
function's parameter as their only source; `audit/code-string-concat.yaml`
takes Express `$REQ.*` and the Next.js router. `process.env`,
`process.argv`, `encodeURIComponent`, and `Buffer.from` appear nowhere in the
snapshot. Semgrep declined this profile for JavaScript; it did not fail it,
and the twelve `unsupported` assertions reduce no denominator.
- The preregistered sink-existence hazard did not materialize here: the two
rules matching `execSync` are taint rules, and the fifteen pattern rules match
other constructs entirely, so a negative cell passing a clean local to
`execSync` is flagged by none of them.

See [the JavaScript tool-native probe set](../../docs/javascript-native.md).
1 change: 1 addition & 0 deletions adapters/semgrep/native/javascript/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Semgrep Rules License v1.0. For more details, visit https://semgrep.dev/legal/rules-license
50 changes: 50 additions & 0 deletions adapters/semgrep/native/javascript/provenance.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
{
"kind": "derived",
"language": "javascript",
"model_profile": "tool-native",
"upstream_repository": "https://github.com/semgrep/semgrep-rules",
"upstream_commit": "40b8c63f75dc7c22c8a77482d73bfb864b146f7e",
"upstream_commit_date": "2026-07-29T19:46:44-07:00",
"upstream_paths": [
"javascript/lang/security"
],
"upstream_license": "Semgrep Rules License v1.0",
"upstream_license_url": "https://semgrep.dev/legal/rules-license",
"upstream_license_file": "LICENSE",
"retrieved_at": "2026-08-27",
"vendored_paths": [
"rules/audit/code-string-concat.yaml",
"rules/audit/dangerous-spawn-shell.yaml",
"rules/audit/detect-non-literal-fs-filename.yaml",
"rules/audit/detect-non-literal-regexp.yaml",
"rules/audit/detect-non-literal-require.yaml",
"rules/audit/detect-redos.yaml",
"rules/audit/hardcoded-hmac-key.yaml",
"rules/audit/incomplete-sanitization.yaml",
"rules/audit/md5-used-as-password.yaml",
"rules/audit/path-traversal/path-join-resolve-traversal.yaml",
"rules/audit/prototype-pollution/prototype-pollution-assignment.yaml",
"rules/audit/prototype-pollution/prototype-pollution-loop.yaml",
"rules/audit/spawn-shell-true.yaml",
"rules/audit/sqli/node-knex-sqli.yaml",
"rules/audit/sqli/node-mssql-sqli.yaml",
"rules/audit/sqli/node-mysql-sqli.yaml",
"rules/audit/sqli/node-postgres-sqli.yaml",
"rules/audit/unknown-value-with-script-tag.yaml",
"rules/audit/unsafe-dynamic-method.yaml",
"rules/audit/unsafe-formatstring.yaml",
"rules/detect-buffer-noassert.yaml",
"rules/detect-child-process.yaml",
"rules/detect-disable-mustache-escape.yaml",
"rules/detect-eval-with-expression.yaml",
"rules/detect-insecure-websocket.yaml",
"rules/detect-no-csrf-before-method-override.yaml",
"rules/detect-pseudoRandomBytes.yaml",
"rules/html-in-template-string.yaml",
"rules/insecure-object-assign.yaml",
"rules/spawn-git-clone.yaml"
],
"selection": "Every `.yaml` rule file under the upstream per-language security path, copied byte for byte with its upstream directory structure preserved. The co-located `.js`/`.ts` rule-test fixtures are not part of the ruleset and are not vendored; nothing else is filtered, edited, reordered, or added.",
"activation": "`semgrep scan --oss-only --config adapters/semgrep/native/javascript`. `taint_assume_safe_functions` is deliberately NOT set: the shipped default is the product under test (docs/native-profile.md#semgrep-ce--11740---oss-only).",
"notes": "Vendored under docs/native-profile.md#provenance-for-vendored-activation-artifacts because Semgrep registry configurations are network-fetched and unpinnable at run time. This snapshot is an activation artifact, not a fixture: it carries `derived` provenance, never `authored`."
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
rules:
- id: code-string-concat
message: >-
Found data from an Express or Next web request flowing to `eval`. If this data is user-controllable
this can lead to execution of arbitrary system commands in the context of your application process.
Avoid `eval` whenever possible.
options:
interfile: true
metadata:
interfile: true
confidence: HIGH
owasp:
- A03:2021 - Injection
- A05:2025 - Injection
cwe:
- "CWE-95: Improper Neutralization of Directives in Dynamically Evaluated Code ('Eval Injection')"
references:
- https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/eval
- https://nodejs.org/api/child_process.html#child_processexeccommand-options-callback
- https://www.stackhawk.com/blog/nodejs-command-injection-examples-and-prevention/
- https://ckarande.gitbooks.io/owasp-nodegoat-tutorial/content/tutorial/a1_-_server_side_js_injection.html
category: security
technology:
- node.js
- Express
- Next.js
subcategory:
- vuln
likelihood: MEDIUM
impact: MEDIUM
languages:
- javascript
- typescript
severity: ERROR
mode: taint
pattern-sources:
- pattern-either:
- patterns:
- pattern-either:
- pattern-inside: function ... ($REQ, $RES) {...}
- pattern-inside: function ... ($REQ, $RES, $NEXT) {...}
- patterns:
- pattern-either:
- pattern-inside: $APP.$METHOD(..., function $FUNC($REQ, $RES) {...})
- pattern-inside: $APP.$METHOD(..., function $FUNC($REQ, $RES, $NEXT) {...})
- metavariable-regex:
metavariable: $METHOD
regex: ^(get|post|put|head|delete|options)$
- pattern-either:
- pattern: $REQ.query
- pattern: $REQ.body
- pattern: $REQ.params
- pattern: $REQ.cookies
- pattern: $REQ.headers
- patterns:
- pattern-either:
- pattern-inside: |
import { ...,$IMPORT,... } from 'next/router'
...
- pattern-inside: |
import $IMPORT from 'next/router';
...
- pattern-either:
- patterns:
- pattern-inside: |
$ROUTER = $IMPORT()
...
- pattern-either:
- pattern-inside: |
const { ...,$PROPS,... } = $ROUTER.query
...
- pattern-inside: |
var { ...,$PROPS,... } = $ROUTER.query
...
- pattern-inside: |
let { ...,$PROPS,... } = $ROUTER.query
...
- focus-metavariable: $PROPS
- patterns:
- pattern-inside: |
$ROUTER = $IMPORT()
...
- pattern: |
$ROUTER.query.$VALUE
- patterns:
- pattern: $IMPORT().query.$VALUE
pattern-sinks:
- patterns:
- pattern: |
eval(...)
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
rules:
- id: dangerous-spawn-shell
message: >-
Detected non-literal calls to $EXEC(). This could lead to a command
injection vulnerability.
metadata:
cwe:
- "CWE-78: Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection')"
owasp:
- A01:2017 - Injection
- A03:2021 - Injection
- A05:2025 - Injection
source-rule-url: https://github.com/nodesecurity/eslint-plugin-security/blob/master/rules/detect-child-process.js
category: security
technology:
- javascript
references:
- https://cheatsheetseries.owasp.org/cheatsheets/Nodejs_Security_Cheat_Sheet.html#do-not-use-dangerous-functions
cwe2022-top25: true
cwe2021-top25: true
subcategory:
- vuln
likelihood: HIGH
impact: MEDIUM
confidence: LOW
languages:
- javascript
- typescript
severity: ERROR
mode: taint
pattern-sources:
- patterns:
- pattern-inside: |
function ... (...,$FUNC,...) {
...
}
- focus-metavariable: $FUNC
pattern-sinks:
- patterns:
- pattern-either:
- pattern-inside: |
require('child_process')
...
- pattern-inside: |
import 'child_process'
...
- pattern-either:
- pattern: spawn(...)
- pattern: spawnSync(...)
- pattern: $CP.spawn(...)
- pattern: $CP.spawnSync(...)
- pattern-either:
- pattern: |
$EXEC("=~/(sh|bash|ksh|csh|tcsh|zsh)/",["-c", $ARG, ...],...)
- patterns:
- pattern: $EXEC($CMD,["-c", $ARG, ...],...)
- pattern-inside: |
$CMD = "=~/(sh|bash|ksh|csh|tcsh|zsh)/"
...
- pattern: |
$EXEC("=~/(sh|bash|ksh|csh|tcsh|zsh)/",[$ARG, ...],...)
- patterns:
- pattern: $EXEC($CMD,[$ARG, ...],...)
- pattern-inside: |
$CMD = "=~/(sh|bash|ksh|csh|tcsh|zsh)/"
...
- focus-metavariable: $ARG
Loading
Loading