add support for zig#6
Open
omkarbhad wants to merge 36 commits into
Open
Conversation
- [X] I ran `make setup && make` to update the generated code after editing a `.atd` file (TODO: have a CI check) - [X] I made sure we're still backward compatible with old versions of the CLI. For example, the Semgrep backend need to still be able to *consume* data generated by Semgrep 1.50.0. See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades Note that the types related to the semgrep-core JSON output or the semgrep-core RPC do not need to be backward compatible!
…mgrep#330) Added necessary interface changes to support 1) a new ecosystem, cocoapods, 2) new lockfile + manifest types for cocoapods, and 3) a new parser type for (you guessed it) cocoapods. - [x] I ran `make setup && make` to update the generated code after editing a `.atd` file (TODO: have a CI check) - [x] I made sure we're still backward compatible with old versions of the CLI. For example, the Semgrep backend need to still be able to *consume* data generated by Semgrep 1.50.0. See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades Note that the types related to the semgrep-core JSON output or the semgrep-core RPC do not need to be backward compatible! Co-authored-by: Aaron Acosta <aaronacosta@aarons-mbp.local.meter>
This is part of renaming the OCaml module `Xlang` to `Analyzer`. This type name change doesn't change the data format because type names don't occur in JSON data. - [x] I ran `make setup && make` to update the generated code after editing a `.atd` file (TODO: have a CI check) - [x] I made sure we're still backward compatible with old versions of the CLI. For example, the Semgrep backend need to still be able to *consume* data generated by Semgrep 1.50.0. See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades Note that the types related to the semgrep-core JSON output or the semgrep-core RPC do not need to be backward compatible!
test plan: make - [x] I ran `make setup && make` to update the generated code after editing a `.atd` file (TODO: have a CI check) - [x] I made sure we're still backward compatible with old versions of the CLI. For example, the Semgrep backend need to still be able to *consume* data generated by Semgrep 1.50.0. See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades Note that the types related to the semgrep-core JSON output or the semgrep-core RPC do not need to be backward compatible! Co-authored-by: pad <pad@r2c.dev>
The goal is to make it as close as possible as the general CallFormat so at some point we can merge them. test plan: see related PR in semgrep - [x] I ran `make setup && make` to update the generated code after editing a `.atd` file (TODO: have a CI check) - [x] I made sure we're still backward compatible with old versions of the CLI. For example, the Semgrep backend need to still be able to *consume* data generated by Semgrep 1.50.0. See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades Note that the types related to the semgrep-core JSON output or the semgrep-core RPC do not need to be backward compatible! Co-authored-by: pad <pad@r2c.dev>
As title. It seems that our `Makefile` is improperly specified and does not regenerate `ast_generic_v1.py`. I fixed this in this PR also. - [X] I ran `make setup && make` to update the generated code after editing a `.atd` file (TODO: have a CI check) - [X] I made sure we're still backward compatible with old versions of the CLI. For example, the Semgrep backend need to still be able to *consume* data generated by Semgrep 1.50.0. See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades Note that the types related to the semgrep-core JSON output or the semgrep-core RPC do not need to be backward compatible!
) Adds a new flag to pass from the app to the CLI that forces the CLI to resolve dependencies for all subprojects, all the time (even for diff scans). Some customers want this so that we get a full picture of their dependencies for every branch. - [x] I ran `make setup && make` to update the generated code after editing a `.atd` file (TODO: have a CI check) - [x] I made sure we're still backward compatible with old versions of the CLI. For example, the Semgrep backend need to still be able to *consume* data generated by Semgrep 1.50.0. See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades Note that the types related to the semgrep-core JSON output or the semgrep-core RPC do not need to be backward compatible!
…st (semgrep#333) The backend should not use those fields anymore so we can remove them. Note that even if old versions of semgrep are still producing those fields, it is fine because when the backend is consuming the JSON, any unknown field is skipped and those fields should not be used anymore. test plan: see related PR in semgrep - [x] I ran `make setup && make` to update the generated code after editing a `.atd` file (TODO: have a CI check) - [x] I made sure we're still backward compatible with old versions of the CLI. For example, the Semgrep backend need to still be able to *consume* data generated by Semgrep 1.50.0. See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades Note that the types related to the semgrep-core JSON output or the semgrep-core RPC do not need to be backward compatible! Co-authored-by: pad <pad@r2c.dev>
…emgrep#338) test plan: wait for green CI checks and see related PR in semgrep - [x] I ran `make setup && make` to update the generated code after editing a `.atd` file (TODO: have a CI check) - [x] I made sure we're still backward compatible with old versions of the CLI. For example, the Semgrep backend need to still be able to *consume* data generated by Semgrep 1.50.0. See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades Note that the types related to the semgrep-core JSON output or the semgrep-core RPC do not need to be backward compatible! --------- Co-authored-by: pad <pad@r2c.dev>
It was experimental and it's not maintained nor used, so let's get rid of it. test plan: see related PR in semgrep - [x] I ran `make setup && make` to update the generated code after editing a `.atd` file (TODO: have a CI check) - [x] I made sure we're still backward compatible with old versions of the CLI. For example, the Semgrep backend need to still be able to *consume* data generated by Semgrep 1.50.0. See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades Note that the types related to the semgrep-core JSON output or the semgrep-core RPC do not need to be backward compatible! Co-authored-by: pad <pad@r2c.dev> Co-authored-by: Martin Jambon <github@mjambon.com>
…of target files (semgrep#337) This is used by https://github.com/semgrep/semgrep-proprietary/pull/2878. - [x] I ran `make setup && make` to update the generated code after editing a `.atd` file (TODO: have a CI check) - [x] I made sure we're still backward compatible with old versions of the CLI. For example, the Semgrep backend need to still be able to *consume* data generated by Semgrep 1.50.0. See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades Note that the types related to the semgrep-core JSON output or the semgrep-core RPC do not need to be backward compatible!
This PR adds "symbol analysis", a dump of the third-party symbols used by a project, to the scan results for Semgrep. This will enable the `scrat` project. - [X] I ran `make setup && make` to update the generated code after editing a `.atd` file (TODO: have a CI check) - [X] I made sure we're still backward compatible with old versions of the CLI. For example, the Semgrep backend need to still be able to *consume* data generated by Semgrep 1.50.0. See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades Note that the types related to the semgrep-core JSON output or the semgrep-core RPC do not need to be backward compatible!
test plan: see related PR in semgrep - [x] I ran `make setup && make` to update the generated code after editing a `.atd` file (TODO: have a CI check) - [x] I made sure we're still backward compatible with old versions of the CLI. For example, the Semgrep backend need to still be able to *consume* data generated by Semgrep 1.50.0. See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades Note that the types related to the semgrep-core JSON output or the semgrep-core RPC do not need to be backward compatible! --------- Co-authored-by: Ben Kettle <ben@semgrep.com>
As title. - [X] I ran `make setup && make` to update the generated code after editing a `.atd` file (TODO: have a CI check) - [X] I made sure we're still backward compatible with old versions of the CLI. For example, the Semgrep backend need to still be able to *consume* data generated by Semgrep 1.50.0. See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades Note that the types related to the semgrep-core JSON output or the semgrep-core RPC do not need to be backward compatible!
These are the relevant changes so we can plumb the symbol analysis information through to upload it to the App. - [X] I ran `make setup && make` to update the generated code after editing a `.atd` file (TODO: have a CI check) - [X] I made sure we're still backward compatible with old versions of the CLI. For example, the Semgrep backend need to still be able to *consume* data generated by Semgrep 1.50.0. See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades Note that the types related to the semgrep-core JSON output or the semgrep-core RPC do not need to be backward compatible!
This adds some stuff which makes us able to call uploading symbol analysis from pysemgrep. - [X] I ran `make setup && make` to update the generated code after editing a `.atd` file (TODO: have a CI check) - [X] I made sure we're still backward compatible with old versions of the CLI. For example, the Semgrep backend need to still be able to *consume* data generated by Semgrep 1.50.0. See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades Note that the types related to the semgrep-core JSON output or the semgrep-core RPC do not need to be backward compatible!
Just fixed a typo, the return value is not an error message, it only returns in the success case. It's a success message.
- [x] I ran `make setup && make` to update the generated code after editing a `.atd` file (TODO: have a CI check) - [x] I made sure we're still backward compatible with old versions of the CLI. For example, the Semgrep backend need to still be able to *consume* data generated by Semgrep 1.50.0. See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades Note that the types related to the semgrep-core JSON output or the semgrep-core RPC do not need to be backward compatible! Co-authored-by: Martin Jambon <martin@semgrep.com>
…caParseError (semgrep#350) test plan: see related PR in semgrep - [x] I ran `make setup && make` to update the generated code after editing a `.atd` file (TODO: have a CI check) - [x] I made sure we're still backward compatible with old versions of the CLI. For example, the Semgrep backend need to still be able to *consume* data generated by Semgrep 1.50.0. See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades Note that the types related to the semgrep-core JSON output or the semgrep-core RPC do not need to be backward compatible!
The goal is to remove the redundancy between the type defs in the .atd and in subproject.py (which will help migrate the code from subproject.py to a new Subproject.ml) test plan: see related PR in semgrep - [x] I ran `make setup && make` to update the generated code after editing a `.atd` file (TODO: have a CI check) - [x] I made sure we're still backward compatible with old versions of the CLI. For example, the Semgrep backend need to still be able to *consume* data generated by Semgrep 1.50.0. See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades Note that the types related to the semgrep-core JSON output or the semgrep-core RPC do not need to be backward compatible!
We previously did not distribute a specific license file with this repository since it is included in semgrep/semgrep as a submodule, but for clarity as to how this specific set of files is licensed (and because users may obtain them separately) let's add a copy of the license here. Closes semgrep/semgrep#10850.
We need a different type to represent a dependency graph that is internal to
the Semgrep CLI. We already have these types in Python and in Ocaml. This adds
a corresponding type to the ATD interface so that the internal dependency type
can be sent via RPC.
- [x] I ran `make setup && make` to update the generated code after editing a `.atd` file (TODO: have a CI check)
- [x] I made sure we're still backward compatible with old versions of the CLI.
For example, the Semgrep backend need to still be able to *consume* data
generated by Semgrep 1.50.0.
See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades
Note that the types related to the semgrep-core JSON output or the
semgrep-core RPC do not need to be backward compatible!
This updates only the RPC types, so is backwards compatible.
Towards SC-2082.
TR will require sending the dependency graph in addition to the transitive findings in order to do the filtering step. This adds a single dependency graph to the RPC interface. - [x] I ran `make setup && make` to update the generated code after editing a `.atd` file (TODO: have a CI check) - [x] I made sure we're still backward compatible with old versions of the CLI. For example, the Semgrep backend need to still be able to *consume* data generated by Semgrep 1.50.0. See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades Note that the types related to the semgrep-core JSON output or the semgrep-core RPC do not need to be backward compatible!
) test plan: see related PR in semgrep - [x] I ran `make setup && make` to update the generated code after editing a `.atd` file (TODO: have a CI check) - [x] I made sure we're still backward compatible with old versions of the CLI. For example, the Semgrep backend need to still be able to *consume* data generated by Semgrep 1.50.0. See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades Note that the types related to the semgrep-core JSON output or the semgrep-core RPC do not need to be backward compatible!
Allows `metavariable-name` to appear without a `semgrep-internal` prefix when using the `module(s)` constraint (but not yet for `kind`).
This PR adds the setup.py manifest type! Towards SC-2149 - [x] I ran `make setup && make` to update the generated code after editing a `.atd` file (TODO: have a CI check) - [x] I made sure we're still backward compatible with old versions of the CLI. For example, the Semgrep backend need to still be able to *consume* data generated by Semgrep 1.50.0. See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades Note that the types related to the semgrep-core JSON output or the semgrep-core RPC do not need to be backward compatible!
…p#358) This is for https://github.com/semgrep/semgrep-proprietary/pull/3099 - [x] I ran `make setup && make` to update the generated code after editing a `.atd` file (TODO: have a CI check) - [x] I made sure we're still backward compatible with old versions of the CLI. For example, the Semgrep backend need to still be able to *consume* data generated by Semgrep 1.50.0. See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades Note that the types related to the semgrep-core JSON output or the semgrep-core RPC do not need to be backward compatible!
Add "disabled" option to `UnresolvedReason` and make it a frozen dataclass so we can use it in sets and as a dictionary key in python - [x] I ran `make setup && make` to update the generated code after editing a `.atd` file (TODO: have a CI check) - [x] I made sure we're still backward compatible with old versions of the CLI. For example, the Semgrep backend need to still be able to *consume* data generated by Semgrep 1.50.0. See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades Note that the types related to the semgrep-core JSON output or the semgrep-core RPC do not need to be backward compatible!
Adds an option to the `ResolveDependencies` RPC to instruct the resolver to download dependency source code. - [x] I ran `make setup && make` to update the generated code after editing a `.atd` file (TODO: have a CI check) - [x] I made sure we're still backward compatible with old versions of the CLI. For example, the Semgrep backend need to still be able to *consume* data generated by Semgrep 1.50.0. See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades Note that the types related to the semgrep-core JSON output or the semgrep-core RPC do not need to be backward compatible!
…rep#356) In semgrep/semgrep-proprietary#2872, we updated the SCA subproject logic to scan only changed subprojects in diff scans. Subprojects that remained unchanged were intentionally left unresolved with the reason UnresolvedSkipped. Previously, we didn't include this information in our subproject stats, leaving us without visibility into why a subproject was unresolved. This PR introduces two optional fields, unresolved_reason and errors, allowing us to capture more granular stats and better understand subproject resolution failures. - [x] I ran `make setup && make` to update the generated code after editing a `.atd` file (TODO: have a CI check) - [x] I made sure we're still backward compatible with old versions of the CLI. For example, the Semgrep backend need to still be able to *consume* data generated by Semgrep 1.50.0. See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades Note that the types related to the semgrep-core JSON output or the semgrep-core RPC do not need to be backward compatible!
Let's install check-jsonschema via depexts! See semgrep/semgrep-proprietary#3273 for more context - [X] I ran `make setup && make` to update the generated code after editing a `.atd` file (TODO: have a CI check) - [X] I made sure we're still backward compatible with old versions of the CLI. For example, the Semgrep backend need to still be able to *consume* data generated by Semgrep 1.50.0. See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades Note that the types related to the semgrep-core JSON output or the semgrep-core RPC do not need to be backward compatible!
To make things more concrete about the interface we want with the backend. test plan: make - [x] I ran `make setup && make` to update the generated code after editing a `.atd` file (TODO: have a CI check) - [x] I made sure we're still backward compatible with old versions of the CLI. For example, the Semgrep backend need to still be able to *consume* data generated by Semgrep 1.50.0. See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades Note that the types related to the semgrep-core JSON output or the semgrep-core RPC do not need to be backward compatible!
This will allows us to dogfood SSC and TR with Opam packages on the semgrep-pro repo test plan: see related PR in semgrep - [x] I ran `make setup && make` to update the generated code after editing a `.atd` file (TODO: have a CI check) - [x] I made sure we're still backward compatible with old versions of the CLI. For example, the Semgrep backend need to still be able to *consume* data generated by Semgrep 1.50.0. See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades Note that the types related to the semgrep-core JSON output or the semgrep-core RPC do not need to be backward compatible!
- [x] I ran `make setup && make` to update the generated code after editing a `.atd` file (TODO: have a CI check) - [x] I made sure we're still backward compatible with old versions of the CLI. For example, the Semgrep backend need to still be able to *consume* data generated by Semgrep 1.50.0. See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades Note that the types related to the semgrep-core JSON output or the semgrep-core RPC do not need to be backward compatible!
Adds an RPC type that allows Python to ask Ocaml to match subprojects based on a list of dependency source files. This will allow us to switch to ocaml-based subproject matching independently from the rest of file targeting. - [x] I ran `make setup && make` to update the generated code after editing a `.atd` file (TODO: have a CI check) - [x] I made sure we're still backward compatible with old versions of the CLI. For example, the Semgrep backend need to still be able to *consume* data generated by Semgrep 1.50.0. See https://atd.readthedocs.io/en/latest/atdgen-tutorial.html#smooth-protocol-upgrades Note that the types related to the semgrep-core JSON output or the semgrep-core RPC do not need to be backward compatible!
Register Zig as a development-stage language with .zig and .zon extensions. This only updates the shared language registry. It does not change opengrep behavior until the opengrep repo bumps this submodule and wires the matching Lang.Zig/parser dispatch. When opengrep consumes this change, the submodule bump must be paired with working Zig parser dispatch or an explicit rejection path so users do not get accepted -lang zig scans with empty parser results.
|
Changed the base to |
|
Please rebase onto |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds Zig to the shared language registry as a development-stage language.
Registered extensions:
.zig.zonexample_extis set to.zigso snippet-to-file generation uses normal Zig source files rather than.zonbuild config files.Scope
This PR only updates
semgrep-interfaces.It does not change opengrep behavior until opengrep bumps this submodule and adds the matching parser/language wiring.
Future opengrep work is tracked in
omkarbhad/opengrep-zig, which provides the Zig tree-sitter language repo needed for the eventual full Zig support PR.Sequencing Note
When opengrep consumes this change, the submodule bump must be paired with working Zig parser dispatch or an explicit rejection path. Otherwise
-lang zigcould become accepted before Zig scans produce meaningful parser results.Verification
python3 ./generategenerate.pyLanguage.mlLanguage.mlilang.jsonopam exec -- ocamlc -c Language.mli && opam exec -- ocamlc -c Language.ml