Skip to content

perf: add single-walk file filter that prunes ignored directories#657

Draft
z4ce wants to merge 1 commit into
snyk:mainfrom
z4ce:cc
Draft

perf: add single-walk file filter that prunes ignored directories#657
z4ce wants to merge 1 commit into
snyk:mainfrom
z4ce:cc

Conversation

@z4ce

@z4ce z4ce commented Jul 16, 2026

Copy link
Copy Markdown

This PR is in relation to this ticket. My testing shows that the changes here reduce bundle creation time from between %50 to %90. Feel free to take whatever you want from this PR and remix it with whatever design might suite engineering better.

Add GetFilteredFilesSingleWalk, which traverses the directory tree once instead of twice (GetAllFiles + GetFilteredFiles) and prunes wholly- excluded directories such as node_modules and .git rather than globbing every file beneath them — where almost all the time on real projects was spent.

A directory is pruned only when a rule excludes the directory itself (node_modules, /node_modules/, /dist), never when a rule excludes only its contents (src/*, obj/) or when the directory contains its own ignore file, since those may be re-included by a ! negation. Per-file emission still uses the full rule set, so output is identical to the original pipeline.

Includes equivalence, negation-edge-case, and pruning tests plus a benchmark against the old pipeline.

Description

Provide description of this PR and changes.

Checklist

  • Tests added and all succeed (make test)
  • [N/A] Regenerated mocks, etc. (make generate)
  • Linted (make lint)
  • Test your changes work for the CLI
    1. Clone / pull the latest CLI main.
    2. Run go get github.com/snyk/go-application-framework@YOUR_LATEST_GAF_COMMIT in the cliv2 directory.
      • Tip: for local testing, you can uncomment the line near the bottom of the CLI's go.mod to point to your local GAF code.
    3. Run go mod tidy in the cliv2 directory.
    4. Run the CLI tests and do any required manual testing.
    5. Open a PR in the CLI repo now with the go.mod and go.sum changes.
    • Once this PR is merged, repeat these steps, but pointing to the latest GAF commit on main and update your CLI PR.

Add GetFilteredFilesSingleWalk, which traverses the directory tree once
instead of twice (GetAllFiles + GetFilteredFiles) and prunes wholly-
excluded directories such as node_modules and .git rather than globbing
every file beneath them — where almost all the time on real projects was
spent.

A directory is pruned only when a rule excludes the directory itself
(node_modules, /node_modules/, **/dist), never when a rule excludes only
its contents (src/*, obj/**) or when the directory contains its own
ignore file, since those may be re-included by a ! negation. Per-file
emission still uses the full rule set, so output is identical to the
original pipeline.

Includes equivalence, negation-edge-case, and pruning tests plus a
benchmark against the old pipeline.
@snyk-io

snyk-io Bot commented Jul 16, 2026

Copy link
Copy Markdown

Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
Open Source Security 0 0 0 0 0 issues
Licenses 0 0 0 0 0 issues
Code Security 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

@snyk-io

snyk-io Bot commented Jul 16, 2026

Copy link
Copy Markdown

Snyk checks have passed. No issues have been found so far.

Status Scan Engine Critical High Medium Low Total (0)
Open Source Security 0 0 0 0 0 issues
Licenses 0 0 0 0 0 issues
Code Security 0 0 0 0 0 issues
Secrets 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant