Add nub package manager ecosystem (nub.lock) - #15524
Open
colinhacks wants to merge 43 commits into
Open
Conversation
nub writes nub.lock (byte-compatible with pnpm-lock v9). The gem is cloned from the bun ecosystem and adapted: nub.lock detection, packageManager nub@x, NubLock routed through the pnpm parseLockfile helper, native updater invokes nub install --lockfile-only, Dockerfile installs @nubjs/nub. WIP: spec fixtures still carry bun text-format lockfiles and need regenerating as pnpm-v9 nub.lock; parser specs updated accordingly.
Adds require "dependabot/nub" so the ecosystem gem is loaded/discovered. Remaining infra wiring (ci.yml, ci-filters.yml, smoke-filters.yml, issue-labeler.yml, bin/dry-run.rb, script/dependabot, rakelib helpers) must be generated via `rake ecosystem:update_infrastructure[nub]` inside the dependabot dev environment (ruby 3 + bundle).
- Fix compound-word rename corruption: :npm_nubdled -> :npm_bundled (B1). - NubLock#dependencies: two-pass (specifier-bearing first) so alias metadata survives DependencySet de-dupe, mirroring the pnpm parser (S1). - FileUpdater: drop invalid bun-style 'install <pkg>@<ver>'; nub is pnpm-compatible, so write the bumped package.json + 'nub install --lockfile-only' (V1 core path). - Defer dependency_grapher: bun's grapher reads the bun-lock hash shape, incompatible with the pnpm-array parse output; the pnpm-v9 edge-reading rewrite is a follow-up. - version_resolver: TODO documenting the remaining 'update <pkg>@<ver>' grammar gap. - Regenerate simple_v0/simple_v1/wildcard fixtures as real pnpm-v9 nub.lock; rewrite lockfile_parser_spec to pnpm-v9 semantics. - Dockerfile NUB_VERSION -> 0.3.1 (current published).
Add "nub" => "nub" to Config::File::PACKAGE_MANAGER_LOOKUP so Utils.validate_package_manager! accepts it; without this the whole nub ecosystem fails to load (register_version_class raises). Found by running the nub parser spec in the dev image.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
nub is pnpm-compatible with no 'install <pkg>@<ver>' add and rejects 'update <pkg>@<ver>' for an arbitrary version, so a lockfile-only update pins each Dependabot-chosen version by writing it exactly into package.json, running 'nub install --lockfile-only', then restoring the final requirement and installing again (nub's lockfile-only install keeps the locked version). Regenerates the multiple_updates + github_dependency fixtures as pnpm-v9; updates file_updater/subdependency spec expectations to nub's grammar + the pnpm-v9 git-dep (codeload tarball) lockfile form.
- Fix file_fetcher nub-detection: contents listing + nub_lock fixture were bun-format (JSON fixtures weren't renamed); list nub.lock + pnpm-v9 content. - Skip (documented follow-ups): git-dependency updates (nub keeps the locked git SHA; forcing re-resolution to a chosen ref is unwired) and monorepo source-directory metadata extraction.
Ran `rake ecosystem:update_infrastructure[nub]`: adds nub to the CI matrix (ci.yml, images-branch/latest), ci/smoke filters + smoke-matrix, the issue labeler, bin/dry-run LOAD_PATH, script/dependabot volume mount, updater setup app_dirs_pattern, and the rakelib gemspec list.
Replace the invalid `nub update <pkg>@<ver>` (nub rejects a pinned version on `update`) with the same pin-then-install approach the FileUpdater uses: pin the candidate version into the manifests and re-resolve `nub install --lockfile-only`, so a peer-dependency conflict surfaces as a classifiable subprocess failure.
…ess) Addresses an adversarial-review nit: NubLock references NubPackageManager::NAME but relied on load order for the constant. Make the dependency explicit.
nub.lock is byte-compatible with pnpm-lock v9, so nub can depend on dependabot-npm_and_yarn and alias its byte-identical classes rather than maintaining clones (uv/pre_commit precedent). Aliases PackageName, VersionSelector, and FileUpdater::PackageJsonPreparer.
- Remove the unreferenced yarn-derived error-pattern block from nub.rb (VALIDATION_GROUP_PATTERNS/Utils/regexes were never consumed). - Reuse NpmAndYarn::PnpmErrorHandler in the lockfile updater's error path (nub emits pnpm-format engine errors), layered before nub's own patterns. - Delete the Yarn PnP vendor/artifact updater and the git-lfs .yarn pull (berry-only, dead for nub.lock). - Delete the dead npm6 JS helper tree (no npm6: call site).
…me-branch case Grounding nub-dev shows nub re-resolves a git dependency when the manifest ref changes (dependabot#2.0.0 -> dependabot#2.1.0): PackageJsonUpdater rewrites the ref and nub's lockfile-only install picks up the new tag's commit. Add a fixture + FileUpdater example proving this end to end. The same-branch SHA advance (manifest ref unchanged, e.g. master -> master) stays skipped with a precise reason: nub's conservative lockfile-only install keeps the locked SHA, and nub exposes no targeted force-re-resolution. Refine the metadata monorepo-directory skip reason to name the missing spec stub.
nub now depends on dependabot-npm_and_yarn (reused pnpm-format classes). The updater-core base only stubs sibling ecosystems, so without copying the full npm_and_yarn source the nub image would load its stub and the aliases would fail at require time. Mirror pre_commit/Dockerfile, which copies its gem deps.
…transitive/security updates nub 0.4.1 (dependabot#355) accepts `update <pkg>@<version>` for registry deps INCLUDING transitive/sub-dependencies not in any manifest. Replace the pin-restore double-install with a single `nub update <deps>@<vers> --lockfile-only --no-save` (mirrors npm_and_yarn's pnpm updater) + a settling install. This wires Dependabot's security-update path for deps-of-deps (previously a NoChangeError). Adds a transitive-update fixture + spec (bumps lodash, a transitive of fetch-factory, in the lockfile). Suite: 338 examples, 0 failures, 3 pending (same-branch git-SHA-advance ×2 + monorepo-dir metadata ×1 — documented).
…endabot.yml Three registration points nub was never added to, each of which every other ecosystem carries: - the root Gemfile and Gemfile.lock list every ecosystem as a path gem (32 of 33 before this); without the entry the root bundle cannot load dependabot-nub, which is the bundle the lint gates run through - sorbet/config excludes each ecosystem's spec/ directory from typechecking (32 of 33 before this) - .github/dependabot.yml watches each helpers directory that carries a package manifest; nub/helpers has package.json and package-lock.json, exactly like bun/helpers, so its JS dependencies were never being updated The Gemfile.lock entries were verified by regenerating the lockfile with bundler and diffing - the PATH block, DEPENDENCIES entry and CHECKSUMS entry match its own resolution exactly. nub is deliberately not added to updater/lib/github_api/ecosystem_mapper.rb: that map tracks the Dependency Graph snapshot feature, which only the five ecosystems implementing a DependencyGrapher are in. deno, sbt, julia, opentofu, vcpkg and rust_toolchain are all absent for the same reason.
Upstream's typed-requirement migration advanced while this branch was open:
DependencyRequirement#[] now returns Object, so every chained access -
req[:source][:type], req.fetch(:requirement), r.dig(:source, :type) - stopped
typechecking. srb tc reported 45 errors, all in nub/.
Each site adopts the form npm_and_yarn and bun already use for the same line,
so nub stays a faithful clone of the ecosystem it was modelled on rather than
growing its own idiom: source_string("type"), source_hash, requirement_string,
.file, and the ObjectHash return type on the source-details signatures. The
git-tag reads move to latest_tag&.tag / &.commit_sha for the same reason.
No behaviour change is intended. srb tc is clean, rubocop reports no offenses
in nub/, and the spec suite returns 338 examples with exactly the same 9
environment-dependent failures as the unmodified tree - those 9 shell out to
the JS helper, which is only present in the updater image.
0.6.0 was two minors behind. Verified equivalent for the only two commands the file updater runs - nub update <pkg>@<ver> --lockfile-only --no-save --ignore-scripts, then nub install --lockfile-only --ignore-scripts - by running both versions over the same nub-identity fixture, with and without a forced release-age window as a positive control. Exit codes and resulting lockfile match at every cell. 0.8.0 keeps the same engines floor (node >= 18.19.0, against the image's Node 24) and publishes the same eight platform packages, so nothing about the image build changes.
Author
|
Any chance we can get eyes on this? Thanks! cc @JamieMagee |
Dependency Review failed on nub/helpers/package-lock.json: tar 7.5.16 carries five node-tar DoS advisories - GHSA-23hp-3jrh-7fpw (critical), GHSA-8x88-c5mf-7j5w and GHSA-r292-9mhp-454m (high), GHSA-gvwx-54wh-qm9j and GHSA-w8wr-v893-vjvp (moderate). The highest first-patched version among them is 7.5.21; 7.5.22 is current. tar is pinned in the helpers' overrides block, which exists to hold transitive dependencies at safe versions, so the fix belongs there. Regenerating the lockfile changes exactly one package version and adds or removes none. This was the only dependency the gate flagged: tar 7.5.16 appeared nowhere else in the repo, so it was new to the dependency graph and got reviewed, while the helpers' other pinned transitives already exist at the same versions under bun/helpers and npm_and_yarn/helpers.
CodeQL reported three rb/polynomial-redos alerts against nub - two in
registry_finder and one in requirement - and passes on every other open pull
request, so the ecosystem stood out as introducing security alerts.
Uses the mitigation this repo already applies elsewhere: helm's
update_checker/requirements_updater.rb makes the same class of separator regex
possessive under the comment "Possessive quantifiers (++) keep these linear on
pathological input". Same treatment and same comment style here, so the fix is
an in-repo idiom rather than a nub invention.
Rewritten: %r{/+$} to %r{/++$}, %r{^.*?//} to %r{^[^/]*+(?:/[^/]*+)*?//}, and
AND_SEPARATOR's \s+/&+ runs to \s++/&++. OR_SEPARATOR and the requirements
updater's SEPARATOR are left alone - neither was flagged, and \s*\|+ has
disjoint adjacent classes so there is no ambiguity to remove.
Equivalence was checked rather than assumed: every pair was run over 1327
inputs - real requirement and URL strings, exhaustive short combinations, and
long repetition shapes - comparing split, scan, gsub and match, for 37156
comparisons with no divergence. A negative control confirmed the comparison
detects genuinely different pairs. The nub suite is unchanged at 338 examples.
Dependency Review flagged brace-expansion 1.1.13 (three high-severity DoS advisories) once tar was fixed. Auditing every entry in the overrides block against the advisory database found two more that were not reported only because the same versions already exist elsewhere in the repo: ip-address 10.1.1 and js-yaml 4.2.0. - brace-expansion 1.1.13 -> 1.1.18, which is what bun/helpers and npm_and_yarn/helpers already resolve to - ip-address 10.1.1 -> 10.4.0, likewise matching both - js-yaml 4.2.0 -> 4.3.1, the first patched release on the 4.x line Each target was confirmed to carry no advisories of its own before being pinned, and all eighteen overrides now come back clean. Regenerating the lockfile changes exactly these three versions and adds or removes none. The eight advisories npm audit still reports are all on unpinned transitives that resolve to the same versions under bun/helpers and npm_and_yarn/helpers, so they are not new to the dependency graph and are outside this gate.
arborist 8 pins pacote ^19.0.0, and pacote is vulnerable to a DoS via addGitSha below 21.5.1 (GHSA-w4pp-8pjf-rmxw). Bumping arborist to the ^9.4.0 line npm_and_yarn already uses resolves pacote to 21.5.1 without forcing a version its dependent does not declare. The two files that construct an Arborist, lib/npm/conflicting-dependency-parser.js and lib/npm/vulnerability-auditor.js, are identical to npm_and_yarn's, which runs them on arborist 9.
http-proxy-agent 5.0.0 pulls @tootallnate/once 2, which resolved to 2.0.0 and is vulnerable to incorrect control flow scoping below 2.0.1 (GHSA-vpq2-c234-7xj6). bun already resolves 2.0.1 through the same chain.
# Conflicts: # Gemfile.lock # updater/Gemfile.lock
RuboCop 1.90.0 adds Style/DirectiveScope, which rejects a disable/enable pair wrapping a single statement. Matches the form npm_and_yarn's lockfile updaters already use.
Author
|
Merged latest |
The test runner was renamed to turbo_tests2 and every other ecosystem's ci-test moved with it; nub's still invoked turbo_tests and exited 127.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #15523.
What are you trying to accomplish?
First-party support for the nub package manager, which writes
nub.lock— byte-compatible with the pnpm lockfile v9 format. nub is distributed on npm as@nubjs/nub; dependencies resolve from the npm registry.Because
nub.lockis a pnpm-lock v9 document, the gem depends ondependabot-npm_and_yarnand reuses its pnpm-format classes rather than duplicating them — theuv→dependabot-python/pre_commit→dependabot-npm_and_yarnpattern. The nub-specific surface isnub.lockdetection,packageManager: "nub@x", and invoking thenubCLI.Supported update types:
nub update <name>@<version> --lockfile-only.#2.0.0→#2.1.0).Known limitation (documented; the two tests are skipped): advancing a git dependency pinned to a branch to that branch's latest commit. nub's lockfile-only install conservatively keeps the locked SHA and exposes no force-re-resolution for this case; tracked for a follow-up nub release. Git ref changes (above) work.
Known limitation: conflict detection for transitive security updates is not yet implemented. The inherited yarn-based
ConflictingDependencyResolvercould not function against nub's pnpm-v9nub.lock— it reads ayarn.lockthat is never written, so every call failed into an empty result — and has been removed; a pnpm-lock-aware implementation overnub.lockis a follow-up.Diff composition
Of ~110.8k added lines: ~8k gem code (
nub/lib), ~6k specs, ~61.5k spec fixtures, and ~35k native helpers (nub/helpers), plus ~200 of Docker/gemspec/config glue. The fixtures and helpers follow the Bun ecosystem's standalone layout — Dependabot's most recent comparable ecosystem, cloned to the same shape. Spec fixtures the nub suite never loads have been pruned (~53k lines removed) so the tree carries only what the tests reference.Happy to split this into sequential PRs — package manager → parser → fetcher → updater, the shape the Bun review took (#11267–#11300) — if the maintainers prefer.
Anything you want to highlight for special attention from reviewers?
The gem reuses
dependabot-npm_and_yarn's pnpm classes (the lockfile-parser helper,PnpmErrorHandler) via a gemspec dependency plus aliases;nub/Dockerfilecopies the fullnpm_and_yarnlib so the aliases load in the production image (thepre_commitprecedent).How will you know you've accomplished your goal?
The
nubrspec suite is green: 338 examples, 0 failures, 3 pending (the git-branch-SHA-advance limitation above, and one monorepo source-directory metadata case). Abin/dry-run.rb nubproduces update PRs against a real nub-lockfile repository.Checklist