spike(compile): nub compile launcher model, proven end-to-end (darwin-arm64) - #536
Draft
colinhacks wants to merge 109 commits into
Draft
spike(compile): nub compile launcher model, proven end-to-end (darwin-arm64)#536colinhacks wants to merge 109 commits into
colinhacks wants to merge 109 commits into
Conversation
- nub-core: shared payload container (compile.rs) + host Node provision/resolve helpers - nub-launcher: minimal runtime binary (own workspace) — section read, embed decompress-to-cache, --smol discover/shell-out provision, flag injection, signal-forwarding spawn - nub-cli: nub compile arm behind the compile feature (Rolldown in-process bundle, strip+resign+zstd-19 Node, libsui section-inject + ad-hoc sign) Claude-Session: https://claude.ai/code/session_01EcDRwN5Z9arPRLCdZUsafL
- launcher: dispatch __pdeath-watch verb (spawn.rs re-invokes current_exe); ruzstd 0.8 (match nub-core); build.rs reserves Mach-O headerpad so libsui section injection does not shift __TEXT (root cause of a post-injection SIGILL on the optimized binary) - cli: register compile in SUBCOMMANDS + CLAP_HELP_COMMANDS so dispatch reaches it - verified: AC1 embed (PATH-scrubbed cold 2.7s / warm 0.05s / dedup), AC2 real app (dep+TLA+dynamic import), AC3 --smol 0.56MB discovery Claude-Session: https://claude.ai/code/session_01EcDRwN5Z9arPRLCdZUsafL
The post-resign --version verification could not exec the staged Node (fs::write lands 0644), so every compile fell back to unstripped. chmod +x the staged copy before strip/verify. Embed binary now 25.9 MB (stripped) vs 29.4 MB (unstripped). Claude-Session: https://claude.ai/code/session_01EcDRwN5Z9arPRLCdZUsafL
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
…erify, Init --help - nub-core decode: checked_add on cumulative offsets + .get() slices (a corrupted section can no longer panic-abort the launcher); same for decode_app_region. Test: an overflowing length Errs cleanly. - launcher --smol: verify the shell-out-provisioned Node's sha256 against SHASUMS256.txt before committing into the shared store (which nub run + the embed dedup path trust un-re-verified). Test: a bad checksum aborts. - Init --help: the Compile variant's insertion had stolen Init's doc comment; restored so both summaries render. Claude-Session: https://claude.ai/code/session_01EcDRwN5Z9arPRLCdZUsafL
… self-probe, CI leg - cli.rs: gate the Compile clap variant + its SUBCOMMANDS/CLAP_HELP_COMMANDS entries behind #[cfg(feature = "compile")] so a shipped build exposes no compile verb (verified: default build → "not a nub command"). - launcher: reject payload file names that escape the extraction dir (absolute / .. / leading separator) before writing; test added. - compile.rs: error early on a non-macOS host (Mach-O-only injection) instead of failing deep in libsui. - self-probe: the launcher gains a __probe verb (section read + decode + String alloc, no extract/spawn); nub compile runs it on the produced binary so an under-padded/corrupt injection is caught at compile time, not as a runtime SIGILL. - ci.yml: lint + build the excluded nub-launcher crate (bit-rot guard). - gitignore the launcher's in-tree target/. Claude-Session: https://claude.ai/code/session_01EcDRwN5Z9arPRLCdZUsafL
… / smol range / error on none) `nub compile` had its own standalone default (`--target` defaulting to "24") rather than reading the project's Node pin, so a compiled binary could silently target a different version than `nub run` uses in the same tree. `--target` is now optional and overrides an inferred pin; with the flag omitted the version comes from `resolve_pin_chain` — the same chain `nub run` walks (devEngines.runtime, .node-version, .nvmrc, .tool-versions, engines.node) — and provisioning reports it through the existing `Using Node.js <v> (resolved from <source>)` surface rather than a compile-specific one. The two shapes consume the pin differently. Embed resolves it to one exact release at compile time (`resolve_host_pin`: a range/major/alias collapses to the newest satisfying release) because exactly one Node is baked in. Smol keeps the requirement open: the manifest records the acceptance floor (`pin_floor`) in `node_version`, which the launcher's existing `discovered >= floor` rule already enforces, plus the raw spec in a new `node_range` field for provenance. With nothing pinned and no `--target`, compile errors instead of falling back to "latest" — a deliberate divergence from `nub run`, since a compiled artifact's Node version has to be reproducible. Also anchors the self-probe's spawn path: `Command::new` PATH-searches a bare name, so the default `--out` (the entry stem, no directory component) failed to spawn and made `nub compile app.ts` fail at its last step.
The launcher resolved its extraction directory through discovery::cache_dir(), which returns Some whenever HOME is set and never tests writability, so the temp fallback chained off its None could not fire on the boxes it existed for. On AWS Lambda, a readOnlyRootFilesystem pod, or any read-only container the launcher failed with EROFS while a writable /tmp went unused. Replace it with a write-probed chain in the launcher: NUB_COMPILE_CACHE_DIR, XDG_CACHE_HOME/nub, HOME/.cache/nub, then a per-uid directory under the temp dir. Each candidate is created, written to, and checked for a noexec mount; a rejection falls through to the next, and exhausting the chain reports every path tried with its obstacle plus the four remedies. discovery::cache_dir() is unchanged, so nub run and the package manager keep their current behavior. The temp candidate is per-uid, created 0700, and refused when it is a symlink or already owned by another user — the hardcoded world-shared /tmp/pkg shape behind CVE-2024-24828. Foreign ownership disqualifies every candidate; other-write does too, while group-write is tolerated outside the temp dir so a umask-002 host keeps the shared cache. Node acquisition, app extraction, and smol provisioning now share the one probed base rather than resolving separately, and an exec denial under that base reports the noexec mount and its remedies instead of a bare permission error. Claude-Session: https://claude.ai/code/session_01FP7osXQ1YL9zSAZaBzzF6y
A cold start spends seconds decompressing the embedded Node, or longer downloading one under --smol, with nothing on screen. The launcher owns the terminal before it does any of that work, so it now prints a line first. One mechanism covers both shapes: the message is baked into the manifest at compile time and printed once, either before decompression or before provisioning, replacing the smol-only "Installing Node ..." line. The text defaults to "Setting up <output-name>" and is set with --install-message; --no-install-message keeps the old silence. It is printed to stderr on a terminal only, so piped output and logs are unchanged, and a warm run never reaches the call. Claude-Session: https://claude.ai/code/session_01FP7osXQ1YL9zSAZaBzzF6y
Decompression went through read_to_end into a Vec, so a cold run held the whole ~94 MB decompressed Node in memory on top of the bytes it was writing. Streaming the decoder into a buffered file writer measures the same wall time (the decode dominates at ~3 s; the write is ~60 ms of it) at roughly half the peak RSS, which is what decides the run on a memory-capped host whose only writable filesystem is tmpfs charged against the same limit. Claude-Session: https://claude.ai/code/session_01FP7osXQ1YL9zSAZaBzzF6y
Decompression is essentially the whole cold start, and the pure-Rust decoder is what makes it slow. Measured on darwin-arm64 against the 25 MB zstd-19 payload, back to back on one contended host: ruzstd decode 94 MB 2.81 s / 4.04 s / 4.29 s libzstd decode 94 MB 0.70 s / 0.82 s / 0.83 s write 94 MB to disk 0.055 s / 0.059 s / 0.064 s End to end through a real compiled binary, cold, alternating runs: libzstd 2.74 / 3.36 / 3.82 s against ruzstd 5.69 / 5.72 / 11.96 s. Disk I/O is under 2% of the extraction; there is no fsync, hash, or verification pass in the launcher's cold path to account for the rest. ruzstd was chosen to keep the C zstd out of the launcher on size grounds. The fat-LTO'd template measures 522,912 bytes with libzstd against 523,056 with ruzstd, so the premise does not hold, and the launcher already links C through nub-core. Extraction is byte-identical under both decoders, verified by sha256 against the source Node. Claude-Session: https://claude.ai/code/session_01FP7osXQ1YL9zSAZaBzzF6y
The install-message section described a smol-only flag and a progress bar that does not exist; both shapes now print a static line and the default text derives from the output name. Adds a section for the cache the default shape extracts into: the candidate order, NUB_COMPILE_CACHE_DIR, what happens on a read-only or noexec filesystem, and warming the cache at image-build time for a locked-down deploy. Claude-Session: https://claude.ai/code/session_01FP7osXQ1YL9zSAZaBzzF6y
The `nub compile` spike pulled in rolldown 1.2.0, which requires oxc ^0.140,
while nub-native and nub-phantom-core pinned =0.132. oxc is pre-1.0, so those
never unify: a `--features compile` build compiled two full oxc stacks, and the
root Cargo.lock carried both.
Bump every consumer to one version:
- nub-native: oxc/oxc_napi 0.132 -> 0.140, oxc_sourcemap 6.0.1 -> 8.1.0
- nub-phantom-core: the lean parser subset 0.132 -> 0.140
- the root [workspace.dependencies] declaration (read by version-check)
- npm @oxc-project/runtime 0.132.0 -> 0.140.0 (the emit helpers are the JS
half of the same oxc release) plus the three npm lockfiles
Two API breaks in nub-native, both fixed without a behavior change:
- CompilerInterface::handle_errors now takes oxc_diagnostics::Diagnostics
instead of Vec<OxcDiagnostic>.
- DecoratorOptions gained strict_null_checks. Set to true, which reproduces
pre-0.140 emit exactly; wiring it to tsconfig's strictNullChecks would
change existing decorator metadata and is a separate decision.
oxc 0.140 requires rustc 1.95, which nub-native already declared, so the main
workspace MSRV moves 1.93 -> 1.95 and the CI check job installs one toolchain
for all three workspaces.
The lockstep guard the duplication argues for: scripts/check-oxc-lockstep.mjs
asserts every manifest, the npm dependency and all three npm lockfiles name one
oxc version, and that no Cargo.lock resolves two versions of any oxc crate. It
runs as `make oxc-lockstep-check` (folded into `make verify` and `make
version-check`, which no longer carries its own copy of the npm-vs-Cargo
assertion) and as an unconditional ci-gate-required job.
…platform nub compile was Mach-O-only: it errored on a non-macOS host and called libsui's Macho writer unconditionally. Add the ELF (note) and PE (RCDATA resource) legs and route every platform-dependent decision through the TARGET, not the host. - nub-core gains TargetPlatform/ContainerFormat: the eight supported triples, their container format, exe suffix, and whether they are the host. - Per-target strip and sign policy. macOS strips only when the result can be re-signed (stripping invalidates the signature and arm64 will not run an unsigned image); Linux and Windows are never signed, and a foreign format is stripped only with the multi-format llvm-strip. - A foreign target's Node is provisioned into a triple-scoped store so it can never be mistaken for a runnable host Node. - The produced artifact is verified by scanning it for its payload the way the target's loader will; the exec-it __probe check additionally runs when the target is the host, and is skipped with a note otherwise. - The launcher template lookup is target-aware and names the missing file when a foreign template is absent.
…freezes The wrapper existed to satisfy the WHATWG requirement that `MessageEvent.ports` be a read-only array. Node 22.3 replaced the global with undici's (nodejs/node#52370), whose own `get ports()` freezes, so from 22.3 on the wrapper is a strict no-op — but the `typeof globalThis.MessageEvent` guard that reached it was not free. That global is a V8 lazy data property, and any read of it realizes `internal/deps/undici/undici` plus ~112 further builtins (node:http, net, the whole internal/streams tree, worker_threads) on EVERY augmented startup. Gate the wrapper on the Node version instead, so it installs only below 22.3 where the native `ports` really is mutable, and the global is never touched above it. Measured on Node 26.5: bootstrap modules 225 -> 113, startup 128.0 -> 91.0 CPU-ms. The version gate is not a stylistic choice — a feature detect is impossible here. Reading, describing, or redefining the lazy property all materialize it; the one operation that does not (a plain assignment) discards the native constructor irrecoverably, so there is no lazy wrapper to install in its place.
`installLazyEsmPolyfills` decided whether it was running in a worker by looking for `NativeModule worker_threads` in `process.moduleLoadList`. That only reports residency, which anything running earlier in the process can make true on the main thread — and something did: touching the lazy `MessageEvent` global in `installSyncPolyfills` loaded worker_threads, so the main thread took the worker branch and eagerly loaded worker-polyfill.mjs and navigator-locks.mjs, which is exactly the cold-start cost this lazy path exists to defer. Keep the module-list probe as a gate, then confirm with `worker_threads.isMainThread`. Consulting it is free because we only do so once the module is already resident, and it cannot be poisoned: a worker's bootstrap always loads worker_threads before any preload runs.
For an importer inside nub's own runtime directory, `resolveSpec` resolves natively via `createRequire(parentURL).resolve(specifier)`. That call runs through Node's CJS resolver, which invokes the registered resolve hook — back into `resolveSpec` with the same specifier and parent. The two called each other until V8 exhausted the stack, at which point the RangeError landed in the surrounding catch and the resolution completed through the fallback path. It produced correct answers, so nothing failed; it just did so ~849 frames deep. The fast-tier preload made 1698 resolve-hook invocations before user code ran, for three internal requires. Guard the branch: a re-entrant call returns null and delegates to `nextResolve`. The outer frame still short-circuits the user loader chain, and Node's default resolver is what `require.resolve` was going to consult anyway. Resolve-hook invocations at startup drop 1698 -> 5 and startup 91.0 -> 50.4 CPU-ms on Node 26.5.
The MessageEvent regression cost ~112 extra internal modules on every augmented startup with no functional symptom, so no behavior test could have caught it. Measure the module set directly instead: assert that an augmented startup never has `NativeModule http` or `NativeModule worker_threads` resident (either means a preload step realized a lazy global), and that on the fast tier the augmented count stays within 25 modules of the same fixture under `--node`. The budget is a cliff detector for a whole internal tree, not a golden count — `--node` supplies the baseline so it tracks whatever Node the runner is on rather than a pinned number.
…(esm) The lazy `Worker` and `navigator.locks` accessors load their ESM side-effect module on first access, which only works while `loadEsmSideEffect` can use a synchronous require(esm). Under `--no-experimental-require-module` it falls back to a dynamic `import()`, so the getter returned before the module had defined the real global and `new Worker(...)` threw. The path was unreachable until now — the poisoned worker detection sent the main thread down the eager branch, which loaded the same module by a route that tolerated the async fallback. With that corrected, load eagerly whenever require(esm) is disabled. The startup cost is the right trade there: the user opted out of the mechanism the lazy path is built on, and correctness is not negotiable against it.
…e Windows Node correctly Two defects found running the pipeline end to end. The post-strip verification asked the staged Node for --version with the environment inherited. A developer machine routinely exports a NODE_OPTIONS aimed at a different Node — nub's own dev shell does — and Node rejects the whole invocation when one of its flags is unknown to that binary, so a perfectly good stripped Node looked broken and the fallback silently embedded the unstripped one, ~27 MB heavier, behind a note. The check now scrubs the ambient Node configuration so it tests the binary. The launcher looked for its Node under bin/node everywhere. The Windows dist zip puts node.exe at the version-dir root, so a Windows artifact would have failed to find the Node it had just extracted.
Fallout of the version bump. flake.nix pinned the @oxc-project/runtime tarball name + sha512 by hand, so the Nix build would have assembled the embedded runtime/ tree with the 0.132 helpers against a 0.140 transformer. The remaining edits are the doc comments that name the mirrored oxc release. detect.rs's transformable_syntax set carries a "RE-DERIVE THIS SET ON ANY oxc BUMP" mandate. Re-derived: oxc 0.140 has the same es20xx transform modules as 0.132 (es2015..es2022 plus es2026, no es2023/24/25), and a per-syntax probe of both versions at target es2022 shows an identical lowered/verbatim verdict for using, await using, the RegExp v flag, RegExp modifiers, duplicate named groups, the d flag, import attributes, static blocks, private-in, logical assignment, top-level await, optional chaining and import.meta. The set is unchanged. Also corrects transform.rs's claim that the in-process mirror keeps existing cache entries valid with no NUB_VERSION bump: that held for the JS-to-native move, not across oxc versions.
added 5 commits
July 31, 2026 17:43
A fixed-root CJS file loaded through an absolute --require publishes the builtin accessors the generated preamble needs, so an inherited loader cannot redirect them before compiled startup.
added 2 commits
July 31, 2026 18:02
Seven conflicts — six textual, one semantic — all resolved by keeping both
sides' intent rather than picking a side:
- Cargo.toml, crates/nub-native/Cargo.toml: main's jsonc-parser 0.32.4 bump
(required by nub.jsonc) plus this branch's oxc 0.140.0 — the adjacent comment
now names 0.140.0, the version the workspace actually pins.
- crates/nub-cli/Cargo.toml: main's nub-json-guard path dep alongside the
rolldown/oxc compile-feature block.
- Makefile: main's schema-snapshot equality check added to version-check, on
top of this branch's oxc-lockstep-check prerequisite and inlined-manifest
version loop. The inline @oxc-project/runtime check main extended is not
re-added — this branch moved that assertion into
scripts/check-oxc-lockstep.mjs, which is a superset of it.
- .github/workflows/ci.yml: ci-gate needs is the union — windows-embed-roundtrip
and oxc-lockstep from this branch, site-build from main.
- runtime/preload-common.cjs: main's core.dataExtsFor(url) (DATA_EXTS no longer
exists) with this branch's getBuiltin("node:fs"), which keeps an inherited
user loader from redirecting builtins before compiled startup.
- crates/nub-core/src/node/spawn.rs: merged clean but did not compile. This
branch moved NEUTRALIZE_LOCALSTORAGE_ENV into flags.rs so nub-launcher (a
separate workspace) can reach it; main added a new bare-name use of it in
augmentation_environment_restoration. Qualified as flags::.
# Conflicts: # .claude/skills/release/SKILL.md
… fixture The immutable-release jobs are new here and lacked the ignore every other setup-node in this workflow carries; neither runs a package install, so the default cache is never populated. The set-version fixture came from main and creates only nub-native's inlined manifest, while this branch also stamps nub-core's — merged, the script exits on ENOENT before reaching the schema snapshot the test asserts.
…r grant directory_is_stable evaluated INHERIT_ONLY ACEs as if they controlled access to the directory carrying them. Stock Windows always ships C:\ with `Authenticated Users:(OI)(CI)(IO)` granting GENERIC_WRITE|DELETE, so the volume root — the first component of every chain — was rejected, and no runtime-cache or compiled-artifact base could be validated or created on any Windows machine. Skip inherit-only ACEs on non-leaf components, where they grant nothing and the propagated ACE is examined on the next component anyway; the leaf still counts them, because the cache contents created inside do receive them. Also open the runtime_tree fixture's handle for write: Windows SetFileTime needs FILE_WRITE_ATTRIBUTES, which File::open does not carry.
macOS serves the temp dir as /var/... while a process reports its own path as /private/var/..., so the compiled-path assertions failed on every macOS host. This masked a real Node 24 cluster-argv defect behind an environment artifact.
…uses
On Windows the fixture asserted the opposite of its own preceding assertion.
`PathBuf::push` deletes a `.` component from a VERBATIM (`\\?\`) path — std
normalizes there because Windows does not resolve it — so `base.join(".")` on the
canonicalized base returned the canonical target itself, and the alias assertion
demanded that target be rejected. Strip the verbatim prefix first, which is the
spelling a real alias arrives in (`find_public_preload` hands over a stripped
path) and a no-op off Windows, and pin the `.`'s survival byte-wise, since
`Path`'s component-wise `PartialEq` drops it.
…of skipping mklink rejects the \\?\ verbatim spelling tmp_base produces, so on CI this test always took its skip branch and its green proved nothing. Fall back to symlink_dir — same FILE_ATTRIBUTE_REPARSE_POINT the check reads — and skip only on a genuinely withheld privilege, matching the sibling base-reparse test.
runtime_tree refuses a reparse-point ROOT rather than following it, so hashing the link can never be clean and the control was unsatisfiable — invisible while the test skipped. Hash the directory behind it, matching the unix sibling.
Of ~221 integration tests behind the compile feature, CI ran two by name, and the second was unreachable anyway: set -e aborts the step at the first failure and the topology test runs ahead of it. A silent require(esm)-from-CJS defect, where a deferred require resolved to undefined, sat in that gap. The addon these tests need is already built earlier in the same job.
A compiled artifact publishes the outer executable as process.execPath, and
child_process.fork defaults its executable to that value, so fork has to be
redirected back to the real Node binary. That override lived in the bundled
preamble, which runs too late: the emitted entry chunk hoists the program's
static builtin imports above its body, so node:cluster is evaluated first and
internal/cluster/primary.js binds `const { fork } = require("child_process")`
into a module-local const before the preamble can patch the exports object.
Every cluster worker therefore re-executed the ARTIFACT with the forwarded Node
command line as ordinary arguments, and its process.argv carried nub's private
flags plus a duplicated entry: ten entries where the contract is two. An app
that parses argv in a cluster worker got a wrong answer with no error.
Move the policy into compile-bootstrap.cjs, which --require runs before the ESM
graph, so nothing can capture fork ahead of it. It reads both inputs at call
time rather than taking them from the preamble: process.execPath having moved
off the value captured at bootstrap is the compiled-identity signal, and it is
absent in a plain forked child, which loads the bootstrap but no preamble and
must keep stock fork behavior. preload-common.cjs keeps only the
NODE_COMPILE_CACHE concern, leaving one owner for the identity policy.
Verified on Node 22.15, 24.17 and 26.5: cluster worker argv is 2, was 10.
In a compiled artifact `() => require("./dep.mjs")` evaluated to undefined
instead of the module namespace. No error, no warning, successful compile: an
app reading a property off the result got a TypeError far from the cause, and
one reading it defensively got a silently wrong value.
The cause is upstream, in rolldown 1.2.0 and still on rolldown main.
process_global_require_call decides a require's result is discarded by walking
outward to the nearest ExpressionStatement, and oxc represents a CONCISE arrow
body as a FunctionBody holding exactly that node. So `() => require(x)` is
flagged IsRequireUnused and the finalizer emits a bare init_xxx() in place of
`(init_xxx(), __toCommonJS(xxx_exports))`. A CommonJS importee is unaffected,
because its wrapper returns module.exports either way; only an ES-module
importee loses its value. Any surrounding expression, including a block body or
a member access, ends the walk early and escapes the misread.
Give such a body an explicit block and return before rolldown scans it. The
rewrite is a pure syntax normalization -- `() => e` and `() => { return e; }`
are the same program for every arrow, async included -- so it needs no CommonJS
gating and is inert rather than wrong on a module rolldown classifies as ESM.
This is a stopgap; the real fix belongs upstream and is not yet reported.
Replaces the lazy half of the mixed CommonJS/ESM cycle test, which asserted
output no Node version produces: run directly, that fixture throws
ERR_REQUIRE_CYCLE_MODULE on 20.19 through 26.5, because the back-edge is called
during the ES module's own evaluation. The acyclic deferred require is the
contract that is unambiguously Node's, so the test now pins that instead and
keeps the immediate cycle half unchanged.
Its cache-directory probe chain — owner and DACL checks, exec bits, junction rejection — ran on ubuntu only, from the clippy job. That is exactly where the inherit-only-ACE defect lived, and no CI leg could see it. The launcher is already built on each leg for the topology test.
Apple's /usr/bin/strip, given no flags, rewrites a Mach-O executable's dyld export trie: Node 26 darwin-arm64 goes from 159 exported napi_* symbols to 4. macOS addons bind Node-API through the host executable's flat namespace, so every native addon in a compiled artifact then failed dlopen with symbol not found in flat namespace '_napi_create_error' and napi-rs addons, which resolve lazily, loaded with zero exports after printing "Node-API symbol ... has not been loaded". llvm-strip does not have this behaviour, which is why it reproduced only on hosts without Homebrew LLVM - including every GitHub macOS runner, and so every release build. Pass -x on Mach-O (same spelling for Apple strip, GNU strip and llvm-strip): all 159 exports survive and 36 of the 38 MB the default flags recover are still recovered (145.3 MB original, 104.6 MB default, 107.0 MB with -x). Also verify it. Executing the stripped binary cannot catch this class - a Node with no exported Node-API answers --version perfectly - so read the export trie directly and compare against the untouched original. The check is a differential and its only action is to embed Node unstripped, so an image it cannot parse can never fail a build. Reading the classic symbol table instead would report the working llvm-strip case as broken; the trie is what dyld consults. The fixture leaked the host nub's own NODE_OPTIONS into the artifact run, which both preloaded an outside runtime into a run whose point is self-containment and made any --target below the host Node exit 9 on a flag it does not accept.
fs::canonicalize returns the Windows verbatim form, and the compile root embeds that as a module specifier, so every Windows compile died in the bundler with "Could not resolve '\\?\C:\...'". Strip the prefix where a path stops being a filesystem handle and becomes a specifier. The worker-dedup hash needs the same spelling as worker_root or one file derives two ids and emits twice; cwd and launch_root feed resolution and importer paths. No-op off Windows.
3539b65 unpublished the JSON Schemas on purpose — nubjs.com was documenting a config file no installed nub reads — but project_config.rs include_str!s one, so main has been red since: Check, Clippy and every Test leg fail to read it. A unit test's oracle must not be a web-published asset a product decision can withdraw. Keep the crate's own copy and leave the site unpublished.
cache::resolve canonicalizes, and Rust's fs::canonicalize returns the \?\ form. Node cannot resolve a module through one: CJS resolution ends in fs.realpathSync, whose Windows walk lstats the path root, and Node's native ToNamespacedPath re-resolves \\?\C:\ to \\?\C: the volume device rather than its root directory so the call fails EISDIR ... lstat 'C:'. The launcher's own --require bootstrap preload died there, before any application code ran, so no compiled artifact started on Windows. Strip the prefix on the two cache paths that become Node arguments. The canonical spelling stays everywhere the Rust side uses it, including the cache's ownership and containment checks, and libuv re-applies the prefix itself past MAX_PATH. Reproduced and one-variable differential on a real windows-latest runner: same tree, same Node 24, plain entry path resolves and dlopens; verbatim entry path fails with the byte-identical error.
# Conflicts: # crates/nub-cli/src/project_config.rs # crates/nub-core/src/node/spawn.rs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
nub compileturns a JS/TS entry into a standalone executable: a Rust launcher carrying an embedded payload, which extracts real files into a content-addressed cache and runs them under stock Node.Closes #67
Refs #567
The request in #67 suggested Node's SEA. This uses a Rust launcher with an embedded payload instead: SEA cannot carry a per-file executable bit or relocate a native package closure, both of which the addon cases below need.
What this adds
--smol,--platformcross-compilation, and the extraction cache..nodeaddon survives bundling, its owning package and that package's production dependency closure are relocated under__nub_native/<hash>/, preserving real install geometry as ordinary files. This is what makessharpwork: itslibvipsshared library lives in a different package, and the addon resolves it by relative path at load time.Verified
Every claim below is a run, not an inference. Platform is named where it matters.
nub-coreembed-runtime 429/0 and the launcher cache chain 47/0, on a realwindows-latestrunner. Both were failing before: a stockC:\carries anINHERIT_ONLYACE, the DACL walk counted it as an effective grant onC:\itself, andensure_safe_basetherefore returnedNonefor every candidate — no cache base was usable on any Windows machine.sharpcompiles and runs with the source tree andnode_modulesdeleted, its transitivelibvipsresolving from the extracted island. This previously depended onllvm-stripbeing installed: with only Applestripon PATH, the embedded Node lost its exportednapi_*symbols and every addon failed todlopen. Since release builds have nollvm-strip, that shape shipped broken; the strip step is now guarded.clusterworkers see[artifact, entry]and nothing else. They previously also saw nub's private flags and a duplicated entry, because the entry chunk evaluatesnode:clusterbefore the preamble runs and Node'sprimary.jscapturesforkinto a module-local const at that moment.require()of an ES module resolves to the namespace. It previously resolved toundefined, silently, on every Node version; the cause is upstream in rolldown, where a concise arrow body is treated as a discarded expression statement.Test coverage this also fixes
CI ran two of roughly 221 compile integration tests, by name, both gated to a single Node version — and the second was unreachable regardless, since the step aborts at the first failure and the other test ran ahead of it. The Linux leg now runs the whole target. Separately, the launcher's cache-directory probe chain ran on ubuntu only, which is precisely why a defect that made every Windows cache base unusable was invisible; it now runs on the platforms it targets.
Known limitations
Documented in
compile.mdx, with--externalas the remedy where one applies.bindings, or through a specifier assembled at runtime, is not followed. Only static requires are.createRequire(...)result is an ordinary call the bundler cannot rewrite; use a static import.require()normalization is a stopgap until the rolldown fix lands upstream and the pin moves.