Skip to content

Glob walk: a drive root is a base, and two ways a directory could be taken for absent - #575

Merged
ronleizrowice-ant merged 14 commits into
mainfrom
fix/glob-walk-root-base-and-failures
Oct 10, 2026
Merged

ronleizrowice-ant merged 14 commits into
mainfrom
fix/glob-walk-root-base-and-failures

Conversation

@ronleizrowice-ant

@ronleizrowice-ant ronleizrowice-ant commented Sep 19, 2026 •

Copy link
Copy Markdown
Collaborator

Four faults in the walk that expands a glob entry. The walk was rewritten under this branch (#620, #627, #628, #644, #645), so each fault was checked again on today's main, and each fails its test there.

What changes

  • A drive root or a share root is a base like any other (Windows). C:/Users*/id.pem and //server/share/x*/y.key found nothing: the base came back with the root's separator, a component the pattern does not have. C:/*.pem listed the drive's current directory, not its root. / is still refused. A doubled separator before the first wildcard (C:\a\\*.pem, or ~\*.pem when the home directory is a drive root) is walked as on main.
  • A directory that answers ENOENT under one name is asked for again under its others. It was taken for absent under all of them, and its matches stayed readable.
  • What the shorter name says is added to what the real path said. A call is made on the real path, which crosses no link, and after any failure on the shorter name the walk got there by, as on main. The short name crosses links that a sandboxed command can repoint, at nothing or at an empty directory, and main took its answer in place of the real path's: a directory that could not be read came back as absent or empty, and nothing was denied.
    • Now a real path that fails with anything but ENOENT or ENAMETOOLONG has the directory denied whole (a link that matches: covered under its own spelling), whatever the short name says. For a link's own stat and realpath, ENOTDIR and ELOOP are absences too.
    • What the short name lists or leads to is walked as well, so what a link in there leads out to is still masked. Where the short name alone says a matching link leads is denied besides.
    • Of two failures, the one that is not an absence counts.
  • A pattern that is no regular expression says which pattern it is. Still a SyntaxError, with the engine's error as its cause.

How it meets #645

#645 keeps a failed listing (any error but ENOENT) for the patterns that follow in the same wrap. That stays. In a walk's own map nothing is kept, so a second name still gets its try. An absence is never kept.

Where it mounts more than main

  1. The ENOENT case above: the matches are masked.
  2. A directory reached by a name shorter than its real path (a rule spelled through a link, such as ~/**/.env with a linked home directory), whose listing by the real path fails with anything but ENOENT or ENAMETOOLONG: denied whole, where main went by the short name. This hurts when the failure is momentary. With the working directory HOME/proj, one failed first listing of it gives --bind CWD CWD … --tmpfs CWD and nothing restores it, so what the command writes there is lost. It lasts one wrap. Under a plain spelling main does the same.
  3. The same for a matching link's own stat and realpath: covered as spelled.

Where it mounts less than main

Within one pattern, nowhere, in 28,980 injected cases under Node and 18,900 of them again under Bun. They include a link that matches inside a directory the same pattern denies, reached through a shorter directory link.

Between patterns, in the one case main took for an absence: the real path fails while the short name says ENOENT. main kept nothing, and every later pattern asked again. Here the failure is kept for the wrap like any other and the directory is denied whole. If the failure has cleared by then, a later pattern does not find what a link leads out of that directory to. main does the same after any failure it recognises.

Not covered, as on main

A matching link that is kept under its own spelling, beneath a directory the same pattern denies, gets no mount of its own: the directory above it counts as hiding it. So where such a link leads is masked only if a name said where that is. A follow-up takes this up.

Dropped from this branch

  • The cap of 1024 pattern pieces: capped patterns lost a match behind a link that main finds.
  • Three refactors that changed no behaviour.

Testing

  • 19 tests fail on main's source. 48 mutations each fail a test.
  • File system calls for one wrap with 300 rules, main and here: 9,337 and 9,337 (names present), 8,137 and 8,137 (absent), 8,116 and 8,116 (bases absent), 31,919 and 31,919 (20 unlistable directories). The mounts are identical.
  • Worst case found: a directory that answers ENOENT under many short names costs two failed listings per name and rule. 200 such links and 300 rules, injected: 120,300 tries against 300, 1.7 s against 0.9 s. Under Bun the state can be real: links to /proc/self/fd/<n> of a removed directory pass stat and realpath while readdir says ENOENT.
  • With real bubblewrap, against main: a folder with a backslash in its name under Bun, whose realpathSync fails on such a path; ENOENT once on a listing; a link leading out of a directory whose listing fails once; a directory that never lists by its real path; a matching link inside a denied directory whose stat or realpath fails once, under Node and Bun. All masked or hidden.
  • Windows: eight rows run on the Windows jobs only, four through expandWindowsFsPaths with doubled separators and four that walk from a real drive root. All eight pass there, on x86-64 and arm64. The share root is only computed.

Merge order

Conflicts with #672, #607 and #573 in the walk and the README.

…or on one

path.dirname keeps the separator of a root it returns, so the base of a
pattern whose first wildcard sits right below one is 'C:/', or
'//server/share/' on a share. Split into path components to seed the walk,
either ends in an empty name no position can consume: the automaton starts
nowhere, the pattern matches nothing, and the entry is dropped with no error
and no warning. On Windows that is every denyRead, denyWrite, allowRead and
allowWrite glob of the shape C:\Users*\**\.env.

A base now never carries a separator, and one that is a filesystem root --
'/', a drive root, the root of a UNC share -- is refused and warned about the
way the POSIX root already was, in the walk and in the manager's glob-pattern
warnings, through one predicate so that the two cannot drift.
…ough a link

A pattern that cannot be read one path component at a time is matched against
whole paths, and no directory is listed through a symlink under it. A link
leading out of the walk's tree was dropped: what the pattern matches in there
is found under no other name, so the deny lost it. Such a link is now recorded
as a place the walk could not enumerate, which the read-deny expansion denies
whole and binds nothing back beneath. A link within the tree still loses
nothing, since every directory there is listed under its own name.

The link the walk follows is now named where it is followed, which is the one
place that knows it: what is found through it is reported where it really
lives, so the warning about a mount outside the pattern's base cannot name it.
The positions a directory had been listed for were marked before the listing
and never rolled back, so a listing that failed answered for every later name
that led there: a transient EMFILE, or a real path too long to name, dropped
every match beneath the directory for the rest of the walk. A failure is
uncached again, with one record per real directory, so the directory is named
once in `unlisted` however many names fail on it, and a retry costs one
listing per name that leads to it -- each of them an entry of a directory that
is itself listed once per position.

A filesystem call also no longer answers from a second name for anything but a
path too long to be one. Every other errno belongs to the object rather than
to the name, and the shorter name crosses links a sandboxed command owns:
EACCES on the real path followed by ENOENT on the short one read as "nothing
is there", which is the fail-open the unlistable and uninspectable branches
exist to prevent.
Both of globPositions' regular expressions are compiled inside the try that
guards the automaton, and a pattern that compiles into no regular expression
at all is raised under its own name rather than thrown from wherever the
sandbox happened to be setting up. A pattern the automaton cannot be built for
-- including one so long that building it would overflow the stack, which a
bound on the piece count now refuses -- falls back to matching whole real
paths with a warning that says so, rather than quietly covering less: on a
deny path that fallback also stops the walk descending symlinked directories.

Beside that: the directory-form automaton is built only for the caller that
asks for it, the positions are no longer sorted on every step for an order
nothing reads, and the state written back into the automaton is held rather
than cast.
… the separator

Nothing in the tree held the walk's automaton and globToRegex together, though
they are the two halves of one rule and are edited by different hands. A
seeded, bounded property test compares them over generated trees and patterns:
what the walk returns must be exactly what the regular expression matches
beneath the pattern's base. It takes about 30ms.

Also: a bracket range that holds the separator ([+-9]) joins the set that does
([s/]), and the fixture that builds a path near PATH_MAX no longer spins for
ever when the temporary directory is exactly the length that leaves it nothing
to add.
A link back up the tree is not descended, so it denies what it reaches only
when the link is itself a match; the README said it always did. It now also
says what a pattern that cannot be split does with a link that leads out of
the starting directory.
@ronleizrowice-ant ronleizrowice-ant mentioned this pull request Sep 19, 2026
The spy type is not disposable under the type packages the test
typecheck runs with, so `using spy = spyOn(...)` does not compile there.
Each becomes a const restored in a finally, as the other suites do it,
with the same scope the declaration had.
…base

Two things the earlier commits on this branch got wrong.

A pattern that cannot be read one component at a time denies whole a
link that leads out of its starting directory, since nothing in there is
listed under any name. "Out of the starting directory" included every
ANCESTOR of it, and the test that leaves a link leading up alone came
after that branch, where such a pattern never reaches it. So with an
unsplittable deny glob, one `ln -s ../.. x` in the tree put a tmpfs over
the directory holding the project, with nothing bound back beneath it,
for every later command. A link that leads back up to the starting
directory or above it is now left alone, as it is for a pattern that
can be followed.

The refusal of a filesystem root as a base was written for `/`, which is
refused so that a whole filesystem is not listed. It also matched a
drive root and a share root, so every Windows pattern whose only literal
directory is one (`\\server\share\*.pem`, `D:\**\*.key`) matched nothing
and was not enforced, where the last release walks and enforces them.
Only `/` and a pattern with no literal directory are refused now. A base
never carries a trailing separator; a drive root is handed to the
filesystem as `C:/`, since `C:` alone is that drive's current directory.

The README also says what is still not covered: a link whose target
cannot be looked at, and that the pattern would continue through rather
than match, is not followed and nothing is denied for it.
Comments only: the JavaScript emitted with comments removed is
byte-identical for every file, and the test names are unchanged.

Kept: every invariant and what could be done without it, the
documentation of exported names, and the reasons for what is not done
the obvious way. Cut: how the code came to be as it is, narration of
the lines that follow, and explanations repeated at several sites.
…e alone again

For a pattern that cannot be followed one path component at a time (a
set holding a wildcard, `id[*].pem`), the walk denied the target of
every symbolic link to a directory outside the base, anywhere in the
base's subtree, whether or not the pattern could match beneath it, and
bound nothing back under it. One link such as `vendor/sdk/conf -> /usr`
hid /usr, and every sandboxed command failed to start. Anything able to
write the tree can make such a link.

That rule is removed: such a pattern descends no symbolic link, as
before this branch, and the README says what that leaves uncovered.

Also: the over-long-pattern fallback logs once and says why; the
comment on the piece cap no longer names a number that is not true.
dylan-conway
dylan-conway previously approved these changes Sep 28, 2026
ronleizrowice-ant added a commit that referenced this pull request Oct 4, 2026
#645 keeps a failed listing for the patterns that follow. #575 lets a
second name for a directory have its try after a listing that failed.
Here a failure is kept only in a map the caller handed in, where the deny
patterns of one configuration meet; a walk's own map keeps none. Between
deny patterns nothing is lost: the first failure has the directory denied
whole. One join case says so.
The walk was rewritten under this branch. What is left of it on today's:

- a drive root, and the root of a share, is a base like any other;
- a directory that is not there under one name is asked for again under its
  others. A failure is kept for the patterns that follow only in a map the
  caller shares between them, so a walk by itself still tries each name;
- the shorter name is tried when the real path is too long to name, and
  for no other error;
- a pattern that is no regular expression says which pattern it is.

The cap of 1024 pieces is gone. main splits such patterns, and capped they
were matched against real paths only, which lost a match behind a link.
Three refactors that changed no behaviour are gone too.
@ronleizrowice-ant ronleizrowice-ant changed the title Glob walk: a drive-root base no longer drops the pattern, and three ways a failure could lose a match Glob walk: a drive root is a base, and two ways a directory could be taken for absent Oct 8, 2026
The short name was asked after ENAMETOOLONG only. That masked less than
main: after ENOENT on a listing, after any failure of a link's own stat or
realpath, and for what a link leads out of a directory that is denied whole.

It is asked after any failure again, as on main. A real path that failed
with anything but an absence or ENAMETOOLONG still has the directory denied
whole, whatever the short name says; what that lists is walked as well. Of
two failures the one that is not an absence counts.

The base of a pattern is main's code with the posix dirname: stripping a
separator lost patterns with a doubled one before the first wildcard. A
pattern that does not compile is a SyntaxError again, with its cause.
…enied too

A matching link whose own stat or realpath failed by the real path is
covered under its own spelling. Inside a directory the same pattern denies,
that spelling is hidden by the directory above it, and what the link leads
to, outside that directory, got no mount, though the short name had said
where it is.

That place is now a match as well. The link stays covered as spelled.
@ronleizrowice-ant
ronleizrowice-ant enabled auto-merge (squash) October 9, 2026 20:50
Conflicts with #672 in the walk's listing call and two README bullets: the
listing asks for the type of an entry listed without one, and passes what
counts as an absence and what to do with a directory that was not read.
@ronleizrowice-ant
ronleizrowice-ant merged commit 8436183 into main Oct 10, 2026
16 checks passed
@ronleizrowice-ant
ronleizrowice-ant deleted the fix/glob-walk-root-base-and-failures branch October 10, 2026 19:17
ronleizrowice-ant added a commit that referenced this pull request Oct 10, 2026
One conflict, with #575, at two call sites in the walk: main's side, asking
realPathOf.
ronleizrowice-ant added a commit that referenced this pull request Oct 10, 2026
#575 is on main as one commit. Against main this branch holds its own change
and nothing else.
ronleizrowice-ant added a commit that referenced this pull request Oct 10, 2026
It brings main, with #575. Two of #575's test helpers stand in for the call
that gives the real path: they stand in for the native form.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants