Skip to content

fix(core): only honor search-relevant ignore files, not every .*ignore - #430

Open
EricSeastrand wants to merge 1 commit into
zilliztech:masterfrom
EricSeastrand:upstream-pr/ignore-file-allowlist
Open

fix(core): only honor search-relevant ignore files, not every .*ignore#430
EricSeastrand wants to merge 1 commit into
zilliztech:masterfrom
EricSeastrand:upstream-pr/ignore-file-allowlist

Conversation

@EricSeastrand

Copy link
Copy Markdown

Problem

Context.findIgnoreFiles() (packages/core/src/context.ts#L1221-L1243) globs every .*ignore file in the codebase root:

if (entry.isFile() && entry.name.startsWith('.') && entry.name.endsWith('ignore')) {
    ignoreFiles.push(path.join(codebasePath, entry.name));
}

So any build-tool filter that happens to sit in the repo root shapes the search index. A .dockerignore containing a single * line — an ordinary thing in a repo that ships a narrow build context — filters out the whole corpus:

[Context] 📁 Found 0 code files

I hit this on a 2,300-file repository that indexed to nothing. Nothing in the output points at .dockerignore; it just looks like the indexer found no code.

.dockerignore, .npmignore, .eslintignore and friends answer a different question than code search does. "Don't ship this in the image" and "don't lint this" are not "don't let a human read this."

Change

Ignore-file discovery becomes an allowlist of ignore files that actually describe source a reader wouldn't want:

Honored Why
.gitignore build output, deps, generated files
.contextignore this tool's own file
.cursorignore, .codeiumignore other code-search/assistant tools, same intent

Every other .*ignore file present in the root is logged by name so the omission is visible instead of silent:

[Context] 📄 Skipping non-index ignore files: .dockerignore, .npmignore (set CUSTOM_IGNORE_FILES to honor them)

and can be opted back in per-deployment:

CUSTOM_IGNORE_FILES=".dockerignore,.npmignore"

named to match the existing CUSTOM_IGNORE_PATTERNS / CUSTOM_EXTENSIONS.

Selection is a pure function in a new packages/core/src/ignore-files.ts, so it's testable without a filesystem and the honored list is exported for reuse.

Tests

  • src/ignore-files.test.ts — 7 unit tests on the pure selection/parsing logic.
  • src/context.ignore-files.test.ts — 3 integration tests through Context: .dockerignore's * no longer reaches the effective pattern set, .contextignore still does, and the CUSTOM_IGNORE_FILES opt-in restores the old behavior.

I confirmed the first integration test fails on master before the fix (* present in the effective patterns) and passes after. Full core suite: 39/39 green, tsc --noEmit clean in both packages/core and packages/mcp.

Things worth pushing back on

Flagging these myself rather than making you find them:

  1. Allowlist vs. surgical denylist. I could instead have excluded a known set (.dockerignore, .npmignore, …) and kept globbing the rest. I chose the allowlist because the failure mode is unbounded — any tool can drop a new .*ignore in the root and silently shrink someone's index — and an allowlist fails closed against files nobody has thought of yet. But a denylist is strictly less disruptive to existing users, and if you'd rather have that, I'm happy to flip it; the pure module makes it a one-line change.

  2. This changes results in the other direction for some users. Anyone currently relying on .eslintignore / .prettierignore to keep files out of their index will get a larger corpus after this. That's the real cost of the change. It's mitigated by the skip log naming the exact files and the env var restoring the old behavior, but it is a behavior change for existing installs, not purely a bug fix.

  3. The four names are a judgment call, not an exhaustive list. .aiignore, .aiexclude, .claudeignore and others exist in the wild. I deliberately kept the list conservative rather than guessing — adding a name is a one-line PR, and CUSTOM_IGNORE_FILES covers the gap in the meantime. Happy to seed it with more if you have preferences.

Relationship to my other PRs

Independent, no conflicts, mergeable in any order.

#427 fixes the downstream symptom of this bug: when a scan does legitimately yield 0 files, setCodebaseIndexed()'s Issue-#295 guard returns without clearing indexingCodebases, pinning the codebase at "indexing, 100%" forever. That's how the .dockerignore case presented to me in the first place — this PR stops the corpus being erased, #427 makes a genuinely empty scan report as failed instead of hanging.

Also open: #428 (neutral force-reindex messaging), #429 (force during an active index should abort-and-await rather than wipe).

findIgnoreFiles() globbed every `.*ignore` file in the codebase root, so any
build-tool filter that happens to live there shapes the search index. A
one-line `.dockerignore` containing `*` — a completely ordinary thing for a
repository that ships a narrow build context — filters out the entire corpus:
"Found 0 code files", and the codebase indexes to nothing.

`.dockerignore`, `.npmignore`, `.eslintignore` and friends answer a different
question than code search does. "Do not ship this in the image" and "do not
lint this" are not "do not let a human read this."

Discovery is now an allowlist of ignore files that describe source a reader
would not want: `.gitignore`, `.contextignore`, `.cursorignore`,
`.codeiumignore`. Any other `.*ignore` file found in the root is logged by
name so the omission is visible rather than silent, and can be opted back in
with CUSTOM_IGNORE_FILES=".dockerignore,.npmignore" (named to match the
existing CUSTOM_IGNORE_PATTERNS / CUSTOM_EXTENSIONS).

Selection is pulled into a pure module (`ignore-files.ts`) so it is testable
without a filesystem, and exported for callers that want to reuse the list.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant