Skip to content

matchPromptWithReason: phrases/allOf/anyOf use unbounded .includes() while noneOf uses \b — short phrases substring-match inside unrelated words #51

Description

@kaiohenricunha

Summary

hooks/src/prompt-patterns.mts scores skills with an asymmetric matcher: the noneOf branch uses a word-boundary regex, but the positive branches (phrases, allOf, anyOf) use plain String.prototype.includes(). Any skill declaring a short phrase ("PPR", "stale", "use", "deploy", etc.) force-injects itself whenever the prompt contains a larger English word that happens to embed that substring.

This is a narrow, mechanical bug. It's distinct from (though aggravated by) the architectural issue in #38 and the over-broad-patterns issue in #19: even if both of those ship, "ppr" will still match "appropriate" inside a legitimate Vercel repo because the matcher itself is wrong.

Plugin version: 0.32.5.

Minimal repro

echo '{"prompt":"please plan a fix to avoid this if appropriate","cwd":"/tmp","session_id":"repro","hook_event_name":"UserPromptSubmit"}' \
  | node hooks/user-prompt-submit-skill-inject.mjs

Actual:

[vercel-plugin] Best practices auto-suggested based on prompt analysis:
  - "next-cache-components" matched: phrase "ppr" +6
…
You must run the Skill(next-cache-components) tool.

Expected: no match. The prompt has nothing to do with Next.js or Partial Prerendering. "ppr" matched the substring appropriate. skills/next-cache-components/SKILL.md declares phrases: ["PPR"], which normalizes to "ppr".

The same class of false positive triggers on any short phrase embedded in an ordinary word — approach, approval, appropriate, etc.

Root cause

hooks/src/prompt-patterns.mts, matchPromptWithReason:

// noneOf — CORRECT (uses word boundaries)
for (const term of compiled.noneOf) {
  const escaped = term.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
  const re = new RegExp(`(?:^|\\b|\\s)${escaped}(?:\\b|\\s|$)`);
  if (re.test(normalizedPrompt)) { /* suppress */ }
}

// phrases — BROKEN (plain substring)
for (const phrase of compiled.phrases) {
  if (normalizedPrompt.includes(phrase)) {        // ← no word boundary
    score += 6;
  }
}

// allOf — BROKEN
for (const group of compiled.allOf) {
  const allMatch = group.every(t => normalizedPrompt.includes(t));  // ←
  if (allMatch) score += 4;
}

// anyOf — BROKEN
for (const term of compiled.anyOf) {
  if (normalizedPrompt.includes(term)) anyOfScore += 1;              // ←
}

findMatchedPhrases (same file) has the same bug.

The asymmetry — noneOf using \b, everything else using .includes() — looks like an oversight rather than an intentional design choice.

Proposed fix

Extract the noneOf regex into a small helper and use it for all four sites. Multi-word phrases like "partial prerendering" and "use cache" still match, because the boundaries sit on the outer edges only, not between interior words.

function phraseMatchesWordBoundary(normalizedPrompt: string, phrase: string): boolean {
  if (!phrase) return false;
  const escaped = phrase.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
  return new RegExp(`(?:^|\\b|\\s)${escaped}(?:\\b|\\s|$)`).test(normalizedPrompt);
}

Then replace the four .includes() sites in matchPromptWithReason and findMatchedPhrases.

Verification

Using a locally patched prompt-patterns.mjs:

Prompt Before After
please plan a fix to avoid this if appropriate phrase "ppr" +6 → injected no match
i need to approach this approval process with appropriate care phrase "ppr" +6 × 3 → injected no match
help me add partial prerendering to this next.js 16 app injected (legit) still injected via "partial prerendering" and "next.js"
how do I use cacheTag in next.js 16 injected (legit) still injected

No regressions on legitimate multi-word phrase matches.

Related

Happy to open a PR with the helper plus a unit test in hooks/prompt-patterns.test.ts if that's useful.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions