fix(pii): detect and redact JSON Web Tokens (JWTs) - #2
Merged
Conversation
The PII guard's comment claimed to flag "JWT-like strings", but the only
token rule matched `Bearer <token>` — and that pattern stops at the first
`.`, so a bare JWT passed through with safe=true while a Bearer-prefixed
JWT had only its header segment redacted. In both cases the payload (which
routinely carries `sub`/`email`/`name` PII claims) and signature leaked.
Add a dedicated JWT rule anchored on the base64url header prefix `eyJ` (the
encoding of `{"`), spanning all three header.payload.signature segments.
Score 92 > bearer-token (90) so overlap resolution redacts the whole token
even when `Bearer`-prefixed. The anchor plus minimum segment lengths keep
false positives near zero — benchmark category thresholds still met.
Adds 3 regression tests and updates the LLM02 coverage row in the README.
Co-authored-by: mattia-mamini-gh <281593356+mattia-mamini-gh@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The LLM02 PII guard documents JWT detection (`pii-output.ts` comment: "JWT-like strings") but never implemented it. The only token rule matches `Bearer `, and its body excludes `.`, so:
eyJ...header.payload.signature, noBearerprefix) returnssafe=trueand leaks completely — verified empirically.Bearer-prefixed JWT redacts only the header segment; the payload (which routinely carriessub/email/namePII claims) and signature still leak.A JWT in model output is both a bearer credential and a PII carrier, so this is a real LLM02 false negative.
What
jwtpattern inPII_PATTERNS, anchored on the base64url header prefixeyJ(the encoding of{"that begins every JWT header JSON object), spanning all threeheader.payload.signaturesegments.Bearer-prefixed.eyJanchor + minimum segment lengths keep false positives near zero. Benchmark category thresholds still met (npm run bench).Verification
npm run typecheck✅ ·npm run lint✅ ·npm test→ 305 passed / 1 skipped ✅ ·npm run bench→ All category thresholds met ✅