HoundShield Local Scanner — one engine, both surfaces, proof it never phones home - #316
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Updates to Preview Branch (claude/houndshield-local-scanner) ↗︎
Tasks are run on every commit but only new migration files are pushed.
❌ Branch Error • Fri, 21 Aug 2026 05:03:29 UTC View logs for this Workflow Run ↗︎. |
Supabase Preview ❌ — pre-existing, not this PRNot fixing it here, and here is the evidence it isn't mine. This branch changes 0 files under the repo-root Root cause
It is four indexes, not one — lines 93, 96, 99 and 102 of The bigger findingThe integration watches the repo-root
So this check has never validated a real migration, and the single file it can see has never applied anywhere. Proposed fix — a topology decision, deliberately not made in this PRThe tempting one-liner is wrong. Making the four indexes immutable ( The real fix is to point the integration at the schema that ships:
Both are settings rather than code in this diff, and choosing one silently would set the deploy topology as a side effect — the same class of mismatch that previously disabled three subsystems at once via a root-directory setting. Consequence for this PRMigration Vercel Preview is green on this branch. Generated by Claude Code |
Five defects found by reading live production, not the build. The Vanta
research that prompted this concluded there is nothing to copy: a scripted
browser pass over vanta.com found no interactive AI-scanning widget at all,
only an email-gated "Get a demo" form. HoundShield already ships the ungated
version at /demo#snapshot. So the work was fixing what makes ours leak.
1. /demo ran TWO scanners sharing no code. Deleted the canned one (~400 lines):
a second 9-regex registry, a FAKE 1400ms setTimeout manufacturing latency on
a product whose claim is a sub-10ms local scan, and a "Connect Your Company"
form that connected to nothing. The page said "9 categories" three times
before mentioning the real 53. InstantSnapshot — the real engine, ending on
the PDF — is now the whole page, and the page is a server component because
nothing on it is interactive any more.
Nothing was thrown away with it: the remediation guidance is ported to
CATEGORY_REMEDIATION (keyed by category, so it cannot rot as patterns are
added) and now renders per finding in a native <details>; the sample
scenarios are ported, re-measured against the REAL engine, and extended to
cover all three verticals. The weakest original ("Network Scan") produced a
single finding against the shipped engine because it leaned on patterns only
the canned scanner had; replaced with a legal/M&A scenario, measured at four.
2. A fabricated pattern count was live on TWO surfaces, not one. app/demo said
"16 detection engines across 90 local patterns" and app/features said "90
shipped patterns" — in a file that already imports PATTERN_COUNT and whose
header comment says never to retype the number. 90 is the double-counted
figure engines.ts was written to delete; both survived as string literals
where its guard could not see them. Now interpolated, and engines.test.ts
gained the missing sibling check: it validated "N engines" and never "N
patterns". The new check accepts PATTERN_COUNT or ENGINE_COUNT, because "16
pattern families" is correct English for the engine list and a guard that
forces correct copy to change is measuring the wrong thing.
3. /api/health reported status "degraded" because of a false alarm about
itself: payments_webhook returns "configured" — its HEALTHY status — and
"configured" was missing from OPERATIONAL_VALUES. Third time this module has
been bitten by a value its own vocabulary did not know, so the fix closes the
class: a test now reads the state literals out of the producers and fails on
any that is neither operational nor a listed degraded state. reset_app_url
moved to INFORMATIONAL_KEYS ("set"/"default" are both working states).
4. Snapshot leads were emailed and stored NOWHERE, and the route returned 503
without recording anything when RESEND_API_KEY was unset — so an
unconfigured mailer dropped the lead outright. Migration 037 adds
snapshot_leads (counts only; there is deliberately no column for prompt
text), the route writes BEFORE the mailer gate, and neither rail failing
loses the lead. Also rate limited: it was unauthenticated and sent two
emails per call. /api/health names 037 while it is unapplied.
5. /partners still offered "Up to 20% revenue share on subscriptions" beside
the correct $399 wholesale offer. HoundShield sells no subscription — same
defect as /terms section 4 describing refunds for monthly plans we never sold.
Also fixed, found by the suite: the cold-outreach test guide told buyers to
click "Scan for Threats" — a control on the deleted scanner — and named the old
sample buttons from its own hardcoded copy. Sample names and control labels now
live in one module both the page and the emails import, and the guide gained
the PDF step the demo script mandates it end on. CLAUDE.md's competitive map
called Vanta "docs only, no AI gateway"; they now ship SPRS/SSP/POA&M and court
the same RPO channel, and their AI Governance is waitlist-only and does agent
inventory, not prompt-content inspection.
The pre-commit secrets guard blocked this commit on the demo's AWS-key
fixtures. It uses the same AKIA regex as the product's own detector, so any
fixture demonstrating key detection necessarily trips it, and every future edit
to these scenarios would need --no-verify. Two exact synthetic literals are now
neutralized before matching rather than the lines being skipped: a real key
sharing a line with one is still caught, probe-tested both ways.
Verified in a real browser against the production build, because the privacy
boundary cannot be proven from a build: pasting the defense scenario yields 7
critical findings across 3 NIST controls in a measured 2ms, working remediation
panels, the PDF button and the $499 CTA — and none of the SSN, AWS key, CAGE
code or contract number is echoed back anywhere on the page.
Gates: tsc 0 · 3027 tests / 214 files (was 2998/213) · lint 0 errors, 0
warnings in any touched file · build exit 0 · coverage exit 0 (36.25%) ·
structure + leak guards pass. Every new guard probe-tested in both directions.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RAJGs35LcXGuqyiiwHUi3j
The after-login dashboard sent the customer's pasted text off their machine.
components/dashboard/live-scanner.tsx POSTed it to /api/scan, which forwards to
Bytez (api.bytez.com) and Google Gemini. On a product sold to organisations
handling CUI and PHI, the one place a logged-in customer was invited to paste a
real prompt was the one place it left their device — and even with both API keys
unset the text still reached the Vercel plane, which CLAUDE.md records as NOT
the CUI data path and not FedRAMP-authorized.
/command-center/scanner now runs the same in-browser engine as the public demo.
The route and its nav entry already existed, so this needed no new route and no
nav change. live-scanner.tsx is deleted.
WHAT IS SHARED. lib/scan/local-engine.ts wraps the existing snapshot engine
rather than replacing it, and components/scan/ renders one component tree on
both surfaces with only design tokens differing — light on /demo, dark in the
Command Center. glass-card and the button idioms were already common to both;
re-declaring them per theme is how two skins drift apart.
TWO RUGGEDNESS DEFECTS, BOTH FOUND BY MEASURING.
1. The browser scan never decoded base64 or hex. decodeObfuscation in
lib/classifier/patterns.ts does this for the server engine, but it is built
on Buffer, a Node global, so it could not run in a browser — the demo
under-reported its own engine and encoded CUI passed it. There is now an
atob/TextDecoder equivalent, and findings say which encoding hid them.
2. The input ceiling was guessed at 200,000 chars and the suite caught it: the
scan blew a 5-second timeout. Benchmarking showed cost is not driven by size
but by the length of a single unbroken non-whitespace RUN, because several
shipped patterns backtrack across a uniform character class:
realistic, with whitespace 100,000 chars → 38ms
one unbroken run 50,000 chars → 10,030ms
Bounding the SHAPE rather than the input takes that 50,000-char run to 21ms
while leaving realistic pastes untouched, and the decode path still sees the
original so base64 CUI is still caught (asserted). Patterns themselves are
untouched — CLAUDE.md requires compliance-specialist for that.
THE PROOF. The claim "your prompt never leaves your device" was prose, and prose
cannot fail. It is now an instrument: the page wraps fetch, XMLHttpRequest,
navigator.sendBeacon and WebSocket for the duration of the scan and renders the
count, with any observed request listed rather than summarised away. Scoped to
the scan window on purpose — a counter running for the page lifetime would
eventually catch route prefetching and cry wolf. Plus a counts-only proof
receipt hashed in-browser with crypto.subtle.
scripts/smoke-local-scan.mjs drives the on-disk Chromium over CDP against the
production build with the Network domain enabled and asserts ZERO data-bearing
requests across nine adversarial inputs: base64- and hex-smuggled CUI,
zero-width joiners, homoglyphs, CRLF, whitespace-only, 60k realistic, and an 80k
unbroken run. 53/53 passed. Its first version failed on a lazy-loaded .woff2
that landed in the window — a same-origin static font cannot carry the prompt,
so the rule now targets what exfiltration actually looks like (cross-origin, a
request body, or a secret in the URL), which is stricter than a bare count.
The auth-gated dashboard route is reported SKIPPED, never passed.
GUARDS, each probe-tested red then green. no-network.test.ts walks the engine's
transitive import graph and fails if it reaches an I/O module or contains an
outbound call — comments stripped first, since the engine's own docblock
explains the boundary in those words. theme-parity.test.ts pins BOTH the token
map and the rendering: probing found the first version only compared the DOM
against the map, so corrupting the map moved the goalposts.
Also: docs/LAUNCH-CHECKLIST-2026-06.md directed a "C3PAO outreach wave", which
CLAUDE.md records as legally prohibited under 32 CFR Part 170 / ISO 17020, and
sold $159/mo with eight price-IDs for tiers that do not exist. Rewritten against
Stage 1 as LAUNCH-CHECKLIST.md; the old path now redirects to the correction
rather than 404ing. Two guards that read `git ls-files` now filter by existence,
so a tracked-but-unstaged deletion no longer fails them for an unrelated reason.
Gates: tsc 0 · 3073 tests / 217 files (was 3064/216) · lint 0 errors, 0 warnings
in any touched file · build exit 0 · proxy 92 tests + tsc 0 · smoke 53/53.
Mode B verified without Docker (unavailable here): booted the proxy, sent the
demo prompt, got HTTP 403 "CAGE code (AC.L2-3.1.3)", one audit event, and
confirmed the audit store contains no raw CUI value.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RAJGs35LcXGuqyiiwHUi3j
Repository and user-facing copy should carry our own brand. The homepage
comparison advertised two rivals by name in card headings; it now names the
CATEGORY each belongs to — "Cloud-routed DLP" and "Productivity-suite
governance". The comparison is just as informative, and the claim is about
architecture rather than about them.
Two guards pinned the old names and are inverted rather than deleted: they now
assert the category headings, so the policy is enforced by test instead of by
memory. Scoped to `getByRole('heading')` because the intro paragraph legitimately
uses the same phrase in prose.
DELIBERATELY UNCHANGED, because "keep it where it is needed" applies:
· /subprocessors and the legal pages name every processor. GDPR Art. 28(2)
REQUIRES a named list; genericising it would be a compliance gap on a
compliance product.
· Market statistics keep their source attribution. Removing the citation
turns a sourced figure into an unsourced one, which is the fabricated-metric
failure on the NEVER-DO list.
· Vendor SDK imports and integration code cannot be renamed.
· /compare/[slug] is left intact and flagged for a separate decision. A
comparison page cannot function without naming what it compares, and
CLAUDE.md ranks that content as the highest-value SEO article — deleting it
removes an acquisition channel, so it is not a change to make silently.
Gates: tsc 0 · 3073 tests / 217 files · build exit 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RAJGs35LcXGuqyiiwHUi3j
c135d40 to
f48d464
Compare
…scanner PR #316 landed on main and rewrote the same surfaces this branch touches: it deleted the fake nine-pattern demo scanner and extracted the real one into a shared components/scan/LocalScanPanel used by both /demo and the after-login /command-center/scanner, with a network-witness proof panel and named sample scenarios. That architecture is better than what this branch had, so the three conflicted files take main's version wholesale. Resolving in main's favour alone would have silently dropped the change the founder asked for: main's commerce mode still hands the visitor a downloadable gap-report PDF for free, which is exactly the give-away this branch removed. So the gate is re-applied on top of the new structure rather than re-litigated -- LocalScanPanel's commerce block now renders the assessor-ready report LOCKED behind a single "Unlock the full report -- $499" CTA, and generatePdf and its now-dead imports and phase are removed. Two guards were updated to describe the new behaviour rather than the old: InstantSnapshot's two download-era tests now assert nothing is ever saved to the device and that the locked CTA is present, and the theme-parity guard keys off "unlock the full report" -- still commerce-only, so it continues to prove a paying customer is never re-sold the product inside their own dashboard. Auto-merge had also produced an await inside a non-async test; corrected. Gates: tsc 0 - lint 0 errors - 3091 tests passing / 222 files - build exit 0. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TSBrYp65drrzFV9J1wRyrj
Summary
The after-login dashboard sent the customer's pasted text off their machine.
components/dashboard/live-scanner.tsx:81POSTed it to/api/scan, which forwards to two third-party AI vendors. On a product sold to organisations handling CUI and PHI, the one place a logged-in customer was invited to paste a real prompt was the one place it left their device — and even with both vendor keys unset the text still reached the hosted plane, whichCLAUDE.mdrecords as not the CUI data path and not FedRAMP-authorized./command-center/scannernow runs the same in-browser engine as the public demo. The route and its nav entry already existed, so this needed no new route and no nav change. The cloud scanner component is deleted.Closes #
Change type
Impact and operating considerations
Operational impact, threat-model note, and rollback plan:
This removes an egress path rather than adding one: pasted text in the dashboard no longer leaves the browser. No new environment variable, no migration, no schema change. Detection patterns are untouched —
CLAUDE.mdrequirescompliance-specialistbefore any engine change, and the fix here is a defensive bound in the caller./api/scanis left in place but now has zero runtime callers. It remains public and unauthenticated and still performs outbound vendor calls. Flagged, not removed — that was a separate decision.Rollback: revert the commit. Nothing outside the app changes.
Validation
npx tsc --noEmitnpm run lintnpm run test:coveragenpm run buildnpm run test:coveragenpm run bench— not run; no proxy source changedResults:
npx tsc --noEmitnpm run testnpm run lintnpm run buildscripts/smoke-local-scan.mjsBrowser proof.
scripts/smoke-local-scan.mjsdrives the on-disk Chromium over CDP against the production build with the Network domain enabled and asserts zero data-bearing requests during scan and PDF generation, across nine adversarial inputs: base64-smuggled CUI, hex-smuggled, zero-width joiners, homoglyphs, CRLF, whitespace-only, 60k realistic, and an 80k unbroken run. Two independent measurements agree — CDP's log and the page's own witness both read zero. No planted secret reached the DOM or the redacted preview. The auth-gated dashboard route is reported SKIPPED, never passed.Mode B verified (no Docker available in CI container): booted the proxy directly, sent the demo prompt → HTTP 403
CAGE code (AC.L2-3.1.3), one audit event recorded, and the audit store confirmed to contain no raw CUI value.Two ruggedness defects, both found by measuring
1. The browser scan never decoded base64 or hex. The server-side decoder is built on Node's
Buffer, so it could not run in a browser — the demo under-reported its own engine and encoded CUI passed it. There is now anatob/TextDecoderequivalent, and findings state which encoding hid them.2. The input ceiling was guessed at 200,000 chars, and the suite caught it — the scan blew a 5-second timeout. Benchmarking showed cost is not driven by size but by the length of a single unbroken non-whitespace run, because several shipped patterns backtrack across a uniform character class:
Bounding the shape rather than the input fixes it while leaving realistic pastes untouched, and the decode path still sees the original so base64 CUI is still caught (asserted). A perf guard locks the numbers in.
The proof panel
"Your prompt never leaves your device" was prose, and prose cannot fail. It is now an instrument: the page wraps
fetch,XMLHttpRequest,navigator.sendBeaconandWebSocketfor the duration of the scan and renders the count, listing anything observed rather than summarising it away. Scoped to the scan window on purpose — a counter running for the page lifetime would eventually catch route prefetching and cry wolf. Plus a counts-only proof receipt hashed in-browser withcrypto.subtle, so it can be re-hashed independently.Guards — each probe-tested red, then green
lib/scan/__tests__/no-network.test.tswalks the engine's transitive import graph and fails if it reaches an I/O module or contains an outbound call. Comments are stripped first, since the engine's own docblock explains the boundary in those words.components/scan/__tests__/theme-parity.test.tsxpins both the token map and the rendering. Probing found the first version only compared the DOM against the map, so corrupting the map moved the goalposts.Also
docs/LAUNCH-CHECKLIST-2026-06.mddirected an outreach wave at assessor organisations that are legally prohibited from recommending tools to clients they assess (32 CFR Part 170 / ISO 17020) — on the NEVER-DO list — and sold $159/mo with eight price-IDs for tiers that do not exist. Rewritten against Stage 1 asLAUNCH-CHECKLIST.md; the old path now serves the correction rather than 404ing.Two guards reading
git ls-filesnow filter by existence, so a tracked-but-unstaged deletion no longer fails them for an unrelated reason.Review checklist
Generated by Claude Code