Skip to content

Add a credential-leak example to the demo data - #343

Merged
jcsalterego merged 1 commit into
mainfrom
jchen/add-credentials-scan-to-argus-demo
Aug 19, 2026
Merged

Add a credential-leak example to the demo data#343
jcsalterego merged 1 commit into
mainfrom
jchen/add-credentials-scan-to-argus-demo

Conversation

@jcsalterego

Copy link
Copy Markdown
Contributor

Why

The secret scanner (#327) has no fixture that trips it, so the exposed-credentials warning banner, count badge, recommendations entry, and timeline marker are all invisible in the public demo — there's nothing to demonstrate the feature with.

What

  • Added one scenario turn (sales-agent-ops project → "Set up the HubSpot MCP server for the outreach agent"): Rachel pastes a Slack bot token into the terminal to test a webhook directly. The token is fabricated (sequential digits, never issued) but shaped to match the slack-bot-token rule, and is joined from string parts rather than written as one literal so it doesn't read as a real token to GitHub's own push-protection secret scanning.
  • Wired the same inline secret scan the real indexing pipeline runs (scanSessionForSecrets in toMaterializeSessions) into scripts/demo/generate.ts. The demo builds MaterializeSession records directly and calls store.materializeSessions() without going through toMaterializeSessions, so without this the scenario text alone would never produce a finding — the scan just never ran on demo data.

Testing

  • bun run scripts/demo.ts --no-serve --seed 42, then queried the seeded store directly:
    session_id                            category     interaction_seq  chunk_type  hint
    ------------------------------------  -----------  ---------------  ----------  ---------
    5b8d2a47-2bff-463b-9e75-92178616924f  slack_token  1                response    xoxb…j3sC
    
    One finding, on the intended task/interaction, secret_scan_version stamped.
  • Confirmed in the running app: warning banner, dismiss action, count badge, recommendations entry, and timeline marker all show up on that session.
  • bun test test/secret-scan.test.ts test/demo.test.ts and tsc --noEmit are clean.

The secret scanner (#327) has no fixture that trips it, so the exposed-
credentials warning banner, count badge, recommendations entry, and
timeline marker are all invisible in the public demo.

Add one scenario turn where Rachel pastes a Slack bot token to test a
webhook directly (obviously fabricated — sequential digits, never
issued — but shaped to match the slack-bot-token rule; joined from
parts rather than one literal so it doesn't read as a real token to
GitHub's own push-protection secret scanning).

That alone isn't enough: the demo builds MaterializeSession records
directly and calls store.materializeSessions(), bypassing
toMaterializeSessions in src/indexing/pipeline.ts, the only place the
inline secret scan runs. Wire the same scan into the demo generator so
seeded sessions carry secretFindings like real indexed sessions do.

Verified with `bun run scripts/demo.ts --no-serve --seed 42`: one
slack_token finding lands on the intended session/interaction.
@jcsalterego
jcsalterego merged commit a20016f into main Aug 19, 2026
1 check passed
@jcsalterego
jcsalterego deleted the jchen/add-credentials-scan-to-argus-demo branch August 19, 2026 20:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant