Subscription Email Analyzer (SEA)

A Node.js tool for Google Workspace administrators to detect shadow SaaS usage. SEA scans recent Gmail messages across a Workspace domain, uses OpenAI to classify subscription activity, and aggregates the results in both CSV and web dashboard formats. It also inventories OAuth tokens granted to third-party applications via the Admin Directory API.

Features

• Email intelligence: Fetches Gmail messages (plain text & HTML) for each user, downloads attachments, extracts invoice links, and classifies subscription events via OpenAI.
• Data enrichment: Normalises vendors, merges SaaS domain metadata, and writes findings to SQLite for analytics and CSV export.
• OAuth visibility: Calls Admin SDK token APIs to report which third-party apps hold access tokens, per app and per user.
• Web dashboard: React/Vite frontend with charts, filters (invoice-only, attachment-only, search), attachment downloads, and OAuth token explorer.
• Resilience: Supports resume-from-checkpoint, incremental Gmail queries, and batch throttling for both Gmail and OpenAI.
• Reset & filtering: One-command cleanup script as well as sender/domain exclusion flags to keep noisy mail out of reports.

Architecture Overview

• CLI (src/index.js) orchestrates configuration, user enumeration, Gmail fetch, OpenAI classification, attachment capture, token inventory, and report generation.
• Services for Gmail (EmailService), OpenAI (AnalysisService), Admin tokens (TokenService), storage/reporting (StorageService, ReportService), attachments, and state persistence.
• Storage via SQLite (data/sea-findings.db) plus JSON state snapshots (state/) and attachment files (attachments/).
• Dashboard (frontend/) uses SEAs REST API (src/server/server.js) to display message insights and OAuth data.

Prerequisites

• Node.js 18+ (Node 20+ recommended).
• Yarn/npm for dependency management (npm install).
• Google Cloud project with:
  • Gmail API enabled.
  • Admin SDK Directory API enabled.
• Google Workspace admin account with permission to use domain-wide delegation.
• OpenAI API key.

Google Workspace Setup

1. Create a service account in your Google Cloud project. Enable “Enable Domain-wide Delegation” and note the client ID.
2. Generate a JSON key for the service account. Store it outside the repository (e.g., credentials/service-account.json). See docs/service-account.sample.json for format.
3. Authorize scopes in Workspace Admin Console (Security > API Controls > Domain-wide delegation):

   https://www.googleapis.com/auth/gmail.readonly
   https://www.googleapis.com/auth/admin.directory.user.readonly
   https://www.googleapis.com/auth/admin.directory.user.security
   

4. Delegate an admin user: supply the admin email to the CLI (--admin or SEA_ADMIN_EMAIL).

Installation

npm install

Configuration

1. Copy the sample environment file and edit values:

   cp .env.example .env

2. Set at minimum:
   • SEA_DOMAIN
   • SEA_SERVICE_ACCOUNT_KEY (path to your service account JSON, e.g., ./credentials/service-account.json)
   • SEA_ADMIN_EMAIL
   • OPENAI_API_KEY
3. Optional environment variables include analysis window, output paths, attachment directory, OpenAI batch delay, sender exclusions, and more (see .env.example).

CLI Usage

node src/index.js \
  --domain example.com \
  --key ./credentials/service-account.json \
  --admin admin@example.com \
  --saas-list ./data/saas-domains.json \
  --output ./reports/sea-report.csv \
  --period 90 \
  --state ./state \
  --db ./data/sea-findings.db \
  --attachments ./attachments \
  --gmail-delay 200 \
  --openai-batch-delay 500 \
  --exclude-sender billing@vendor.com \
  --exclude-sender-domain marketing.vendor.com \
  --openai-key $OPENAI_API_KEY

Key flags:

• --exclude-sender / --exclude-sender-domain: skip messages from specific senders or entire domains (comma-separated or repeatable).
• --max-users: limit the number of Workspace users processed.
• --verbose: enable debug logging.
• --attachments: where attachments are downloaded (served via /attachments/* in the API).

Flags map to environment variables prefixed with SEA_. See .env.example for the complete list.

Resetting the Workspace

To clear state snapshots, attachments, reports, and the SQLite database:

npm run reset

Re-run the CLI afterwards to regenerate fresh data.

Web Dashboard

Start the API and Vite dev server in separate terminals:

npm run api
npm run dev:web

Build static assets with npm run build:web.

Dashboard features:

• Emails tab: trend chart, vendor tables, invoice vs. general stacked bar, and a filterable Recent Messages grid with drill-down modals (attachments, invoice links, and quick link to user OAuth tokens).
• OAuth Tokens tab: top third-party apps, searchable user list, and per-user token scopes/fetch timestamps.

REST API Endpoints

• GET /api/health
• GET /api/findings/summary
• GET /api/findings/companies?limit=50
• GET /api/findings/timeseries?days=30
• GET /api/findings/messages?limit=50&company=Slack&search=invoice
• GET /api/tokens/apps?limit=100
• GET /api/tokens/users?limit=200
• GET /api/tokens/users?userEmail=user@example.com
• GET /attachments/<path> (serves downloaded files)

Security & Privacy

• Never commit real service-account credentials. Keep them in credentials/ (ignored) and reference via environment variables.
• Outputs (attachments, reports, SQLite DB) may contain sensitive data—store them securely.
• OpenAI responses include message snippets; ensure API usage complies with your privacy policies.
• The reset script deletes local artifacts but does not revoke OAuth tokens—manage revocation via Workspace Admin if required.

Development

• npm run lint – lint server/CLI code.
• npm run api – run the Express API.
• npm run dev:web – Vite dev server for the dashboard.
• npm run build:web – production build of the dashboard.
• npm run reset – clear generated artifacts.

Contributing

Issues and pull requests are welcome. Please:

1. Open an issue describing the change or bug.
2. Run npm run lint and npm run build:web before submitting.
3. Avoid committing credentials, access tokens, or other secrets.

License

Specify your chosen license before publishing (e.g., add an MIT License file).

Acknowledgements

• Google Admin SDK Directory API & Gmail API
• OpenAI Responses API
• React, Vite, Winston, better-sqlite3