Skip to content

feat(billing): hard storage caps per plan + transactional email pipeline - #223

Merged
maakle merged 2 commits into
mainfrom
claude/storage-caps-ROx9U
May 21, 2026
Merged

feat(billing): hard storage caps per plan + transactional email pipeline#223
maakle merged 2 commits into
mainfrom
claude/storage-caps-ROx9U

Conversation

@maakle

@maakle maakle commented May 21, 2026

Copy link
Copy Markdown
Owner

Summary

Closes the downgrade-and-keep loophole: a customer can no longer ingest millions of chunks on Team, drop to Free, and keep enjoying the indexed corpus essentially rent-free. Adds a hard maxStoredArtifacts ceiling per plan. New ingestion pauses at the cap; existing chunks remain queryable.

Also adds the missing transactional email infrastructure (React Email + Resend, with idempotent send) and uses it to notify org owners the first time they hit the cap each period.

Pricing model after this PR

Lever Charges for When
Chat turns (LLM) Per token (input / output / cache) Each agent round-trip
Connector sync Per newly inserted chunk (5cr / 10cr) One-time at ingest
maxConnectors (existing) Hard cap Gate
maxStoredArtifacts (new) Hard cap Gate

Retrieval is free at the retrieval layer; it's metered indirectly via the LLM token cost of the chat turn that triggers it.

Cap values

Plan maxConnectors maxStoredArtifacts
Free 1 10,000
Starter 5 100,000
Team unlimited 1,000,000
Business unlimited 10,000,000
Enterprise unlimited unlimited

"1 artifact" = 1 row in chunks (one embedding vector). UI labels this "indexed items" with a "what counts?" tooltip — a Slack message ≈ 1 chunk, a Notion page ≈ 5–20, a long PDF ≈ 150.

Backend

  • packages/billing/src/limits.ts — new checkStorageQuota(db, orgId, deltaCount=0). Mirrors the shape of canAddConnector / checkCreditPool. Returns { allowed: true, currentCount, limit } or { allowed: false, reason: 'storage_cap', currentCount, limit, currentPlanSlug, currentPlanName, suggestedUpgradeSlug }. Index-only scan against chunks_org_idx — fast even at 10M rows. No caching needed.
  • packages/db/src/schema/billing.ts — extend features JSONB type with maxStoredArtifacts: number | null. JSONB column, so no DDL.
  • Migration 0067_storage_caps.sql — pure UPDATE seeding the 5 plans.
  • apps/worker/src/queues/sync-processor-base.ts — gate at sync start. Returns skipReason: 'storage_cap_reached', emits holo.storage.cap_reached PostHog event, fire-and-forget owner email.
  • apps/worker/src/queues/embed.ts — defensive secondary check at batch level. A fresh GitHub code sync can emit 10K+ chunks in a single embed batch; this prevents a single fat batch from blasting past the cap between sync-start gate ticks. Whole batch is skipped (no silent partial-fill).

UI

  • New StorageCard on /settings/billing alongside BalanceCard. Shows X / Y count, neutral-fill progress bar (DESIGN.md: single accent per page is owned by the existing credit balance bar), warning banner ≥90%, error banner + "View plans" CTA at 100%.
  • PlanGrid tiles surface the maxStoredArtifacts line.

New package @holo/email

The repo had Resend wiring only for auth's OTP/invitation flow. This adds a proper general transactional pipeline.

  • sendEmail / sendIdempotent — low-level + dedup-on-key wrapper.
  • React Email for templates (@react-email/render + @react-email/components).
  • First template: StorageCapReached.tsx — light surfaces, single accent CTA, matches the visual posture of packages/auth/src/email-templates.ts. Plain-text fallback auto-extracted.
  • sendStorageCapReachedEmail wrapper hides the JSX from the worker so consumers don't need JSX configured. (Worker's tsconfig gains jsx: react-jsx only because tsc transitively parses the .tsx.)
  • email_log table for at-least-once-delivery dedup; PK on idempotency_key. Failure rolls back the log row so retry is possible.
  • Migration 0068_email_log.sql.

Auth's existing OTP/invitation templates are NOT migrated to React Email in this PR (scope discipline — they work, and breaking them breaks sign-in). New transactional emails should use the React Email path going forward.

Wiring the email

When sync-processor-base trips the storage cap, it looks up the org owner via the member table (role='owner', earliest by member.createdAt), constructs the upgrade URL from BETTER_AUTH_URL, and fires the email. Idempotency key: storage_cap_reached:<org_id>:<period_start_iso>.

So an org sitting over cap for weeks gets one email per billing period (not one per blocked 6h sync tick). The "launch backfill" happens naturally — the first sync tick after deploy for any over-cap org triggers the notification; no separate one-shot script needed.

Files

New:

  • packages/email/ (entire package)
  • packages/db/src/schema/email.ts
  • packages/db/migrations/0067_storage_caps.sql
  • packages/db/migrations/0068_email_log.sql
  • apps/web/src/app/(app)/settings/billing/_components/storage-card.tsx
  • packages/billing/test/storage-quota.test.ts

Modified:

  • packages/billing/src/limits.ts — add checkStorageQuota
  • packages/billing/src/plans.ts, packages/billing/src/index.ts
  • packages/db/src/schema/billing.ts, packages/db/src/schema/index.ts
  • apps/worker/src/queues/sync-processor-base.ts — gate + email
  • apps/worker/src/queues/embed.ts — secondary cap check
  • apps/worker/tsconfig.json — enable JSX parsing for transitively-imported .tsx
  • apps/web/src/app/(app)/settings/billing/page.tsx
  • apps/web/src/app/(app)/settings/billing/_components/plan-grid.tsx

Out of scope

  • Migrating auth's OTP/invitation templates to React Email (separate PR).
  • Per-source storage quotas (org-level cap is enough for now).
  • Auto-eviction at cap (founder chose "block new, keep existing").
  • Per-retrieval billing (retrieval is metered indirectly via the LLM token cost).

Test plan

  • pnpm install resolves React Email deps
  • pnpm -r typecheck — clean
  • pnpm exec eslint . — clean
  • pnpm --filter @holo/billing test — 27/27 pass (incl. 9 new storage-quota tests covering boundary, unlimited, missing-feature-key, CE bypass, plan-tier suggestion)
  • pnpm --filter @holo/email test — 4/4 pass (render JSX → HTML+text, idempotent dedup, transport-failure rollback)
  • pnpm --filter @holo/worker test — 124/124 pass
  • Migration meta valid
  • After merge with test DB: insert 99,999 chunks for a Starter-plan test org, trigger a sync that would add 15 → sync_runs has skipReason='storage_cap_reached', email_log has a row, owner inbox receives the email (verify with EMAIL_PROVIDER=console first, then resend)
  • Upgrade lifts the cap; next sync goes green
  • HOLO_BILLING_ENABLED=false short-circuits both gates

https://claude.ai/code/session_01Mvr1Stxz7czdDcyB5XsjHD


Generated by Claude Code

claude added 2 commits May 21, 2026 19:00
Closes the downgrade-and-keep loophole: a customer can no longer ingest
millions of chunks on Team, drop to Free, and keep the indexed corpus.
Adds a hard `maxStoredArtifacts` ceiling per plan. New ingestion is paused
at the cap; existing chunks remain queryable. The per-chunk credit debit
(PR 1) and connector cap (existing) are unchanged.

Caps (chunks = rows in the `chunks` table, one per embedding vector):

| Plan       | maxConnectors | maxStoredArtifacts |
|------------|--------------:|-------------------:|
| Free       |             1 |             10,000 |
| Starter    |             5 |            100,000 |
| Team       |     unlimited |          1,000,000 |
| Business   |     unlimited |         10,000,000 |
| Enterprise |     unlimited |          unlimited |

Backend:
- packages/billing: `checkStorageQuota(db, orgId, deltaCount=0)` mirrors
  the existing `canAddConnector` / `checkCreditPool` shape. Returns the
  current count, the plan's limit, and a suggested upgrade slug on deny.
  Uses an index-only scan on `chunks_org_idx`; no caching needed.
- packages/db/src/schema/billing.ts: extend `features` JSONB type with
  `maxStoredArtifacts: number | null` (already-JSONB so no DDL).
- Migration 0067_storage_caps.sql: pure UPDATE seeding the 5 plans.
- apps/worker sync-processor-base.ts: gate at sync start (deltaCount=0),
  return `skipReason: 'storage_cap_reached'`, emit PostHog event.
- apps/worker embed.ts: secondary defensive check at the batch level
  (deltaCount=batch.length) so a single fat sync (e.g. fresh GitHub code
  ingest emitting 10K+ chunks) can't blast past the ceiling between gate
  ticks. Whole batch is skipped to avoid silent partial-fill.

UI (/settings/billing):
- New `StorageCard` component alongside `BalanceCard`. Shows X/Y count,
  progress bar in neutral fill (DESIGN.md: single accent per page is
  owned by the credit balance bar), warning banner ≥90%, error banner
  + "View plans" CTA at 100%.
- `PlanGrid` tiles surface the maxStoredArtifacts line.

New package @holo/email + React Email + Resend:
- `sendEmail` / `sendIdempotent` exported from @holo/email.
- React Email templates as .tsx. First template:
  `StorageCapReached` (light surfaces, single accent CTA, matches the
  inline-HTML auth templates).
- `sendStorageCapReachedEmail` wrapper hides the JSX from the worker so
  consumers don't need JSX configured (worker's tsconfig gains
  `jsx: react-jsx` only to compile the template).
- Schema `email_log` table for at-least-once-delivery dedup; PK on
  `idempotency_key`. Failure rolls back the log row so retry is possible.
- Migration 0068_email_log.sql.
- Auth's existing OTP/invitation emails are NOT migrated — they still
  use the inline-HTML pattern. New transactional emails should use the
  React Email path going forward.

Wiring:
- When sync-processor-base trips the storage cap, it fire-and-forget
  emails the org owner. Idempotency key `storage_cap_reached:<org>:<period_start>`
  so an org over cap for weeks gets one email per billing period, not
  one per blocked sync tick. The launch backfill happens naturally —
  the first sync tick after deploy for an over-cap org triggers the
  notification; no separate one-shot script needed.

Verification (next steps after merge):
- `pnpm db:migrate` lands both new migrations cleanly.
- Force an org over cap → next sync produces `skipReason='storage_cap_reached'`
  in `sync_runs`, an `email_log` row appears, the owner receives the email
  (test mode: EMAIL_PROVIDER=console; prod: EMAIL_PROVIDER=resend).
- Upgrade lifts the cap; next sync goes green.
- CE bypass: HOLO_BILLING_ENABLED=false → gates short-circuit.

https://claude.ai/code/session_01Mvr1Stxz7czdDcyB5XsjHD
The /settings/billing PlanGrid already shows maxStoredArtifacts per tile,
but the public pricing page didn't. Prospective customers should see all
three commercial levers (credits / month, connectors, indexed items)
before they sign up.

- apps/web/src/components/landing/pricing-band.tsx: add `indexedItems`
  row per plan tile (10K / 100K / 1M / 10M, matching the seeded caps).

No backend change — landing-page plans are hard-coded, not DB-driven.

https://claude.ai/code/session_01Mvr1Stxz7czdDcyB5XsjHD
@maakle
maakle merged commit 1f8f105 into main May 21, 2026
5 checks passed
@maakle
maakle deleted the claude/storage-caps-ROx9U branch May 21, 2026 19:07
maakle added a commit that referenced this pull request May 21, 2026
#224)

The dashboard's PlanGrid tiles were advertising "Unlimited indexed items"
for every plan, including Free. Root cause: the tiles read
`plan.features.maxStoredArtifacts` directly from the DB JSONB, but
migration 0061 (pricing-model-v2) created new plan rows with `features`
JSONB that didn't include the storage key. Migration 0067 then merges
the key in via UPDATE — but in any environment where 0067 hasn't landed
yet (or where a future pricing migration creates fresh rows without the
key), the UI silently falls through to the "Unlimited" branch and lies.

Fix: add a `plan-defaults.ts` module with canonical caps per slug, and
a `resolveStorageCap(slug, featureValue)` helper that returns:
  1. the row value if explicitly set (numbers OR null)
  2. the slug-keyed default otherwise
  3. null (unlimited) for unknown slugs

Use the resolver in BOTH:
  - `checkStorageQuota` so enforcement stays armed even when migration
    is pending (previously the gate also fell through to "unlimited"
    when the key was missing — silent under-enforcement)
  - PlanGrid so the dashboard tiles match the landing page

What we advertise now matches what we enforce. Landing-page PricingBand
already had the right hardcoded values from PR #223; leaving it alone
to keep the diff small (it'll drift if values change, but landing copy
needs human review anyway).

Tests: 5 new for `resolveStorageCap` covering explicit values, explicit
null (intentional unlimited), undefined (fallback), unknown slugs, and
the constants map. All 32 billing tests pass.

https://claude.ai/code/session_01Mvr1Stxz7czdDcyB5XsjHD

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants