Skip to content

Bill Pro plans on active time-series#918

Draft
epompeii wants to merge 1 commit into
develfrom
active-series-billing
Draft

Bill Pro plans on active time-series#918
epompeii wants to merge 1 commit into
develfrom
active-series-billing

Conversation

@epompeii

@epompeii epompeii commented Jun 25, 2026

Copy link
Copy Markdown
Member

Add a value-based billing dimension for the Pro plan: monthly-active series,
where a series is a distinct testbed x benchmark x measure an organization
reports to.

  • series_last_seen cache: a migration adds a per-series table holding the
    greatest report.created (the server-side ingestion time) seen for each
    series, written on ingest in the same transaction as the metric inserts (so
    it cannot drift) and backfilled from existing metrics. Reads are a single
    index range scan over (organization_id, last_seen); the cascaded entity
    foreign keys are indexed as well.
  • Billing: Pro moves to a single tiered price on the pro product (tier 1 flat
    base fee plus per-series step-ups) backed by a new active_series Stripe meter
    with last aggregation. After each report, the org's cumulative period-to-date
    active-series count (all project visibilities) is posted to the meter; the
    final post of the period is the period total, and a missed post self-heals on
    the next report. The credit grants and the nightly sweep are removed.
  • Trials: Pro gets a native 30-day subscription-level trial instead of the
    base-fee waiver coupon. The trial is granted once per customer: any prior
    subscription (any status, including canceled) disqualifies it, so
    cancel-and-resubscribe cannot restart the trial.
  • Plan lifecycle: creating a plan prunes a terminally lapsed metered plan row
    (canceled, incomplete-expired, or gone in Stripe) so the organization can
    subscribe again; live or recoverable (dunning) subscriptions still conflict.
  • Active-series billing is Pro-only; legacy Team and Enterprise stay on the
    metrics meter. The ingest upsert is plan-agnostic; the billing read decides
    who is billed. Ships dark until the pro product's tiered price is configured.

Includes unit and property tests asserting the cache equals a from-scratch
distinct-series oracle, ingest atomicity, backfill correctness, and the
entity-delete cascade behavior, plus a live Stripe test covering the
once-per-customer trial.

@github-actions

github-actions Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

🤖 Claude Code Review

PR: #918
Base: devel
Head: active-series-billing
Commit: 894c2bd6ff9a7ea42395fb1c8215f104922340ae


Code Review: Bill Pro plans on active time-series

Overall this is a high-quality, well-documented change. It replaces the per-metric Pro billing (plus the daily credit-grant/reconciliation sweep) with an active-series meter fed by a series_last_seen cache written transactionally on ingest. The strong typing, exhaustive match gating, transactional cache write, and the test suite (oracle + property test + migration-backfill parity) are all in line with the project standards. Findings below, none blocking.

Correctness / robustness

  • last-aggregation billing assumes a monotonic in-period count. record_series_usage posts the cumulative period-to-date count and relies on Stripe's last aggregation, which bills the last value, not the max. The count is monotonically non-decreasing within a period except for hard entity deletion, which can lower it and thus under-bill. This is explicitly documented in series.rs and accepted, and the "final report of the period fails to post" gap is likewise documented. Worth a second confirmation that this matches the intended contract with Stripe, since it's the crux of the whole billing model. (plus/bencher_billing/src/biller.rs, series.rs module docs)

  • 404 detection is variant-shape–dependent. metered_plan_status maps only BillingError::Stripe(stripe::StripeError::Stripe(_, 404)) to "gone" (plan.rs:437). If the stripe client ever surfaces a missing subscription through a different variant (deserialization/other), it becomes a 503 and blocks re-subscription. That's the safe direction (never orphans a live sub), so it's fine, but a brief comment or test pinning the variant would harden it.

  • get_metered_plan_billing now hard-requires items.first() on the report hot path for all metered levels (not just Pro), returning NoSubscriptionItem where get_metered_plan_status previously tolerated it. Any subscription is expected to always have an item, so this is theoretical, but it slightly narrows what a metered read accepts.

Performance

  • A synchronous DB range scan on every Pro report. post_series_usage acquires a connection and runs count_active on the request path before spawning the Stripe post. Only the Stripe call is detached; the count blocks the report response. It's an indexed (organization_id, last_seen) range scan so it's cheap per-row, but it grows with the org's active-series count and runs per report. Acceptable and documented, but flagging for scale awareness.

  • One Stripe meter event per Pro report. High-frequency reporters will emit a meter event per report. With last aggregation this is inherently redundant (only the final value matters), and it's needed for the self-heal design, but confirm this stays within Stripe meter-event rate/volume limits for busy orgs.

Minor / cleanup

  • count_active_private is unused in production (tests only), documented as "kept for a private-only view." No dead_code warning since it's pub, but it is currently speculative surface area.
  • Migration backfill selects non-grouped columns (p.organization_id, p.id) under GROUP BY testbed_id, benchmark_id, measure_id. It's correct because those columns are functionally dependent on benchmark_id → project → org and SQLite's bare-column rule applies, and the backfill_equals_oracle test covers it. Just noting it relies on SQLite-specific semantics.

Standards compliance

  • #[expect(...)] used (not #[allow]), strong types threaded throughout (OrganizationId, DateTime, CustomerId), write_transaction!/no self-opened transaction in upsert_series_last_seen, exhaustive matches with rationale, OTel counters added for the new adverse/billing events, thiserror variants wrap original errors, no emdashes. openapi.json was regenerated to drop base/trial_coupon. All consistent with CLAUDE.md.
  • The removed daily sweep (credit.rs, spawn_credit_grants, all_metered/delete, ensure_period_credit) is cleanly excised, with lapse handling moved to lazy-on-read + prune-on-resubscribe, and the trial reworked from a coupon to a native subscription trial with a once-per-customer guard (customer_has_prior_subscription) — a stricter and simpler model.

Recommendation: Approve after (1) confirming the last-aggregation billing semantics with Stripe match intent, and (2) a quick sanity check on meter-event volume for high-frequency reporters. Also please run the noted verification gates before merge: cargo check --no-default-features, the plus-feature test suite for bencher_schema/bencher_billing, cargo clippy --all-features, and cargo gen-types (already reflected in openapi.json).


Model: claude-opus-4-8

@github-actions

github-actions Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

🐰 Bencher Report

Branchactive-series-billing
Testbedintel-v1
Click to view all benchmark results
BenchmarkLatencyBenchmark Result
microseconds (µs)
(Result Δ%)
Upper Boundary
microseconds (µs)
(Limit %)
Adapter::Json📈 view plot
🚷 view threshold
4.63 µs
(+0.34%)Baseline: 4.62 µs
4.86 µs
(95.34%)
Adapter::Magic (JSON)📈 view plot
🚷 view threshold
4.48 µs
(-0.28%)Baseline: 4.49 µs
4.70 µs
(95.33%)
Adapter::Magic (Rust)📈 view plot
🚷 view threshold
26.12 µs
(+2.33%)Baseline: 25.52 µs
26.55 µs
(98.36%)
Adapter::Rust📈 view plot
🚷 view threshold
3.56 µs
(+2.13%)Baseline: 3.48 µs
3.60 µs
(98.94%)
Adapter::RustBench📈 view plot
🚷 view threshold
3.55 µs
(+1.92%)Baseline: 3.48 µs
3.59 µs
(98.92%)
🐰 View full continuous benchmarking report in Bencher

@epompeii epompeii force-pushed the active-series-billing branch from 84c9153 to c1977a7 Compare June 25, 2026 13:26
@epompeii epompeii force-pushed the active-series-billing branch from c1977a7 to 98be1a1 Compare June 25, 2026 13:45
@epompeii epompeii force-pushed the active-series-billing branch from 98be1a1 to d427941 Compare June 25, 2026 14:12
@epompeii epompeii force-pushed the active-series-billing branch from d427941 to 33fd222 Compare June 25, 2026 14:53
@epompeii epompeii force-pushed the active-series-billing branch from 33fd222 to d91be06 Compare June 25, 2026 15:25
@epompeii epompeii force-pushed the active-series-billing branch from d91be06 to 9d3170c Compare June 25, 2026 15:58
@epompeii epompeii force-pushed the active-series-billing branch from 9d3170c to 4d313d0 Compare June 26, 2026 00:24
@epompeii epompeii self-assigned this Jun 26, 2026
@epompeii epompeii force-pushed the active-series-billing branch from 4d313d0 to c7253e9 Compare June 26, 2026 04:21
@epompeii epompeii force-pushed the active-series-billing branch from c7253e9 to 090ef76 Compare June 26, 2026 06:32
@epompeii epompeii force-pushed the active-series-billing branch from 090ef76 to 34f168c Compare June 26, 2026 06:48
@epompeii epompeii force-pushed the active-series-billing branch from 34f168c to eca5974 Compare June 26, 2026 06:57
@epompeii epompeii force-pushed the active-series-billing branch from eca5974 to 7a3b125 Compare June 26, 2026 07:09
@epompeii epompeii force-pushed the active-series-billing branch from 7a3b125 to d555f66 Compare June 26, 2026 07:32
@epompeii epompeii force-pushed the active-series-billing branch from d555f66 to 4c86eb2 Compare June 26, 2026 13:22
@epompeii epompeii force-pushed the active-series-billing branch from 4c86eb2 to 0683e88 Compare June 26, 2026 13:30
@epompeii epompeii force-pushed the active-series-billing branch from 0683e88 to 0efb312 Compare June 26, 2026 13:48
@epompeii epompeii force-pushed the active-series-billing branch from 0efb312 to 776d5e6 Compare June 26, 2026 13:55
@epompeii epompeii force-pushed the active-series-billing branch from 776d5e6 to 3b074f9 Compare June 27, 2026 04:07
@epompeii epompeii force-pushed the active-series-billing branch from 3b074f9 to 18ef1d0 Compare June 27, 2026 04:14
@epompeii epompeii force-pushed the active-series-billing branch from 18ef1d0 to 2e6ed84 Compare June 27, 2026 04:26
@epompeii epompeii force-pushed the active-series-billing branch from 2e6ed84 to 93f7008 Compare June 27, 2026 04:41
Use a single tiered Stripe price on the Bencher Pro product to bill both the monthly base fee (tier 1 flat fee) and the per-series step-ups on the active_series meter. This replaces the separate series product, the flat base price, and the monthly included-usage credit grants.

Pro subscriptions now carry the tiered active-series price plus bare metal, are identified by that price, and no longer bill the per-metric metrics meter. The 30-day free trial uses a native subscription-level Stripe trial (Checkout compatible) instead of a coupon. Active-series usage is posted to Stripe after each report (counted on the request connection, posted in a detached task) rather than by a nightly sweep, which is removed along with the account-credit grants.
@epompeii epompeii force-pushed the active-series-billing branch from 93f7008 to 894c2bd Compare July 5, 2026 05:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant