fix(quota): close quota bypass + address P1 reliability findings - #1
Merged
Conversation
Bundles the fixes from the post-ship code review (F1-F10). The P0 bypass
was shipping live; P1s compounded operational risk.
P0: Close the quota bypass. tree-builder.ts, DELETE /api/sessions/:id,
and the post-regen wpilog GET were calling env.BLOBS.{list,get,delete}
directly. Added listBlobs / getBlob / deleteBlob wrappers in
storage/r2.ts that charge before the op, and routed all four call sites
through them. A cookie holder can no longer burn R2 ops for free.
P1: ctx.waitUntil closures in quota/http.ts now run under guardWaitUntil
which logs exceptions via console.error. Previously a throw after the
alerted_* latch flipped lost both the operator email and the audit row
for the full UTC day with no observability.
P1: Resend fetch takes a 5s AbortSignal (postToResend helper, shared by
sendMagicLink and sendOperatorAlert). Retry backoff now adds 0-30%
jitter to avoid synchronized retries during a Resend brownout.
P1: DO charge ordering. Split putBatchJsonl into encodeBatchJsonl +
putBatchJsonlBytes; SessionIngestDO now tracks chargedSeq (persisted
before the R2 PUT), so a D1 batch failure that forces a retry doesn't
double-charge the daily counter. Charge happens once per logical seq
regardless of how many retries land.
P2: First-breach race (F5) + UPSERT RETURNING (F6). chargeQuota's old
three-statement pattern (UPSERT, SELECT, UPDATE-where-0) was racy at
the alert layer — two concurrent racers could both observe
alerted_*=false and both fire sendOperatorAlert. Collapsed into one
db.batch pair: pre-snapshot SELECT + UPSERT with CASE expressions that
atomically flip each alerted_* latch when THIS charge crosses the cap.
Concurrent racers now produce exactly one firstBreach=true. Also halves
the D1 round-trip count per charge.
P2: streamSessionBatches charges Class B once (classB: keys.length)
after the list instead of per-iteration, removing N-1 serialized D1
round-trips on a wpilog regen.
P3: DO emits X-Quota-Breach-Date; worker's scheduleDoAlert uses it to
SELECT the right row even when waitUntil fires after UTC midnight.
res.clone() in /data 429 passthrough. CAP_* positive-value invariant
checked at load time. Unused exports and parameters removed.
New tests:
- daily-quota.test.ts: concurrent first-breach race produces exactly
one alert (14 tests total)
- quota-enforcement.test.ts: documented why cookie-route integration
tests for 429 are handled at unit level
124 -> 125 worker tests, typecheck clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Addresses findings from the post-ship code review on main (commit
fdd6b7f). Main is live in production; this PR doesn't block anything there but closes real gaps in the bill-defense system.buildTree,DELETE /api/sessions/:id, and the post-regen wpilog GET were callingenv.BLOBS.{list,get,delete}directly, letting a cookie holder burn R2 Class A/B ops for freealerted_*latch flipped lost both operator email and audit row for the whole UTC dayalerted_*=falseand both fire the alert emailchargeQuotaran UPSERT then SELECT; collapsed to one batched statementstreamSessionBatchescharged per object in the loopPlus P3 polish (DO emits
X-Quota-Breach-Date,res.clone()in passthrough, CAP sanity check, jitter on retry backoff, cleanups).Changes by file
storage/r2.ts— newlistBlobs/getBlob/deleteBlobwrappers that charge quota;streamSessionBatchesbatches Class B; splitputBatchJsonlintoencodeBatchJsonl+putBatchJsonlBytesso the DO can dedup charges on retryingest-do/session-ingest-do.ts— trackschargedSeqin DO storage (persisted before R2 PUT), skips the charge on retry; emitsX-Quota-Breach-Dateheader on first-breach 429quota/daily-quota.ts— UPSERT with CASE-based conditional latch flip inside a db.batch that also captures the pre-snapshot; halves D1 round-trips and closes the double-email race; adds CAP_* > 0 load-time assertionquota/http.ts—guardWaitUntilwraps every scheduled task;scheduleDoAlertnow reads the D1 row keyed on the DO-emitted breach date; removed unused exportsauth/email.ts— sharedpostToResendwith 5s AbortSignal + jittered retry; used by bothsendMagicLinkandsendOperatorAlertingest/tree-builder.ts,routes/sessions.ts(DELETE),routes/wpilog.ts(post-regen GET) — switched to the charged wrappers; wrapped callers intry/catchforQuotaExceededErrorroutes/telemetry.ts—res.clone()before passing toscheduleDoAlertTest plan
pnpm typecheck— both web + worker cleanpnpm test— 125 worker + 12 web pass (was 124 worker; added a concurrent-race unit test that asserts exactly onefirstBreach=trueacross N concurrentchargeQuotacallers)[quota/scheduleAlertAndAudit] waitUntil task threwif anything fails — previously silencedResidual findings not addressed here (advisory only)
alerted_todayflag would lose per-metric signal.