Integrate app hardening and shared runtime updates - #119
Conversation
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_e6748377-ce1c-4200-9f81-9b79974302ae) |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4ddd3f31fb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| let claimed = false; | ||
| try { | ||
| claimed = await claimStripeWebhookEvent(c.env, event.id, event.type); | ||
| if (!claimed) return c.json({ received: true, duplicate: true }); |
There was a problem hiding this comment.
If the Worker crashes, times out, or otherwise exits after inserting the stripe_webhook_events row but before markStripeWebhookEventProcessed runs, the row remains with processed_at = NULL. On Stripe's retry for that same event id, claimStripeWebhookEvent returns false and this branch acknowledges it as a duplicate, so the subscription/customer side effects can be skipped permanently even though the event was never processed. Check processed_at (or use a stale/in-progress claim) before returning duplicate.
Useful? React with 👍 / 👎.
| row.sourceUrl, | ||
| row.filedDate, | ||
| row.filerName, | ||
| JSON.stringify(row.payload).slice(0, PAYLOAD_LIMIT), |
There was a problem hiding this comment.
When an FMP disclosure row serializes beyond PAYLOAD_LIMIT, this blind .slice() can store an invalid JSON fragment. The probe reads these observations back and calls JSON.parse(providerRow.payload), so one large provider payload can make every matching probe abort instead of updating candidates. Truncate a field inside a valid wrapper or skip storing the oversized payload rather than cutting the JSON string mid-token.
Useful? React with 👍 / 👎.
| @@ -304,10 +301,5 @@ export async function runWatcher(env: Env, now: Date = new Date()): Promise<void | |||
| } | |||
| } catch (err) { | |||
| await recordSourceError(env, 'senate', now.toISOString(), err); | |||
There was a problem hiding this comment.
When the Senate source hits a transient 403/429 or similar upstream failure, this catch now only logs the failure and leaves last_poll:senate at the previous successful poll. Once the configured interval has elapsed, every minute-long cron tick still satisfies shouldPollNow, so the Worker retries and writes ingest_log rows every minute until the source recovers instead of respecting the configured cadence. Stamp a failure/backoff timestamp here, as before, to avoid hammering the source and flooding logs.
Useful? React with 👍 / 👎.
| await Promise.all([ | ||
| fetchFmpLatest(apiKey, 'house', max, fetchImpl), | ||
| fetchFmpLatest(apiKey, 'senate', max, fetchImpl), | ||
| ]) |
There was a problem hiding this comment.
When FMP_DISCLOSURE_WATCH_ENABLED=true, the scheduled handler runs this probe every minute, making two FMP calls per tick, but these calls never increment the shared FMP daily counter used by /enrich-securities/status and the new usage telemetry. That means enabling the latency monitor can consume thousands of FMP requests per day while the app still reports only enrichment/price-refresh usage and the internal FMP_DAILY_CALL_CAP cannot protect the key. Record these calls with the same budget/telemetry path or a separate explicit counter.
Useful? React with 👍 / 👎.
1 participant
Summary
Verification
Notes
Note
High Risk
Touches production deploy gates, billing webhook idempotency, webhook outbound security, and new D1 migrations—areas where mistakes affect live schema, payments, or delivery.
Overview
This PR integrates app hardening and shared-runtime updates: deploy/ops paths, billing, delivery security, admin tooling, and the mobile client API surface.
Deploy & preview:
ship.shnow requiresADMIN_TOKENby default, checks/api/health(not/health), and runs idempotentPOST /api/admin/migrate;--deploy-onlyskips admin steps. Preview deploy auto-provisions when config is missing and validates preview health. Docs (STATUS.md, PR template) steer production schema through admin migrate, not remote Wrangler D1 migration apply.Security & reliability: New CodeQL workflow on
main. Stripe webhooks use a durablestripe_webhook_eventsledger (claim → process → mark/release on failure) with duplicate-event tests. Webhook delivery blocks unsafe production targets (private/metadata IPs), usesredirect: 'manual', and aborts slow POSTs (~10s). Subscription secrets are redacted in admin/client command persistence (hasSecretonly).Ingestion & admin: Migrations 0021 (FMP disclosure latency watch) and 0022 (Stripe events), mirrored in admin migrate. Admin adds
GET /disclosure-latencyandPOST /disclosure-latency/probe; cron runs the FMP probe. Reprocess useshasHardFailureFlags(includesbad_asset_name). House backfill respectsmaxFilingsbefore marking filings seen so capped runs can enqueue later.Client API: Public
GET /api/client/v1/trade/:id,ticker/:ticker, andmember/:memberIdOrNamewith feed-style envelopes, summaries, and bootstrap endpoint pointers.Other: Usage-monitor and FMP disclosure watch env examples; iOS workspace +
.gitignorefor Xcode user state; widespread docs/UI copy from “member” to “politician” (API field names largely unchanged).Reviewed by Cursor Bugbot for commit 4ddd3f3. Configure here.