Skip to content

OUT-3896: make payment.succeeded expense sync idempotent - #63

Merged
SandipBajracharya merged 7 commits into
mainfrom
OUT-3896
Jun 24, 2026
Merged

OUT-3896: make payment.succeeded expense sync idempotent#63
SandipBajracharya merged 7 commits into
mainfrom
OUT-3896

Conversation

@SandipBajracharya

Copy link
Copy Markdown
Collaborator

Summary

Repeated payment.succeeded webhooks were creating duplicate Xero bank transactions and duplicate synced_payments rows. This makes the expense sync idempotent across the failure modes we hit in prod.

What changed

  • Partial unique index on synced_payments (portal_id, tenant_id, copilot_payment_id) WHERE copilot_payment_id IS NOT NULL — durable DB guarantee of one expense per payment. Partial so the NULL-keyed PAYMENT rows are unaffected.
  • Idempotency key threaded into createBankTransaction (copilotPaymentId) — handles the seconds-apart concurrent race within Xero's idempotency window.
  • App-level guard in createPlatformExpensePayment: selects on copilot_payment_id and returns early on a replay; the insert uses onConflictDoNothing() as the race backstop.
  • Reconcile by reference: before creating, look up an existing AUTHORISED Xero SPEND transaction by reference and reuse it. This covers the 2-hourly retry cron, which runs long after Xero's idempotency window. The invoice id moved from reference into the line-item description.

Layers of defense

  1. App SELECT guard → early return on replay
  2. Xero idempotency key → concurrent race
  3. Reconcile-by-reference (AUTHORISED only) → retry outside Xero's window
  4. Partial unique index + onConflictDoNothing → durable backstop

Deployment note (order matters)

The one-time dedupe of existing duplicate synced_payments rows must run before this migration is applied, or the unique index creation will fail on existing duplicates. (Dedupe SQL and the ops scripts are intentionally not part of this PR.)

Out of scope / follow-ups

  • Removing the duplicate BankTransactions already in Xero (manual accounting reconciliation).
  • Idempotency for other webhook events.

🤖 Generated with Claude Code

Repeated payment.succeeded webhooks no longer create duplicate Xero
bank transactions or synced_payments rows.

- Add partial unique index on (portal_id, tenant_id, copilot_payment_id)
  WHERE copilot_payment_id IS NOT NULL
- Thread an idempotency key into createBankTransaction
- Guard createPlatformExpensePayment with a select on copilot_payment_id
  and an onConflictDoNothing insert
- Reconcile by reference: look up an existing AUTHORISED expense in Xero
  before creating, so retries outside Xero's idempotency window reuse it
- Move the invoice id from reference into the line-item description

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@linear-code

linear-code Bot commented Jun 24, 2026

Copy link
Copy Markdown

OUT-3896

@vercel

vercel Bot commented Jun 24, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
xero-integration Ready Ready Preview, Comment Jun 24, 2026 11:51am

Request Review

@supabase

supabase Bot commented Jun 24, 2026

Copy link
Copy Markdown

This pull request has been ignored for the connected project pkdwtcdqcefmlgxmcwmc because there are no changes detected in supabase directory. You can change this behaviour in Project Integrations Settings ↗︎.


Preview Branches by Supabase.
Learn more about Supabase Branching ↗︎.

@SandipBajracharya SandipBajracharya changed the title feat(OUT-3896): make payment.succeeded expense sync idempotent OUT-3896: make payment.succeeded expense sync idempotent Jun 24, 2026
@greptile-apps

greptile-apps Bot commented Jun 24, 2026

Copy link
Copy Markdown

Greptile Summary

Commit 1b8e706 refactors two functions introduced earlier in this PR. getExpenseByCopilotPaymentId is rewritten from a manual .select() chain to db.query.syncedPayments.findFirst() (with syncedPayments correctly added to the schema barrel so the relational API can resolve the table). findLegacyExpenseByInvoice drops the early-return-undefined path when multiple legacy candidates match, instead returning the first candidate and logging a warning.

  • Schema barrel registration (src/db/schema/index.ts): syncedPayments is added, which is required for db.query.syncedPayments to be non-undefined at runtime — the commit correctly pairs the API migration with the registration.
  • findLegacyExpenseByInvoice behavior on duplicates: The old code returned undefined for ambiguous multi-match cases, which caused the caller to fall through to createBankTransaction and create yet another Xero duplicate. The new code adopts candidateTxns[0] (logging a warning for cleanup), which is correct because all same-invoice, same-amount AUTHORISED SPEND transactions in the legacy path are duplicates of the same expense — the partial unique index and onConflictDoNothing backstop prevent any incorrect DB row from being written regardless of which candidate is picked.

Confidence Score: 5/5

Safe to merge — the refactoring is straightforward and the behavioral change in findLegacyExpenseByInvoice moves from a path that created more duplicates to one that reuses existing transactions.

Both changes are tightly scoped. The schema barrel registration is a required companion to the findFirst migration and is correctly paired in the same commit. The multi-match fallback in findLegacyExpenseByInvoice now adopts the first candidate instead of returning undefined; the old path was actually more dangerous because it caused createBankTransaction to run and produce another Xero duplicate — adopting any one of the same-invoice, same-amount candidates is safe since they represent the same underlying expense, and the partial unique index on copilot_payment_id prevents any incorrect DB row from being written regardless of which candidate is chosen.

No files require special attention.

Important Files Changed

Filename Overview
src/features/invoice-sync/lib/SyncedPayments.service.ts Refactors getExpenseByCopilotPaymentId to use the relational query API (db.query.syncedPayments.findFirst); multi-layer idempotency guards are correctly structured inside the try block
src/lib/xero/XeroAPI.ts findLegacyExpenseByInvoice now returns candidateTxns[0] instead of undefined when multiple matches exist, avoiding a new createBankTransaction call that would produce yet another duplicate; correct behavior given that multiple same-amount matches represent the same expense
src/db/schema/index.ts Correctly registers syncedPayments in the schema barrel, required for db.query.syncedPayments to be available to the relational query API used in the refactored getExpenseByCopilotPaymentId
src/db/schema/syncedPayments.schema.ts Adds partial unique index on (portal_id, tenant_id, copilot_payment_id) WHERE copilot_payment_id IS NOT NULL as the durable DB backstop; correctly leaves NULL rows unaffected
src/db/migrations/20260623122124_uq_synced_payments_portal_tenant_copilot_payment_id.sql Migration correctly creates the partial unique index matching the schema definition; missing newline at EOF is cosmetic only

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A([payment.succeeded webhook]) --> B{DB: expense exists\nfor copilot_payment_id?}
    B -- Yes early return --> Z([return undefined])
    B -- No --> C[Fetch invoice + accounts]
    C --> D{Xero: SPEND tx\nwith Reference==payment_id?}
    D -- Found AUTHORISED --> G
    D -- Not found --> E{Xero: legacy SPEND tx\nwith Reference==xero_invoice_id\nand matching amount?}
    E -- Found AUTHORISED\none or more candidates\nwarn if more than one --> G[Reuse existing transaction]
    E -- Not found --> F[createBankTransaction\nwith idempotency key]
    F --> G
    G --> H[Insert synced_payments row\nonConflictDoNothing]
    H -- inserted.length == 0\nconcurrent race won --> I([return transaction\nskip sync log])
    H -- inserted.length > 0 --> J[Write SUCCESS sync_log]
    J --> K([return transaction])

    style B fill:#f0f4ff
    style D fill:#f0f4ff
    style E fill:#f0f4ff
    style H fill:#f0f4ff
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A([payment.succeeded webhook]) --> B{DB: expense exists\nfor copilot_payment_id?}
    B -- Yes early return --> Z([return undefined])
    B -- No --> C[Fetch invoice + accounts]
    C --> D{Xero: SPEND tx\nwith Reference==payment_id?}
    D -- Found AUTHORISED --> G
    D -- Not found --> E{Xero: legacy SPEND tx\nwith Reference==xero_invoice_id\nand matching amount?}
    E -- Found AUTHORISED\none or more candidates\nwarn if more than one --> G[Reuse existing transaction]
    E -- Not found --> F[createBankTransaction\nwith idempotency key]
    F --> G
    G --> H[Insert synced_payments row\nonConflictDoNothing]
    H -- inserted.length == 0\nconcurrent race won --> I([return transaction\nskip sync log])
    H -- inserted.length > 0 --> J[Write SUCCESS sync_log]
    J --> K([return transaction])

    style B fill:#f0f4ff
    style D fill:#f0f4ff
    style E fill:#f0f4ff
    style H fill:#f0f4ff
Loading

Reviews (4): Last reviewed commit: "docs(OUT-3896): log xeroPaymentId after ..." | Re-trigger Greptile

Comment thread src/features/invoice-sync/lib/SyncedPayments.service.ts Outdated
Comment thread src/lib/xero/XeroAPI.ts
Expenses created before the reference change carry the invoice id as
their Xero reference, not the payment id, so the payment-id reconcile
lookup misses them. A retry after deploy would create a duplicate.

- Add findLegacyExpenseByInvoice: search by the old invoice-id reference,
  match on exact integer cents, and adopt only a unique result (warn and
  skip when ambiguous) so we never pick the wrong payment's expense
- Fall back to it in createPlatformExpensePayment before creating
- Drop the unverified quote-escape on the reference filter; references
  are platform-generated ids/GUIDs with no quotes

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@SandipBajracharya

Copy link
Copy Markdown
Collaborator Author

@greptileai re-review

Comment thread src/features/invoice-sync/lib/SyncedPayments.service.ts Outdated
The existing-expense select guard ran outside the try, so a DB error
there propagated undecorated and skipped the sync_logs FAILED row that
every other path produces. Move the guard inside the try so its failures
are wrapped with failedSyncLogPayload like the rest of the flow. The
region precondition check stays outside the try by design.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@SandipBajracharya

Copy link
Copy Markdown
Collaborator Author

@greptileai

Comment thread src/features/invoice-sync/lib/SyncedPayments.service.ts Outdated
Comment thread src/lib/xero/XeroAPI.ts Outdated
… match

- getExpenseByCopilotPaymentId now uses the relational query API
  (db.query.syncedPayments.findFirst) instead of a manual select + slice
- Register syncedPayments in the schema barrel so db.query knows it
- findLegacyExpenseByInvoice adopts the first match (only one fee exists
  per invoice, so multiple matches are duplicates of the same expense);
  warn when duplicates are present so they can be cleaned up

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@priosshrsth priosshrsth left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@SandipBajracharya
SandipBajracharya merged commit 1c9aace into main Jun 24, 2026
5 checks passed
@SandipBajracharya

Copy link
Copy Markdown
Collaborator Author

@greptileai review commit 1b8e706

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants