Skip to content

feat: implement cursor-aware pagination and event-level idempotency - #17

Merged
diiabblo merged 2 commits into
OBP-ORG:mainfrom
Richiey1:fix/issue-14
Jul 30, 2026
Merged

diiabblo merged 2 commits into
OBP-ORG:mainfrom
Richiey1:fix/issue-14

Conversation

@Richiey1

@Richiey1 Richiey1 commented Jun 27, 2026 •

Copy link
Copy Markdown
Contributor

Closes #14

Summary of Changes

Implemented cursor-aware pagination for the Soroban event fetcher and enforced true event-level idempotency to prevent duplicate domain writes across checkpoints.

What changed

  • Replaced simplistic startLedger polling with cursor-based pagination in rpc.getEvents calls inside the indexer engine.
  • Implemented stable cursor persistence to safely survive crashes midway through a ledger range page.
  • Tracked processed events by unique event.id returned by Soroban RPC instead of just tx_hash.
  • Replaced the TreasuryEvent database constraint from tx_hash to event_id to cleanly support multiple distinct treasury events emitted within a single transaction.

Testing / Local Verification

npm run build
npm run test:indexer
npm run lint

@diiabblo diiabblo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: Request Changes

Good direction -- cursor-based pagination and Redis-backed event idempotency are the right approaches. A few issues need fixing before merge:

Blocking

Missing migration for schema change (prisma/schema.prisma)
The TreasuryEvent model drops @unique from txHash and adds a new eventId unique field. No migration SQL is included. Add a prisma migrate dev --create-only migration that adds the event_id column with a unique index and safely removes the tx_hash unique constraint.

No tests
The issue requires tests for paginated responses, replay idempotency, crash recovery, and multiple events per transaction. The PR contains no test files.

Should address

(response as any).cursor (src/lib/indexer/engine.ts:347)
The SorobanRpc response type should include a cursor field. Use proper typing instead of any casting.

Redis Set grows unbounded
orbitpay:indexer:processed_events has no TTL. Add an expiry or periodic cleanup so this does not grow indefinitely as events accumulate.

@diiabblo

Copy link
Copy Markdown
Contributor

great job @Richiey1

@diiabblo
diiabblo merged commit 4fb4326 into OBP-ORG:main Jul 30, 2026
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement cursor-aware Soroban event pagination and event-level idempotency

2 participants