feat(indexer/common): create core ledger cursor table by playmaker410 · Pull Request #41 · Fundable-Protocol/Backend

playmaker410 · 2026-06-25T18:36:59Z

Summary

Implements issue #24 — durable progress tracking for the Soroban event poller.

This PR adds the ledger_cursor table, a migration runner, and a cursor repository so the poller can resume from the last processed ledger after restarts or failures.

Changes

Migration — 0001_create_ledger_cursor.sql

Creates schema_migrations bookkeeping table (idempotent, guarded run)
Creates ledger_cursor table with BIGSERIAL surrogate PK, UNIQUE(source, domain), CHECK constraints (non-empty source/domain, last_ledger_seq >= 0), and a partial index on last_ledger_seq for monitoring and backfill queries

Cursor repository — ledger-cursor.ts

LedgerCursor interface: id, source, domain, last_ledger_seq, created_at, updated_at
SqlClient type alias: minimal tagged-template signature (no direct postgres import needed in callers or tests)
findCursor(sql, source, domain) — SELECT by (source, domain), returns LedgerCursor | null
upsertCursor(sql, source, domain, lastLedgerSeq) — INSERT … ON CONFLICT DO UPDATE with stale-write guard; falls back to findCursor when the WHERE clause suppresses the update

Migration runner — migrate.ts

Reads *.sql files from src/db/migrations/ lexicographically
Executes each file via sql.unsafe() (idempotency enforced inside SQL)
resolveMigrationsDir() walks from import.meta.url up to the package root (package.json), then returns <root>/src/db/migrations — works correctly from both source (bun run src/db/migrate.ts) and a compiled dist build (SQL assets always live under src/ and are not copied to dist/)
INDEXER_DATABASE_URL env var; db:migrate script in package.json
Root indexer:db:migrate convenience script added

Package config

@types/node devDependency added to indexer/common
tsconfig.json updated with types: ["node", "vitest/globals"]
All public API exported from index.ts

Tests — 16 tests covering findCursor and upsertCursor using an in-memory SqlClient mock (no database required): null for missing cursor, insert on first call, advances on higher seq, ignores stale/same seq (idempotent), separate cursors per source and domain, input validation (empty strings, negative seq), and timestamps on returned cursor.

Area

Backend API (src/)
Indexer common infrastructure (indexer/common/)
Streams indexer (indexer/streams/)
Distributions indexer (indexer/distributions/)
Tooling, docs, CI, or Docker

Scope

This PR addresses one scoped issue or task
Unrelated formatting, generated files, and follow-up work were left out
Backend and indexer package boundaries were respected

Verification

bun run type-check
bun run test
bun run lint
bun run indexer:type-check if indexer files changed
bun run indexer:test if indexer files changed
bun run indexer:lint if indexer files changed

Indexer Safety

Event processing changes are idempotent or do not affect event processing
Cursor changes advance only after successful processing
Event names and payload shapes were confirmed from contracts, if relevant
Backfill and replay behavior was considered, if relevant

Notes

Closes #24

Summary by CodeRabbit

New Features
- Added persistent ledger cursor tracking to reliably save and resume indexing progress per source/domain.
- Introduced a database migration workflow, including a migration runner and new commands to apply pending migrations.
Bug Fixes
- Ensured cursor progress only moves forward (prevents regression on stale or identical values) and validates cursor inputs.
Tests
- Added Vitest coverage for cursor retrieval and upsert behavior, including isolation rules and timestamp handling.

…col#24) - Add ledger_cursor SQL migration (0001_create_ledger_cursor.sql) - BIGSERIAL primary key, UNIQUE(source, domain) constraint - CHECK constraints: non-empty source/domain, seq >= 0 - Index on last_ledger_seq for monitoring queries - schema_migrations bookkeeping table for idempotent runs - Add LedgerCursor type and SqlClient type alias (ledger-cursor.ts) - findCursor(sql, source, domain): returns cursor or null - upsertCursor(sql, source, domain, lastLedgerSeq): insert-or-advance with stale-write guard (only advances if new seq > existing) - Add migration runner (migrate.ts) - Reads *.sql files from migrations/ sorted lexicographically - Executes each file idempotently via sql.unsafe() - INDEXER_DATABASE_URL env var; db:migrate npm script - Add @types/node devDependency and node types in tsconfig - Export all public API from index.ts - 16 tests covering findCursor, upsertCursor (in-memory mock) Closes Fundable-Protocol#24

drips-wave · 2026-06-25T18:37:06Z

@playmaker410 Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

coderabbitai · 2026-06-25T18:37:12Z

📝 Walkthrough

Walkthrough

Adds ledger cursor storage, migration execution wiring, and a cursor repository with lookup, upsert, tests, and exports.

Changes

Ledger cursor persistence and migration setup

Layer / File(s)	Summary
Migration tooling and schema `package.json`, `indexer/common/package.json`, `indexer/common/src/db/migrate.ts`, `indexer/common/src/db/migrations/0001_create_ledger_cursor.sql`	Adds migration scripts and dependencies, a migration runner CLI, and the initial `ledger_cursor` SQL migration.
Cursor repository and API `indexer/common/src/cursor/ledger-cursor.ts`, `indexer/common/src/cursor/ledger-cursor.test.ts`, `indexer/common/src/index.ts`, `indexer/common/tsconfig.json`	Defines the cursor row and SQL client types, implements cursor lookup and upsert behavior, adds tests, and re-exports the API with TypeScript config updates.

Estimated code review effort: 4 (Complex) | ~45 minutes

Possibly related issues

Implement cursor repository #26: Adds the cursor repository, tests, and migration needed for cursor table access, matching the cursor persistence work here.

Possibly related PRs

Fundable-Protocol/Backend#44: Shares the same migration runner and schema_migrations execution path used to apply SQL migrations.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	The PR implements the cursor schema, migration, source/domain tracking, timestamps/indexes, and adds relevant tests.
Out of Scope Changes check	✅ Passed	The migration runner, scripts, tsconfig, and exports are supportive package changes and remain in scope for the cursor work.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Title check	✅ Passed	The title is concise and accurately summarizes the main change: adding the core ledger cursor table.
Description check	✅ Passed	The description follows the template and covers summary, area, scope, verification, safety, and notes with sufficient detail.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@indexer/common/src/db/migrate.ts`:
- Around line 45-48: The exported runMigrations helper currently looks for the
migrations directory next to the compiled module, so the SQL files must be
available in the built package. Update the build/package flow for the db
migration path so the src/db/migrations/*.sql assets are copied or otherwise
shipped alongside the emitted JS used by runMigrations, or revert the API to
source-only if that packaging is not intended. Use runMigrations,
listMigrationFiles, and the migrationsDir resolution logic as the main places to
adjust.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 736d7011-eca3-4f7d-81ba-47772b98fe5e

📥 Commits

Reviewing files that changed from the base of the PR and between bcfa08d and a767cc1.

⛔ Files ignored due to path filters (1)

indexer/bun.lock is excluded by !**/*.lock

📒 Files selected for processing (8)

indexer/common/package.json
indexer/common/src/cursor/ledger-cursor.test.ts
indexer/common/src/cursor/ledger-cursor.ts
indexer/common/src/db/migrate.ts
indexer/common/src/db/migrations/0001_create_ledger_cursor.sql
indexer/common/src/index.ts
indexer/common/tsconfig.json
package.json

pragmaticAweds · 2026-06-28T04:26:19Z

Hi @playmaker410

Thank you for your awesome contribution, however after analyzing your implementation, there is a minor fixes to be done, which is related to the migration files. Kindly fix it to merge your PR asap.

Also do not forget to use fundable.finance to offramp.

pragmaticAweds · 2026-06-29T10:42:09Z

Hi @playmaker410

Checking In

playmaker410 · 2026-07-08T10:11:33Z

I just saw your comment

…rce and dist The original resolveMigrationsDir() used join(fileURLToPath(import.meta.url), '..', 'migrations') which only works when the module is loaded from src/db/migrate.ts. When loaded from dist/db/migrate.js the relative path points to dist/db/migrations/ which does not exist because tsc does not copy .sql assets. Replace the naive sibling-relative resolution with a package-root walk: starting from the current file's directory, walk upward until a directory containing package.json is found, then return <root>/src/db/migrations. This ensures the SQL assets — which always live in src/ — are found regardless of whether the runner is executed from source or a compiled build. Also fix import order (dirname, join) and collapse error string concatenation into a single template literal to satisfy biome lint rules. All checks pass: indexer:type-check, indexer:lint, indexer:test (16 tests).

coderabbitai Bot reviewed Jun 25, 2026

View reviewed changes

Comment thread indexer/common/src/db/migrate.ts

Franklivania mentioned this pull request Jun 26, 2026

Implement cursor repository #26

Open

13 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(indexer/common): create core ledger cursor table#41

feat(indexer/common): create core ledger cursor table#41
playmaker410 wants to merge 2 commits into
Fundable-Protocol:devfrom
playmaker410:feat/issue-24-ledger-cursor-table

playmaker410 commented Jun 25, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

drips-wave Bot commented Jun 25, 2026

Uh oh!

coderabbitai Bot commented Jun 25, 2026 •

edited

Loading

Walkthrough

Changes

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

pragmaticAweds commented Jun 28, 2026

Uh oh!

pragmaticAweds commented Jun 29, 2026

Uh oh!

playmaker410 commented Jul 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

playmaker410 commented Jun 25, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Area

Scope

Verification

Indexer Safety

Notes

Summary by CodeRabbit

Uh oh!

drips-wave Bot commented Jun 25, 2026

Uh oh!

coderabbitai Bot commented Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pragmaticAweds commented Jun 28, 2026

Uh oh!

pragmaticAweds commented Jun 29, 2026

Uh oh!

playmaker410 commented Jul 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

playmaker410 commented Jun 25, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jun 25, 2026 •

edited

Loading