- Executive Summary
- Background
- Architecture Overview
- Detailed Design
- ACID Compliance
- GUC Parameters
- Integration Points
- Performance Considerations
- Security Objectives
This document describes the svs PostgreSQL extension, which adds a Vamana index access method backed by Intel's Scalable Vector Search (SVS) library. The extension is a standalone PostgreSQL extension that depends on pgvector (vector extension) for its vector types and operator classes; it does not bundle pgvector source.
- Repository: https://github.com/intel/ScalableVectorSearch
- License: Mixed — see below
- Key Features:
- Vamana graph-based index (similar to HNSW)
- Locally-adaptive Vector Quantization (LVQ) compression
- LeanVec two-level quantization compression
- Optimized for Intel Xeon processors (2nd gen and newer)
- Support for float32, float16
- Distance metrics: Euclidean, Inner Product, Cosine similarity
- Billion-scale performance
Licensing details:
| Component | License |
|---|---|
| Core Vamana index (open-source SVS) | Apache 2.0 |
| LVQ and LeanVec compression | Intel Simplified Software License (binary-only, proprietary) |
LVQ and LeanVec are not open-source. They are distributed in binary form only and run only on Intel CPUs. The Intel Simplified Software License prohibits reverse engineering, decompilation, modification, or disassembly. See ScalableVectorSearch/LICENSE and docs/non-open-source-notice.rst in the SVS repository for the full terms.
- Graph-based approximate nearest neighbor search
- Similar to HNSW but with a single-layer graph and different construction strategy
- Uses greedy search with pruning strategies
- Research paper: DiskANN: Fast Accurate Billion-point Nearest Neighbor Search on a Single Node (Microsoft Research); GitHub
- HNSW: Graph-based, supports parallel build, in-memory + on-disk phases
- IVFFlat: Clustering-based, k-means partitioning, probe-based search
┌─────────────────────────────────────────────────────────────┐
│ PostgreSQL Core │
├─────────────────────────────────────────────────────────────┤
│ pgvector Extension (vector/halfvec/sparsevec types, │
│ HNSW index, IVFFlat index) [separate extension] │
├─────────────────────────────────────────────────────────────┤
│ svs Extension (requires = 'vector') │
├─────────────────────────────────────────────────────────────┤
│ Vamana Index │
│ vamana.c / vamanabuild.c / vamanainsert.c │
│ vamanascan.c / vamanavacuum.c / vamanautils.c │
│ vamanaworker.c / vamana_undo.c / vamanalauncher.c │
│ vamana_databases.c / vamana_teardown.c / vamana_warmup.c │
│ vamana_subxact_guard.c / vamana_subxid_pending_array.c │
└─────────────────────────────────────────────────────────────┘
│
▼
┌──────────────────────────────┐
│ Intel SVS C API Wrapper │
│ svs_wrapper.c/h │
├──────────────────────────────┤
│ - Index Build │
│ - Index Search (batch) │
│ - Dynamic insert/delete │
│ - Compression (LVQ/LeanVec) │
└──────────────────────────────┘
│
▼
┌──────────────────────────────┐
│ Intel SVS Shared Library │
│ libsvs_c_api.so │
└──────────────────────────────┘
Core Implementation Areas:
The extension entry point initializes the access method and registers GUCs, the object-access hook, and a background worker launcher at server start. The launcher supervises one background worker per enabled database; see BGW_LAUNCHER.md for the full launcher and per-database worker model. Throughout this document, "the BGW" or "the worker" refers to the per-database worker serving the database an operation runs in. The index access method handler implements the PostgreSQL AM interface and dispatches to the appropriate subsystem for each operation.
The build subsystem performs a full heap scan, buffers all vectors in memory, calls the SVS dynamic build API to construct the mutable graph, serializes it to disk, creates the replication slot, and submits a synchronous LOAD request to the BGW before returning. The insert subsystem routes each incoming vector through the BGW via IPC, where it is added incrementally to the live graph and recorded in the per-transaction undo log. The scan subsystem executes ANN queries by submitting search requests to the BGW and mapping returned SVS indices back to heap TIDs via the TID mapping. The vacuum subsystem collects dead TIDs from the heap callback and submits batched delete, consolidate, and compact requests to the BGW.
The BGW subsystem owns the in-memory SVS index state for all indexes in its database, processes IPC slots from backends (search, insert, delete, maintenance), manages the replication slot, and performs periodic checkpoints. Indexes are loaded on demand (on first access), not eagerly at worker startup. The undo log subsystem records external IDs inserted within each transaction and submits compensating BGW deletes on transaction abort. The SVS wrapper layer provides a thin, PostgreSQL-friendly interface over the Intel SVS C API for all graph operations.
Test Coverage:
Regression tests cover basic CRUD operations, all distance metrics, all compression types, GUC boundary values, incremental INSERT and DELETE + VACUUM cycles, and REINDEX. TAP tests cover index persistence across restarts, crash recovery, background worker lifecycle, batch search correctness, concurrent query isolation, and LWLock slot leak detection.
SVS provides a high-level batch build API with internal parallelism management.
Implementation characteristics:
- SVS manages parallelism internally via threads
- No manual shared memory management required for build
- PostgreSQL passes thread count via
max_parallel_maintenance_workersfor build operations andsvs.search_num_threadsfor search operations - Dynamic index: Uses
svs_index_build_dynamicto produce a mutable index; INSERT callsSVSAddPointsfor incremental updates, DELETE + VACUUM callsSVSDeletePointswith consolidation and compaction - Must buffer vectors: All vectors loaded into memory before build
After the SVS batch build completes, the index is serialized to a directory on the filesystem using VamanaGetIndexSavePath(). The path is keyed by the index's relation OID. The metapage stores hasSavedIndex, indexDataBlkno, and indexDataSize to track serialization state. Immediately after serialization, vamanabuild creates a logical replication slot (vamana_<db>_<idx>) with confirmed_flush_lsn set to the current WAL flush position. This anchors the slot LSN to the exact point where the on-disk state is complete, ensuring crash recovery replays only changes committed after that point.
Indexes are loaded on demand: on the first access to an index, LoadIndexFromPages() reads the saved index from disk. If no saved copy exists, VamanaRebuildFromTable() performs an in-memory rebuild from the heap. (The worker no longer eagerly loads every index at startup; see BGW_LAUNCHER.md.) Loading also restores the TID map sidecar file (tidmap.bin, one ItemPointerData per allocated slot, written atomically via temp-file + rename) and the metapage's dynamic-index fields (tidMappingCapacity, nextExternalId, numDeleted) into the in-memory cache. hasSavedIndex and the dynamic fields are flipped to their final values together, in a single WAL-logged transaction, only after the on-disk files are durable.
An object-access hook (VamanaInstallObjectAccessHook) cleans up the on-disk save directory when an index or its parent table is dropped.
Inserts: vamanainsert calls VamanaWorkerWaitUntilAvailable before submitting, then routes the insert through the background worker (slot kind VAMANA_SLOTKIND_INSERT). The worker calls SVSAddPoints to add the new vector incrementally to the live in-memory index. External IDs are allocated from a counter held in the BGW's in-memory cache (cache->nextExternalId); the latest value is written back to the metapage by VamanaWriteMetaPageDynamic so the counter survives restart. The inserting backend records the new external ID in the per-transaction undo log (VamanaUndoAppend); on transaction ABORT, the registered XactCallback submits a BGW DELETE for each logged ID to roll back the in-memory graph state. All indexes are born dynamic; there is no static-only fallback path for inserts.
The worker loads indexes on demand, not eagerly at startup, so the first insert against an index it hasn't cached yet (e.g. immediately after a server restart) triggers a synchronous load via VamanaWorkerGetOrLoadIndex: try LoadIndexFromPages, falling back to VamanaRebuildFromTable if no saved copy exists. Subsequent inserts find the index already cached and take the fast path.
Vacuum: vamanabulkdelete iterates the TID mapping, calls PostgreSQL's dead-tuple callback for each live entry, and batches dead IDs to the BGW via VamanaWorkerSubmitDelete. vamanavacuumcleanup submits CONSOLIDATE and COMPACT requests to the BGW via VamanaWorkerSubmitMaintenance. The BGW calls SVSDeletePoints, SVSConsolidate, and SVSCompact respectively, and updates the metapage counters atomically after each operation.
Known Limitation: PREPARE TRANSACTION (two-phase commit / 2PC) is not supported on any transaction that has inserted into a Vamana index. The undo log implementation raises ERRCODE_FEATURE_NOT_SUPPORTED at prepare time. This is a fundamental constraint of the in-memory undo log design: prepared state cannot be persisted to disk and resumed by a later COMMIT PREPARED. Users with max_prepared_transactions > 0 or distributed 2PC coordinators (e.g., Citus, XA drivers) must avoid issuing PREPARE TRANSACTION on transactions that touch Vamana indexes.
Vamana index state lives in background worker processes, one per enabled database, supervised by a launcher registered at server start. The launcher, per-database worker lifecycle, enablement catalog, crash-restart/backoff policy, per-database shared-memory slot layout, demand-driven loading, and the drain-and-stop shutdown path are documented separately in BGW_LAUNCHER.md. This section covers only the worker's data-plane behavior — how it serves backend requests once running.
Request serving:
- The per-database shared-memory region contains a per-backend slot array plus variable-length areas for query vectors, result TIDs, and distances
- Each slot has a kind (
VAMANA_SLOTKIND_SEARCH,_INSERT,_DELETE,_MAINTENANCE,_LOAD); backends set the kind and data, set statusPENDING, and wait on a shared latch - The worker drains all pending slots each cycle: SEARCH slots are batched and dispatched to
SVSSearch; write slots (INSERT,DELETE,MAINTENANCE) are dispatched one at a time viaVamanaWorkerProcessWriteSlot - If the worker is unavailable or startup times out (controlled by
svs.worker_startup_timeout_ms), the backend throws an error rather than silently falling back - Per-index LWLocks (
VamanaIndexLockSlot, up toVAMANA_MAX_INDEXES = 64live indexes) serialize concurrent write operations within the worker. Writers acquire the lockLW_EXCLUSIVE; searches acquire itLW_SHARED, so concurrent searches proceed together and only block against an in-progress writer on the same index - On restart, the worker loads each index's last checkpoint from disk on first access, opens the persisted replication slot, and replays all committed changes from the slot's
restart_lsnforward before serving the first request against that index - Reload signaling: for edge cases such as TRUNCATE or REINDEX, backends write to
reloadRequests[]to signal index invalidation; the worker reloads the affected index on its next cycle. Normal write-path state synchronization uses replication slot replay, not reload signals
Checkpoint and durability: The BGW periodically checkpoints the in-memory SVS graph to disk using an atomic 5-phase sequence: write to temp files, fsync, rename, fsync directory, advance the replication slot. The slot advance is the final step — it marks the on-disk state as durable and bounds the WAL that PostgreSQL must retain. Between checkpoints, the BGW continuously applies committed changes from the slot to the in-memory graph. Checkpoint frequency is governed by the debounce GUCs (svs.checkpoint_min_ops, svs.checkpoint_debounce_window, svs.checkpoint_max_interval). A deliberate stop (database pause or explicit restart) drains and checkpoints every cached index before exiting — see BGW_LAUNCHER.md. See Section 5 for the full recovery and idempotency design.
Transaction safety: Inserts are logged to a per-transaction undo log. On ABORT, an XactCallback/SubXactCallback submits BGW DELETE operations for each logged insert, rolling back the in-memory graph state. On COMMIT, the undo log is discarded.
Workers require the extension to be loaded via shared_preload_libraries = 'svs' (which registers the launcher; requires a server restart) and the database to be enrolled in the vamana_databases catalog. See BGW_LAUNCHER.md.
Vamana indexes support vector (float32) and halfvec (float16) types. sparsevec and bit are not supported.
| Operator Class | Type | Distance Operator | Metric |
|---|---|---|---|
vector_l2_ops |
vector |
<-> |
L2 (Euclidean) |
vector_ip_ops |
vector |
<#> |
Inner product (negated) |
vector_cosine_ops |
vector |
<=> |
Cosine |
halfvec_l2_ops |
halfvec |
<-> |
L2 (Euclidean) |
halfvec_ip_ops |
halfvec |
<#> |
Inner product (negated) |
halfvec_cosine_ops |
halfvec |
<=> |
Cosine |
Two progress phases are reported via pg_stat_progress_create_index:
PROGRESS_CREATEIDX_SUBPHASE_INITIALIZE— initializing build statePROGRESS_VAMANA_PHASE_LOAD— scanning heap and accumulating vectors
Dynamic behavior is split across the same files listed in §3.1: vamanabuild.c builds via SVSBuildDynamicIndex; vamanainsert.c's vamanainsert routes through VamanaWorkerSubmitInsert; vamanavacuum.c's vamanabulkdelete and vamanavacuumcleanup route through VamanaWorkerSubmitDelete and VamanaWorkerSubmitMaintenance; vamanautils.c holds VamanaRebuildFromTable and VamanaWriteMetaPageDynamic; and vamanascan.c's LoadIndexFromPages calls SVSLoadDynamicIndex, with SVSSearch skipping any slot marked InvalidItemPointer (soft-delete).
Metapage fields. Beyond the static-index fields, the metapage (VamanaMetaPageData) carries three fields specific to dynamic indexes: nextExternalId, numDeleted, and tidMappingCapacity.
nextExternalId— SVS external IDs must be unique for the life of the index. The metapage is the single source of truth across backends; the BGW reads it while holding the exclusive per-index LWLock, passes the value toSVSAddPoints, and writesvalue+1back.numDeleted— drives theSVSCompactdecision threshold (svs.compact_threshold_pct) and reports deletion pressure.tidMappingCapacity— after deletes, the TID mapping array has holes.numVectorstracks live entries;tidMappingCapacitytracks the allocated length, which is what the sidecartidmap.binfile serializes. Search bounds-checks against the capacity, then filters outInvalidItemPointerslots.
In-memory cache. The BGW's per-process cache (VamanaIndexCache) mirrors the metapage and adds runtime-only fields (svsIndex, isValid, memCtx, tidMapping, needsSave). Its nextExternalId is a mirror of the metapage, not authoritative — the BGW re-reads the metapage while holding the exclusive per-index LWLock before assigning an ID, so assignment stays monotonic even if the cached value is stale.
SVSSearch is the query-time hot path; two changes affect reads under dynamic indexes:
- Bounds check widened to
tidMappingCapacity. Static indexes always hadnumVectors == tidMappingCapacity. Dynamic indexes can havetidMappingCapacity > numVectorsbecause deleted slots stay allocated (as holes) until compaction, so SVS may return indices up to the full capacity. The search loop insvs_wrapper.cbounds-checks againstcachedIndex->tidMappingCapacityrather thannumVectors. - Soft-delete filtering. The search loop skips any slot whose TID is
InvalidItemPointer. This is the read-side half of the bulk-delete protocol (§5.2): between VACUUM's write-lock release and the nextSVSConsolidate, SVS may still surface deleted vectors as candidates, and this filter ensures they never reach the client. Because the filter can drop results, the loop tracks a separate "surviving results" counter rather than reusing the raw SVS result index — the returned result count may be less than the requestedk, matching the existing pgvector contract for sparse result sets.
The Vamana index stores its graph structure in a background worker (BGW) process using the Intel SVS library, outside PostgreSQL's normal buffer manager. This out-of-process design enables high-performance ANN search but requires explicit machinery to maintain the ACID guarantees that PostgreSQL's heap and WAL provide automatically for ordinary indexes.
| Property | Mechanism |
|---|---|
| Atomicity | BGW undo log for in-flight transactions; WAL slot filtering as safety net |
| Consistency | VACUUM as authoritative dead-entry scanner; idempotent replay prevents divergence |
| Isolation | Delegated to PostgreSQL's MVCC layer — dead or aborted heap TIDs are filtered by HeapTupleSatisfiesVisibility before results are returned, so stale graph entries never surface to queries |
| Durability | Logical replication slot + atomic checkpoint sequence; crash recovery replays committed changes from the slot |
Design Principle: Synchronous operations for visibility. WAL for durability and recovery.
| Concern | Mechanism |
|---|---|
| INSERT visibility | Backend submits to BGW via IPC; BGW calls SVSAddPoints; backend blocks until ACK |
| DELETE visibility | VACUUM submits dead TIDs to BGW via IPC; BGW calls SVSDeletePoints |
| Atomicity (abort) | BGW undo log: on abort, SVSDeletePoints for all externalIds added in that transaction |
| Durability (crash) | Replication slot: on restart, replay committed ops from last checkpoint LSN |
| Consistency (standby) | Replication slot: standby BGW replays from slot incrementally (no IPC available) |
PRIMARY STANDBY
─────── ───────
Backend BGW BGW
─────── ─── ───
INSERT ──IPC──► SVSAddPoints (immediate) (no IPC: read-only)
COMMIT ──WAL──► [WAL record durable] WAL replay ──► SVSAddPoints
│ ▲
▼ │
Checkpoint: Replication slot:
save graph + advance slot read committed changes
│ from slot position
▼
Crash restart:
load last checkpoint
replay from slot LSN
Backend BGW
─────── ───
BEGIN
INSERT INTO t (val) ...
→ heap_insert (WAL record written)
→ vamanainsert():
submit (vector, heapTID) to BGW via IPC
block until ACK ──► SVSAddPoints(externalId, vector)
tidMapping[externalId] = heapTID
record (xid, externalId) in undo log
◄── ACK ──────────────────────────
→ return to executor
COMMIT
→ VamanaXactCallback(COMMIT):
clear undo entries for this xid
Backend BGW
─────── ───
BEGIN
INSERT INTO t (val) ...
→ vamanainsert():
submit to BGW via IPC
block until ACK ──► SVSAddPoints(externalId, vector)
record (xid, externalId) in undo log
◄── ACK ──────────────────────────
ROLLBACK
→ VamanaXactCallback(ABORT):
submit undo entries to BGW ──► SVSDeletePoints(externalId)
invalidate tidMapping[externalId]
VACUUM BGW
────── ───
vamanabulkdelete():
walk index, find dead TIDs
submit dead TIDs to BGW via IPC
block until ACK ──► SVSDeletePoints(externalIds...)
invalidate tidMapping entries
◄── ACK ──────────────────────────
vamanavacuumcleanup():
submit CONSOLIDATE/COMPACT ──► SVSConsolidate() / SVSCompact()
vamanabulkdelete marks each dead TID's slot InvalidItemPointer itself, before submitting the batched delete to the BGW — not after the BGW's SVSDeletePoints call succeeds. If the BGW delete then fails, the heap already reflects the deletion, so the next rebuild from the heap is still correct; leaving the TID valid in the interim would risk a concurrent SELECT returning a dead row that raced the delete path.
BGW starts after crash:
1. Load last checkpoint from disk (SVS graph + tidMapping)
2. Open replication slot (persisted by PostgreSQL across crashes)
3. Slot's restart_lsn = position of last checkpoint
4. Replay all committed changes from restart_lsn to current WAL position:
- INSERT records → SVSAddPoints (idempotency check: skip if TID exists)
- DELETE records → SVSDeletePoints (idempotency check: skip if TID absent)
5. Resume normal IPC operation
The reorder buffer guarantees only committed transactions are delivered. Aborted transactions are automatically filtered — even if the BGW was down when the abort happened, the aborted insert never appears in the slot replay.
On a standby, there is no DML and no IPC. The BGW runs in replay-only mode:
Standby BGW:
1. Create/open replication slot at replay LSN
2. Every poll interval (~200ms):
replay committed changes from slot
SVSAddPoints / SVSDeletePoints
3. Staleness bounded by poll interval
This is identical to the primary's crash-recovery catch-up path, gated by RecoveryInProgress().
On abort, VamanaXactCallback(ABORT) submits undo entries to BGW. BGW calls SVSDeletePoints for each externalId inserted by the aborting transaction. This provides immediate cleanup when BGW is available.
If BGW is down at abort time, the undo is deferred. On BGW restart, the slot replay only delivers committed transactions (reorder buffer filtering). The aborted INSERT never appears in the replay stream. The stale externalId in the SVS graph points to a heap TID whose xmin is aborted — MVCC visibility filtering prevents it from appearing in query results. The next VACUUM cycle detects the dead TID and submits the delete via IPC.
This two-layer approach provides immediate undo when possible and guaranteed eventual correctness always.
Case 1: 3 inserts, crash before rollback (no checkpoint between)
1. BEGIN
2. INSERT v1, v2, v3 → BGW applies SVSAddPoints for each (undo log has 3 entries)
3. CRASH (before ROLLBACK or COMMIT)
Recovery:
- BGW loads last checkpoint (does NOT contain v1, v2, v3)
- Replays slot from checkpoint LSN forward
- Reorder buffer never delivers the uncommitted transaction
- v1, v2, v3 are simply not in the recovered graph
- Result: correct, no cleanup needed
Case 2: 3 inserts, checkpoint occurs, then crash before rollback
1. BEGIN
2. INSERT v1, v2, v3 → BGW applies SVSAddPoints (undo log has 3 entries)
3. Checkpoint fires → graph saved to disk WITH v1, v2, v3; slot advanced
4. CRASH (before ROLLBACK or COMMIT)
Recovery:
- BGW loads checkpoint (CONTAINS v1, v2, v3)
- Replays slot from checkpoint LSN forward
- Reorder buffer never delivers the uncommitted transaction
- No replay command arrives to add or remove v1, v2, v3
- v1, v2, v3 remain in the graph BUT their heap TIDs have aborted xmin
- MVCC filtering in vamanascan hides them from all query results
- Next VACUUM discovers the dead TIDs and submits SVSDeletePoints via IPC
- Result: invisible to users immediately, cleaned from graph at next VACUUM
VACUUM is the authoritative delete-discovery mechanism, scanning the index for dead TIDs and submitting them to the BGW:
vamanabulkdelete():
for each TID in index:
if callback(TID) says "dead":
submit SVSDeletePoints to BGW via IPC
return stats
vamanavacuumcleanup():
if numDeleted > threshold:
submit CONSOLIDATE to BGW
submit COMPACT to BGW (if threshold exceeded)The replication slot provides crash-recovery protection so deletes applied by the BGW are not lost on crash — they'll be re-discovered by the next VACUUM anyway, but the slot makes recovery faster.
The BGW keeps the SVS graph in its private process memory. After CREATE INDEX serializes the graph to disk, the BGW must load it before serving queries. The naive approach (signal BGW to reload asynchronously) introduces a race:
- CREATE INDEX holds AccessExclusiveLock on the index relation
- BGW receives reload signal, tries
ConditionalLockRelationOid, fails (lock held) - CREATE INDEX commits, lock released
- First INSERT arrives, BGW has not loaded the index yet
- INSERT fails with "index not loaded for write"
The backend absorbs the load latency at build time by submitting a LOAD request to the BGW before returning from vamanabuild. The BGW loads from the save directory (no index_open, no lock conflict) and confirms via the IPC slot. The backend blocks until DONE.
The replication slot is created by the backend (not the BGW) immediately after serialization. This eliminates the lazy-creation gap and anchors the slot LSN to the exact point where the on-disk state is complete.
Backend (vamanabuild) BGW main loop
======================== ========================
1. CreateMetaPage(index)
- writes build params to block 0
2. table_index_build_scan
- collect vectors + TIDs into memory
3. SVSBuildDynamicIndex
- build graph in backend process memory
4. VamanaSaveIndexToDisk(relid)
- SVSSaveIndex → $PGDATA/vamana_indexes/<relid>/
- VamanaSaveTidMapAtomically
- VamanaMarkIndexSaved (metapage flag)
5. ReplicationSlotCreate("vamana_<db>_<idx>")
- confirmed_flush = GetFlushRecPtr()
- PG core persists slot to pg_replslot/
6. log_newpage_range (WAL-log index pages)
7. SVSFreeIndex (free backend-local graph copy)
8. VamanaWorkerSubmitLoad(relid, heapRelid, ─── slot PENDING, latch set ───►
dims, graph_degree, alpha, distType,
vectorAttNum)
- packs metadata into IPC slot
- slotKind = VAMANA_SLOTKIND_LOAD
- blocks on slot latch
(heartbeat-aware, no fixed timeout)
9. Wakes from WaitLatch
10. ProcessRequests():
finds LOAD slot, transitions
to PROCESSING
11. VamanaWorkerProcessLoadSlot:
- Reads SVS index from
$PGDATA/vamana_indexes/<relid>/
- VamanaCacheIndex(relid, heapRelid,
dims, ..., tidMapping)
- VamanaReplicationOpen(dboid, relid)
(slot already exists from step 5)
- cache->last_replay_lsn =
GetFlushRecPtr()
12. slot status = DONE
◄── latch set on backend ────────────
13. Backend wakes, sees DONE
- returns IndexBuildResult
═══ PG core commits CREATE INDEX transaction ═══
═══ AccessExclusiveLock released ════════════════
BGW cache is warm.
First INSERT/SEARCH served immediately.
The BGW loads from the filesystem save directory ($PGDATA/vamana_indexes/<relid>/), not from relation pages. The relid and build parameters are passed through the IPC slot. No catalog access or relation lock required. The backend's AccessExclusiveLock protects the BGW's load: no other process can interfere with the save directory while the backend holds the lock.
| Crash point | State on recovery |
|---|---|
| During steps 1–4 (before slot creation) | Transaction uncommitted. PG rolls back catalog entry. Orphaned save directory cleaned by object-access hook or next startup. |
| During steps 5–7 (after slot, before IPC) | Slot exists, save directory complete, WAL-logged pages present. The worker loads the index on first access (demand-driven). No data loss. |
| During steps 8–12 (IPC in flight) | Same as above: slot + save directory complete. Backend's transaction is uncommitted. PG rolls back. The worker loads cleanly from disk on first access. |
| After step 13, before PG COMMIT | PG rolls back catalog entry. Orphaned save directory + replication slot persist. Cleaned on next DROP attempt or by startup reconciliation. |
CREATE INDEX gates on the state of the calling database's worker slot: a missing slot (database not enrolled) is a hard error, a reserved slot with no live worker is a bounded wait, and a live worker proceeds. Once the index is built and its save directory is durable, the worker loads it on first access. See the reservation handshake and three-state gate in BGW_LAUNCHER.md. The first INSERT blocks on VamanaWorkerWaitUntilAvailable until the worker has loaded the index and marked it available, guaranteeing the cache is warm before any write is applied.
A checkpoint makes the in-memory SVS graph durable to disk and marks that point as "fully processed" in the replication slot. Between checkpoints, the BGW continuously applies committed changes from the slot to the in-memory graph.
Three states to keep consistent:
- In-memory SVS graph (volatile, lost on crash)
- On-disk SVS graph (durable, updated at checkpoint)
- Slot LSN position (durable, marks "what's on disk")
The design follows the Redis AND-logic pattern:
Rule 1 (burst completion): ops >= min_ops AND quiet_for >= debounce_window
Rule 2 (safety timeout): ops >= min_ops AND time >= max_interval
should_checkpoint = Rule 1 OR Rule 2
min_ops acts as a filter that prevents expensive checkpoints for trivial changes. max_interval is a safety net for constant-write workloads (never-quiet) that would otherwise cause unbounded WAL accumulation.
Clean shutdown checkpoint: When BGW receives SIGTERM, it performs a final checkpoint before exiting, advancing the slot to current WAL position. Next startup has zero replay cost.
void PerformCheckpoint(VamanaCacheEntry *cache)
{
// Phase 1: Write to temporary files
VamanaSaveIndexToDisk(indexRel, cache->index, MAIN_FORKNUM, cache, graphTmp);
VamanaSaveTidMap(cache->tidMapping, tidmapTmp);
// Phase 2: Fsync temporary files (durability)
fsync_fname(graphTmp, false);
fsync_fname(tidmapTmp, false);
// Phase 3: Atomic rename (crash-safe)
durable_rename(graphTmp, cache->graphPath, LOG);
durable_rename(tidmapTmp, cache->tidmapPath, LOG);
// Phase 4: Fsync directory (make renames durable)
fsync_parent_path(cache->graphPath);
// Phase 5: Advance slot (ONLY after checkpoint fully durable)
current_lsn = GetFlushRecPtr();
VamanaSlotAdvance(cache->slot, current_lsn);
}Critical invariant: Slot LSN only advances after SVSSaveIndex() succeeds.
| Phase | Purpose | Crash result |
|---|---|---|
| 1. Write temp files | Isolate writes from live files | Crash → temp incomplete, old files intact |
| 2. Fsync temp files | Ensure temp files durable | Crash → temp complete but not renamed |
| 3. Atomic rename | Make new files visible | Crash → either old OR new files exist |
| 4. Fsync directory | Make renames durable | Crash → new files guaranteed visible |
| 5. Advance slot | Mark checkpoint complete | Crash before → replay from old LSN; after → skip replay |
Scenario 1: Crash after applying to memory, before checkpoint
t1: BGW reads slot (LSN 100-200) → 101 vectors in memory, 100 on disk, slot LSN 100
t2: CRASH
t3: Restart: Load on-disk graph → 100 vectors
t4: Replay from slot LSN 100 → 101 vectors
Result: ✅ Correct — changes reapplied from slot
Scenario 2: Crash after SVSSaveIndex, before slot advance
t1: BGW applies changes (LSN 100-200) → 101 vectors in memory, 100 on disk, slot LSN 100
t2: Checkpoint: SVSSaveIndex() → 101 vectors on disk, slot LSN still 100
t3: CRASH (before slot advance)
t4: Restart: Load on-disk graph → 101 vectors
t5: Replay from slot LSN 100 → externalId=42 replayed again → idempotent no-op
Result: ✅ Correct — idempotency handles the duplicate
Scenario 3: Successful checkpoint, then crash
t1: SVSSaveIndex() → 101 vectors on disk, slot LSN 100
t2: pg_logical_slot_advance(200) → slot LSN 200
t3: CRASH
t4: Restart: Load on-disk graph → 101 vectors
t5: Replay from slot LSN 200 forward → no changes to replay
Result: ✅ Correct — zero replay cost
| Property | Guarantee |
|---|---|
| Durability | If slot LSN = X, on-disk graph contains all changes up to X |
| No Data Loss | Crash recovery replays from slot LSN forward (bounded cost) |
| Idempotency | Safe to replay the same change multiple times |
| Atomicity | Either old checkpoint OR new checkpoint exists (never partial) |
| Ordering | Save first, advance slot second (never reverse) |
| Search Latency | Paused during checkpoint |
| Insert/Delete Latency | Paused during checkpoint (same window) |
| Clean Shutdown | Zero replay cost (slot advanced to current LSN) |
WAL is PostgreSQL's transaction log — a continuous, append-only stream of records that describes every change made to the database. Changes are written to WAL before they're written to data pages, ensuring durability and enabling crash recovery.
LSN (Log Sequence Number) is a 64-bit unsigned integer representing a byte offset (position) in the WAL stream. It tells you where to find something in WAL, not what it contains. LSNs are displayed as XX/YYYYYYYY (segment number / byte offset within segment).
The BGW does not write WAL — it reads from a logical replication slot. Backends' DML writes normal PostgreSQL WAL. The BGW polls the slot, which delivers only committed changes (reorder buffer filters aborted transactions).
Backend commits transaction:
↓
PostgreSQL writes INSERT + COMMIT records to WAL
↓
fsync() — now durable on disk
↓
BGW polls slot — receives the committed INSERT
↓
SVSAddPoints applied to in-memory graph
Logical decoding encounters transactions that span multiple WAL records. The reorder buffer buffers changes until COMMIT is seen, then delivers all changes atomically. On restart, the slot must start reading from the BEGIN of the oldest buffered transaction — not from the last delivered position — to reconstruct transaction state. This is why restart_lsn is earlier than confirmed_flush_lsn.
What we control:
- When to call
pg_logical_slot_advance()(checkpoint timing) - What LSN to pass (current WAL flush position)
- Ensuring idempotency for duplicate replay
What PostgreSQL controls:
- Calculating
restart_lsnbased on reorder buffer state - Delivering only committed, complete transactions
- Filtering aborted transactions
We never directly read or set restart_lsn — it's managed entirely by PostgreSQL's logical decoding infrastructure.
PostgreSQL retains WAL from restart_lsn forward (not confirmed_flush_lsn). Causes of large lag:
- Long-running transactions —
restart_lsnanchored at BEGIN for duration - BGW down —
confirmed_flush_lsnnot advancing, WAL accumulates - Infrequent checkpoints —
confirmed_flush_lsnadvances only at checkpoint
The svs.max_slot_wal_size GUC is the safety valve: if lag exceeds the threshold, drop slot and rebuild from heap.
| LSN Type | Managed by | When it advances | Affects WAL retention |
|---|---|---|---|
restart_lsn |
PostgreSQL | All buffered transactions delivered | Yes |
confirmed_flush_lsn |
SVS extension (at checkpoint) | pg_logical_slot_advance() call |
No |
| GUC | Type | Default | Range | Scope | Description |
|---|---|---|---|---|---|
svs.search_window_size |
int | 100 | 10–10000 | PGC_USERSET |
Search window (L) for index scans. Higher values improve recall at the cost of latency. |
svs.search_num_threads |
int | 0 | 0–1024 | PGC_SUSET |
Threads SVS uses for search. 0 = auto (nproc-1). Lower values reduce oversubscription under concurrent load. |
The launcher runs one background worker per enabled database. Which databases are enabled is controlled by the vamana_databases catalog, not a GUC; see BGW_LAUNCHER.md for the launcher, enrollment, and restart/backoff model. svs.launcher_database and svs.max_databases require a server restart. The timeout GUCs can be updated at runtime via SIGHUP.
| GUC | Type | Default | Range | Scope | Description |
|---|---|---|---|---|---|
svs.launcher_database |
string | "postgres" |
— | PGC_POSTMASTER |
Database that holds the vamana_databases enrollment catalog and where the launcher connects to read it. |
svs.max_databases |
int | 8 | 1–128 | PGC_POSTMASTER |
Maximum number of databases that can run a worker concurrently. Sizes the shared-memory slot array. |
svs.worker_restart_time |
int | 5 | -1–300 | PGC_POSTMASTER |
Seconds before the crashed launcher is restarted by the postmaster. -1 = BGW_NEVER_RESTART. Per-database worker restart is governed by svs.worker_restart_backoff. |
svs.worker_restart_backoff |
int | 1000 | 100–300000 | PGC_SIGHUP |
Base milliseconds for a crashed per-database worker's exponential restart backoff (capped at 60s, reset after a healthy dwell). See BGW_LAUNCHER.md. |
svs.worker_startup_timeout_ms |
int | 60000 | 1000–300000 | PGC_SIGHUP |
Milliseconds a backend waits for the worker to finish startup before throwing an error. Startup can be slow when many large indexes are deserialized from disk. |
svs.worker_timeout_ms |
int | 5000 | 100–60000 | PGC_SIGHUP |
Milliseconds a backend waits for the worker to respond to an IPC request (search or write) before throwing an error. |
svs.max_batch_size |
int | 0 | 0–1000 | PGC_SIGHUP |
Maximum queries per SVS batch search call. 0 = MaxBackends. |
svs.compact_threshold_pct |
int | 10 | 0–100 | PGC_USERSET |
Percent-deleted threshold that triggers SVS compact during VACUUM cleanup. 0 = compact on every VACUUM with pending deletes; 100 = disable compact (consolidate still runs). |
svs.max_slot_wal_size |
int (MB) | 10240 | 64–2147483647 | PGC_SIGHUP |
Maximum WAL retained by the replication slot. If exceeded, the slot is dropped and the index is rebuilt from the heap. |
svs.checkpoint_debounce_window |
int | 300 | — | PGC_SIGHUP |
Seconds of write inactivity required after a burst before a checkpoint is triggered (AND-logic with checkpoint_min_ops). |
svs.checkpoint_max_interval |
int | 3600 | — | PGC_SIGHUP |
Maximum seconds between checkpoints; safety net for constant-write workloads that never go quiet. |
svs.checkpoint_min_ops |
int | 10000 | — | PGC_SIGHUP |
Minimum write operations required before any checkpoint rule fires. Prevents expensive checkpoints for trivial changes. |
svs.checkpoint_operations |
int | -1 | — | PGC_SIGHUP |
Legacy op-count checkpoint trigger. -1 = off. Setting a positive value activates simple mode (overrides debounce logic). |
svs.checkpoint_interval |
int | -1 | — | PGC_SIGHUP |
Legacy time-based checkpoint trigger in seconds. -1 = off. Setting a positive value activates simple mode. |
svs.shutdown_drain_budget_ms |
int | 30000 | 0–600000 | PGC_SIGHUP |
Time budget for the worker's shutdown drain, checked between indexes. A single in-progress checkpoint is not preemptible, so the real bound is this budget plus one checkpoint's worst case. |
svs.worker_stop_timeout_ms |
int | 30000 | 0–600000 | PGC_SIGHUP |
Milliseconds the launcher waits for a restarting worker to report stopped before giving up. Does not force-kill; the restart stays pending until the worker exits naturally. |
Specified in the WITH (...) clause of CREATE INDEX.
| Parameter | Type | Default | Range | Description |
|---|---|---|---|---|
graph_degree |
int | 64 | 16–256 | Maximum graph connections per node (R parameter). |
alpha |
int | -1 | -1–200 | Pruning parameter scaled by 100 (120 = α 1.20). -1 uses SVS library default. |
build_window_size |
int | -1 | -1–1000 | Build window size (L parameter). -1 = 2 × graph_degree. |
search_window_size |
int | 100 | 10–10000 | Search window during build and initial queries. |
use_search_history |
bool | true |
— | Maintain the visited-node set during search. |
compression_type |
int | 0 | 0–2 | 0 = none, 1 = LeanVec, 2 = LVQ. |
compression_primary |
int | 8 | -8–8 | LeanVec primary quantization: 4=UINT4, -4=INT4, 8=UINT8, -8=INT8. |
compression_secondary |
int | 8 | -8–8 | LeanVec secondary quantization (same values as primary). |
leanvec_dims |
int | -1 | -1–2000 | Reduced dimensions for LeanVec. -1 = dimensions / 2. |
The extension is built with pgxs and requires the Intel SVS C API library:
SVS_INSTALL ?= svs_install_public
# Requires: $(SVS_INSTALL)/lib/libsvs_c_api.soCompiler flags for SIMD optimization:
-march=native -ftree-vectorize -fassociative-math -fno-signed-zeros -fno-trapping-math
Note:
-march=nativeis conditionally disabled on ARM Mac, PowerPC, and RISC-V64 (seeMakefile:17–33). On those platformsOPTFLAGSis empty and only the remaining flags are applied.
comment = 'SVS Vamana index access method for pgvector'
default_version = '0.1.0'
requires = 'vector'
module_pathname = '$libdir/svs'
CREATE FUNCTION vamanahandler(internal) RETURNS index_am_handler ...;
CREATE ACCESS METHOD vamana TYPE INDEX HANDLER vamanahandler;Operator classes for vector and halfvec are registered as shown in Section 4.6.
Users install the extension with:
CREATE EXTENSION svs; -- also installs the 'vector' dependency automaticallyLoading svs via shared_preload_libraries registers the launcher at server start. The launcher then spawns one worker per database enrolled in the vamana_databases catalog; loading the library alone does not start any per-database worker. See BGW_LAUNCHER.md for enrollment.
# postgresql.conf
shared_preload_libraries = 'svs'
svs.launcher_database = 'postgres' # default: 'postgres'; holds vamana_databases
svs.max_databases = 8 # default: 8; concurrent workers
svs depends on the vector extension's types at CREATE EXTENSION time, but vector does not need to be in shared_preload_libraries.
- AVX-512 SIMD: SVS automatically uses Intel hardware optimizations when compiled with
-march=native - Compression: Enable LVQ (
compression_type=2) for memory-constrained systems; LeanVec (compression_type=1) for two-level quantization - Thread count: Tune
svs.search_num_threadsto match workload;0auto-selectsnproc-1 - Background worker: The per-database worker holds the index in memory and amortizes load cost across all backends. Enroll the database and, for latency-sensitive deployments, warm indexes ahead of query traffic with
svs_warmup_index/svs_warmup_databaseso the first query does not pay the demand-load cost. See BGW_LAUNCHER.md - Incremental writes: Inserts update the graph incrementally; periodic
REINDEXrestores optimal graph quality after many mutations
Inserts are applied incrementally via SVSAddPoints and are immediately searchable. Deletes are applied during VACUUM via SVSDeletePoints with graph consolidation. For write-heavy workloads:
- Writes are serialized per-index inside the BGW via a per-index LWLock (
VamanaIndexLockSlot); high-throughput inserts may experience contention - The write lock is scoped by relation OID — concurrent writes to different indexes never serialize against each other
- Checkpoints run on the BGW main thread and block both searches and writes for their duration. This is required because
MutableVamanaIndex::save()is notconst: it callsconsolidate()thencompact()before writing, mutating shared graph state. Concurrent save + search is an unconditional data race. INSERT, DELETE, and SEARCH requests queue on their IPC slot latches and are processed once the checkpoint completes - Periodic
REINDEX CONCURRENTLYproduces a fresh, optimally-constructed graph after many incremental mutations - All writes (INSERT, DELETE, VACUUM) go through the background worker; the worker is the single owner of the live in-memory index, so all backends see a consistent view after the worker processes each slot
The TID mapping array is sized by tidMappingCapacity, not by the live vector count numVectors. After deletions, the capacity grows monotonically until SVSCompact reclaims the deleted slots. In workloads with high delete churn and infrequent compaction, the TID mapping and its on-disk sidecar can be significantly larger than the live vector count alone would suggest. Lowering svs.compact_threshold_pct reduces this overhead at the cost of more frequent compaction during VACUUM.
In addition to the graph edges and stored vectors, the SVS dynamic index maintains a small amount of per-vector bookkeeping (delete flags, insertion metadata). This overhead is small relative to the graph itself but grows linearly with tidMappingCapacity.
This section summarizes the security objectives for each extension-managed asset, derived from a formal threat model using DREAD scoring.
| Asset | Confidentiality | Integrity | Availability |
|---|---|---|---|
| Vamana metapage and TID mapping pages | — | Required (page checksums) | Required |
Serialized SVS index directory ($PGDATA/vamana_indexes/<oid>/) |
Required (contains embeddings) | Required | Required |
Extension binaries (svs.so, svs.control, SQL scripts) |
— | Required (RPATH integrity) | — |
Intel SVS shared library (libsvs_c_api.so) |
Required (proprietary) | Required | — |
Per-transaction undo log (vamana_undo.c) |
— | Required | Required (bounded growth) |
| Launcher and per-database BGW main loops (one worker per enabled database) | — | Required | Required (restart recovery, crash backoff) |
BGW index cache (VamanaIndexCache) |
Required (embeddings in memory) | Required | — |
Per-index logical replication slot (pg_replslot/vamana_*) |
— | Required | Required (WAL retention bound) |
Confidentiality. Stored embeddings (on disk in $PGDATA/vamana_indexes/ and in BGW memory) follow PostgreSQL's $PGDATA permissions posture, which SetDataDirectoryCreatePerm derives from the mode of $PGDATA itself; the umask the postmaster inherits is discarded (postmaster.c umask(PG_MODE_MASK_OWNER), then umask(pg_mode_mask) in miscinit.c). Every mode the extension controls is stated at the point of creation rather than obtained by umask reduction: the three directories it creates itself, vamana_indexes/, vamana_indexes/<dboid>/ and vamana_indexes/<dboid>/<relid>/, via MakePGDirectory (pg_dir_create_mode, 0700 or 0750 with group access), and tidmap.bin via OpenTransientFilePerm plus an fchmod on the still-empty file (pg_file_create_mode, 0600 or 0640) — see VamanaEnsureSaveDir and VamanaSaveTidMapAtomically in vamanaio.c. The pg_*_create_mode globals are used in preference to hardcoded modes so a cluster initialized with initdb -g retains group access. The remaining gap is the SVS payload, which the library creates in its entirety: svs_index_save(index, directory, error) accepts no mode, and the library creates the config/, data/ and graph/ subdirectories requesting 0777 and their files requesting 0666. Nothing the extension does narrows those requests — pg_mode_mask alone is what brings them down to 0700/0600, or 0750/0640 with group access. Like the assumption about SVS error strings below, this is a property of the library's I/O behavior and should be rechecked on each SVS version bump. The extension does not implement access control beyond the filesystem in this release. SVS error strings contain only metadata (operation names, error codes, dimensions), not vector values; this assumption is revalidated on each SVS library version bump.
Temporary file lifetime. tidmap.bin is published atomically by writing tidmap.bin.tmp and renaming it. The temp file is unlinked on every failure path in VamanaSaveTidMapAtomically, and any pre-existing temp file is unlinked before the new one is created with O_EXCL. That delete-before-create is a correctness requirement as well as cleanup: O_CREAT does not alter the mode of a file that already exists, so a stale temp file at a wider mode would keep it and the rename would publish that mode onto tidmap.bin. A temp file orphaned by a crash between the write and the rename is swept at the top of VamanaLoadTidMap, gated on st_mtime < PgStartTime so a concurrent writer's temp file is never removed between its close and its rename.
Integrity. The SVS wrapper trust boundary (svs_wrapper.c) validates output-buffer bounds before returning results to backends. Input-buffer sizing is a caller responsibility documented in svs_wrapper.h. The atomic 5-phase checkpoint sequence ensures on-disk state is never partial. The replication slot mechanism guarantees crash recovery replays only committed transactions (reorder buffer filtering).
Availability. The per-transaction undo log is bounded to prevent memory-DoS from large transactions. BGW saturation from concurrent high-search_window_size queries is bounded by svs.worker_timeout_ms (converts hangs to ERRORs). The BGW restarts automatically after crashes with bounded replay cost from the replication slot.
The following risks are accepted with documented rationale:
| Risk | Severity | Disposition |
|---|---|---|
BGW saturation via search_window_size = 10000 + connection flood |
High | Accepted as bounded: worker_timeout_ms converts hangs to ERRORs; search_window_size GUC bounded 10-10000. |
| CREATE INDEX OOM on oversized table | Medium | Backend-local failure; postmaster not impacted. A pre-flight maintenance_work_mem gate is planned for a future release. |
| BGW crash amplification (reload time proportional to index size) | Medium | Accepted as known. Reload time is linear in total serialized index size; documented in deployment guidance. |
| MKL CPU-feature environment variable forcing SIGILL on incompatible hardware | Medium | Accepted: operator environment is trusted configuration. See the User Guide for AVX-512 requirements. |