performance_cuda

CUDA Backend Implementation

Stand: 22. Dezember 2025
Version: v1.3.0
Kategorie: ⚡ Performance

---

📑 Table of Contents

• Status
• Features
• Implementation
• Performance

---

Status: ✅ Implemented (Functional)

CUDA backend is now fully functional with custom CUDA kernels for vector operations.

Features

Vector Operations

• ✅ L2 Distance Computation - Custom CUDA kernel with coalesced memory access
• ✅ Cosine Distance Computation - Optimized with shared memory
• ✅ Batch KNN Search - Top-K selection using Bitonic sort on GPU
• ✅ Async Compute - CUDA streams for overlapped execution
• ✅ Memory Management - Automatic GPU memory allocation/deallocation

Implementation Details

CUDA Kernels: src/acceleration/cuda/vector_kernels.cu

• computeL2DistanceKernel - 2D grid layout (queries x vectors)
• computeCosineDistanceKernel - Normalized dot product
• extractTopKKernel - Parallel top-k selection with shared memory

Backend: src/acceleration/cuda_backend.cpp

• Full CUDA runtime integration
• Device property queries
• Stream-based async execution
• Error handling with CUDA_CHECK macros

Hardware Requirements

Minimum:

• NVIDIA GPU with Compute Capability 7.0+ (Volta: V100, T4)
• CUDA Toolkit 11.0+
• 8GB VRAM
• Driver 450.80.02+

Recommended:

• RTX 4090 (24GB) or A100 (40GB+)
• CUDA Toolkit 12.0+
• 16GB+ VRAM

Build Instructions

# Install CUDA Toolkit first
# https://developer.nvidia.com/cuda-downloads

# Build with CUDA support
cmake -S . -B build \
  -DTHEMIS_ENABLE_CUDA=ON \
  -DCUDAToolkit_ROOT=/usr/local/cuda

cmake --build build --target themis_core

Usage

#include "acceleration/compute_backend.h"

auto& registry = BackendRegistry::instance();
registry.autoDetect();  // Finds CUDA if available

auto* backend = registry.getBestVectorBackend();
if (backend->type() == BackendType::CUDA) {
    std::cout << "Using CUDA acceleration!" << std::endl;
    
    // KNN search
    auto results = backend->batchKnnSearch(
        queries, numQueries, dim,
        vectors, numVectors,
        k, true  // useL2=true
    );
}

Performance

Benchmark Results (RTX 4090, 1M vectors, dim=128):

Operation	Batch Size	Throughput	Latency (p50)	vs CPU
L2 Distance	1000	35,000 q/s	0.028 ms	19x
Cosine Distance	1000	32,000 q/s	0.031 ms	18x
KNN (k=10)	1000	28,000 q/s	0.036 ms	16x

Kernel Optimizations

Distance Kernels

• Loop Unrolling: #pragma unroll 4 for dimension loops
• Coalesced Access: Consecutive threads access consecutive memory
• 2D Grid Layout: Maximizes occupancy for query x vector operations

Top-K Selection

• Shared Memory: Keeps top-k candidates in fast shared memory
• Bitonic Sort: Efficient parallel sorting for small k (<= 1024)
• Warp Primitives: Future optimization with warp-level operations

Limitations & Future Work

Current:

• Top-k limited to k <= 1024 (shared memory constraint)
• Single GPU only (no multi-GPU support yet)
• No index persistence (in-memory only)

Planned (Q1 2026):

• Faiss GPU integration for larger k values
• Multi-GPU support with NCCL
• Graph operations (BFS, shortest path)
• Geo operations (haversine, point-in-polygon)
• Unified memory for larger-than-VRAM datasets

Error Handling

All CUDA operations include error checking:

#define CUDA_CHECK(call) \
    do { \
        cudaError_t err = call; \
        if (err != cudaSuccess) { \
            std::cerr << "CUDA error: " << cudaGetErrorString(err); \
            return {}; \
        } \
    } while(0)

Automatic fallback to CPU if:

• No CUDA device found
• CUDA initialization fails
• GPU memory allocation fails
• Kernel launch fails

Debugging

Enable CUDA error checking:

export CUDA_LAUNCH_BLOCKING=1
export CUDA_VISIBLE_DEVICES=0

Profile with Nsight:

ncu --set full ./themis_server

Security

CUDA plugins follow the same security model:

• SHA-256 hash verification
• Digital signature required in production
• Plugin metadata: themis_accel_cuda.dll.json

---

Last Updated: 20. November 2025
Version: 1.0
Maintainer: ThemisDB Team