CASSANDRA-21126: Vector search support in cassandra-easy-stress

[dracarys09]

Summary of the changes

• Add VectorSearch workload for benchmarking Cassandra 5.0+ vector search (ANN) capabilities
• Support both synthetic random vectors and realistic datasets via HDF5 files (SIFT, GloVe, etc.)
• Implement recall@K calculation with ground truth comparison for measuring search quality
• Add configurable similarity functions (COSINE, EUCLIDEAN, DOT_PRODUCT) and vector dimensions

Testing

• Run ./gradlew test --tests "org.apache.cassandra.easystress.workloads.VectorSearchTest"
• Verify workload runs against a Cassandra 5.0+ cluster with random vectors
• Test with an HDF5 dataset (e.g., sift-128-euclidean.hdf5) and verify recall metrics are logged
• Confirm ktlint passes: ./gradlew ktlintCheck

[michaeljmarshall]

This classifies recall as 0 if the relevantTruth.size is 0. I wonder if it makes more sense to exclude this query's recall result from the recall averages and instead track/report the number of queries that didn't have any of the ground truth results inserted into the table?

[dracarys09]

That's a good point. Thanks for the review. I've updated the code to now track such queries separately to keep the recall numbers clean.

[maedhroz]

If you run into memory usage problems with this, you could always try a bit set...something like...

// Sized to trainVectors.size, one bit per index
// Using AtomicIntegerArray where each Int holds 32 bits
private lateinit var insertedBits: AtomicIntegerArray

// Initialize after HDF5 load, when trainVectors.size is known:
insertedBits = AtomicIntegerArray((trainVectors.size + 31) / 32)

// Set a bit (in getNextMutation):
fun markInserted(idx: Int) {
    val word = idx / 32
    val bit = 1 shl (idx % 32)
    while (true) {
        val current = insertedBits.get(word)
        if (current and bit != 0) break  // already set
        if (insertedBits.compareAndSet(word, current, current or bit)) break
    }
}

// Check membership (in onSuccess):
fun isInserted(idx: Int): Boolean {
    val word = idx / 32
    val bit = 1 shl (idx % 32)
    return insertedBits.get(word) and bit != 0
}

[maedhroz]

I'm a little worried that this isn't safe if prepare() is called from multiple threads. Might be easy to fix with something like...

@Volatile var calculateRecall = false
...
@Volatile private var hdf5Loaded = false
@Volatile private var hasGroundTruth = false
...
@Synchronized
private fun loadHdf5Data() {
    if (hdf5Loaded) return
    // ... rest of the method unchanged
}

[maedhroz]

Dropped a couple of comments, but overall, LGTM