Vector Storage Analysis
The Storage module is responsible for efficient persistence of the embedding matrix generated by the Representation stage. The production dataset contains 500,000 Wikipedia embeddings, each represented as a 256-dimensional float32 vector.
The primary engineering objective was to minimize memory consumption while preserving retrieval quality and providing a storage interface compatible with downstream indexing and search modules.
Embedding Generator
│
▼
Raw float32 embeddings
(500000 × 256)
│
▼
Vector Storage
├── Float32Store (baseline)
└── SQ8Store (optimized)
│
▼
Vector Index (IVF)
│
▼
Search Engine
| Iteration | Description |
|---|---|
| Iteration 1 |
Float32Store — raw float32 matrix, exact precision, baseline for both memory and accuracy comparisons |
| Iteration 2 |
SQ8Store — custom per-dimension Scalar Quantization to uint8, 4× memory reduction |
The storage layer exposes a common interface (BaseVectorStore) allowing downstream modules to swap storage implementations without changing indexing or search code.
The baseline implementation stores vectors without compression.
dtype: float32 dimensions: 256 contiguous NumPy array zero reconstruction error highest RAM consumption
store_float32.npy
500000 × 256 × 4 bytes ≈ 488 MB
This implementation serves as the ground-truth representation for all quality comparisons.
The baseline storage represents each embedding as a 256-dimensional float32 vector. Although this representation preserves full numerical precision, it is inefficient for large-scale vector collections.
For a corpus of 500,000 Wikipedia articles, the embedding matrix alone occupies approximately 488 MB in memory. As the project scales to larger datasets, the storage footprint becomes increasingly significant and limits the amount of data that can remain resident in RAM.
Since cosine similarity depends primarily on preserving the geometric relationships between vectors rather than exact floating-point values, the storage layer was optimized using Scalar Quantization (SQ8). The objective is to reduce memory consumption while introducing only minimal reconstruction error and maintaining search accuracy.
Each embedding dimension is quantized independently.
For every dimension
min = minimum(column)
max = maximum(column)
step = (max - min) / 255
Quantization
q = round((x - min) / step)
Dequantization
x ≈ q × step + min
The implementation stores
- uint8 matrix
- per-dimension minimum
- per-dimension scaling factor
allowing approximate reconstruction during retrieval.
The Search module operates exclusively on float32 vectors.
Therefore, vectors retrieved from storage are automatically reconstructed before cosine similarity is computed.
uint8
│
▼
Dequantization
│
▼
float32
│
▼
Cosine Similarity
This design keeps storage compact while remaining fully compatible with the search pipeline.
The module implements three interchangeable storage classes.
| Class | Purpose |
|---|---|
| BaseVectorStore | Common API |
| Float32Store | Baseline implementation |
| SQ8Store | Production compressed storage |
Public interface
build()
load()
get_vectors(indices)
get_memory_footprint()
Because both storage types expose identical APIs, downstream modules (Indexing and Search) can switch implementations without code modifications.
| Metric | Float32 | SQ8 |
|---|---|---|
| Storage size | 488.28 MB | 122.07 MB |
| Compression | — | 4× smaller |
Quality was evaluated after dequantization.
| Metric | Result |
|---|---|
| Mean Squared Error (MSE) | 0.00186937 |
| Average Cosine Similarity | 0.999983 |
The low reconstruction error indicates that scalar quantization preserves the geometric structure of the embedding space sufficiently for nearest-neighbor retrieval.
The graph shows the distribution of per-element reconstruction error (original − dequantized) across all 500,000 × 256 values.
The distribution is centered around zero and symmetric — quantization introduces no systematic bias; the error is purely random rather than directional. This is expected: the round() operation in the quantization algorithm produces an unbiased error by design.
The shape is close to bell-shaped (Gaussian-like) — most errors are concentrated in a narrow range of ±0.05–0.1, which is consistent with the small step_d for most dimensions (given the "typical" distribution of embedding values).
Long but rare tails (up to ±0.3–0.4) — these correspond to dimensions with a wider value range (larger max_d − min_d), where the quantization step is coarser, and consequently the maximum possible rounding error is larger. Such cases are few (the tail frequencies are near zero on the graph), so they have almost no impact on aggregated metrics (MSE, cosine similarity).
Conclusion for the project: The unbiased nature and narrow concentration of the error directly explain why, despite the relatively large MSE spread (up to ±0.4 on individual coordinates), the final average cosine similarity of 0.999983 remains practically perfect: the errors across different dimensions are random and independent, so when summed in the dot product (cosine similarity), they statistically cancel each other out rather than accumulate.
The histogram compares the raw float32 embedding values against the raw uint8 codes stored on disk. The original distribution is narrow and centered near 0 (typical of L2-normalized embedding coordinates), while the quantized distribution spans the full 0–255 range — this is expected and not a quality signal: uint8 codes are index positions within each dimension's own [min_d, max_d] range, not embedding values themselves, so the two histograms aren't on a directly comparable scale. The meaningful accuracy check is the dequantized-vs-original error distribution (previous figure), not this raw-code comparison — this plot mainly confirms the quantizer is using the full 8-bit range rather than clustering codes in a narrow band, which would indicate wasted resolution.
| mertric | value |
|---|---|
| Mean error | 0.00000898 |
| Std error | 0.04323627 |
| Max abs error | 0.38805389 |
| Average per-dimension MSE | 0.00186938 |
| Worst dimension | 67 |
| Worst MSE | 0.04806675 |
Per-dimension MSE is low and fairly uniform (mostly under 0.005) across the 256 dimensions, confirming per-dimension scaling is doing its job — no systematic degradation concentrated in a particular region of the embedding. Dimension 67 is a clear outlier, at ~10× the typical MSE (0.048 vs a ~0.002–0.005 baseline). This is consistent with that dimension having an unusually wide max_d − min_d range in the corpus, which produces a larger step_d and therefore coarser quantization resolution for that coordinate specifically. Since it's isolated to one dimension out of 256, its contribution to the aggregate cosine similarity (0.999983) is diluted — but it's worth flagging on the slide as a known limitation of scalar (vs. vector/product) quantization: outlier dimensions get worse relative precision than well-behaved ones, because the scheme allocates the same 8 bits regardless of each dimension's actual entropy.
| vectors | time | throughput |
|---|---|---|
| 100 vectors | 0.326 ms | 306,949 vectors/sec |
| 1000 vectors | 1.936 ms | 516,564 vectors/sec |
| 10000 vectors | 20.919 ms | 478,032 vectors/sec |
get_vectors() (dequantization) throughput stabilizes around ~480,000–520,000 vectors/sec once batch size is large enough to amortize call overhead (jump from 100→1000 vectors). At the batch size actually used in production — Person 5's IVF candidate pool of ~15,632 vectors per query — this throughput implies a dequantization cost on the order of ~30 ms, well within the sub-millisecond-to-low-single-digit-millisecond latency budget of the two-stage ANN search. The 100-vector case is noticeably slower per-vector (306K/sec vs ~480–520K/sec), indicating fixed NumPy call overhead dominates at very small batch sizes — not a concern here since production batches are two orders of magnitude larger.
| Metric | Float32 | SQ8 | Change |
|---|---|---|---|
| Recall@5 | 1.000 | 0.868 | -13.2% |
| Avg Latency | 60.29ms | 56.95ms | -5.6% |
| P50 Latency | 56.17ms | 55.13ms | -1.8% |
| P95 Latency | 88.69ms | 75.85ms | -14.5% |
A direct brute-force comparison of SQ8 against Float32 on the same 50-query set shows that Recall@5 drops from 1.000 to 0.868 — a 13.2% degradation. This contrasts with the earlier reconstruction-quality metrics (MSE ≈0.0019, mean cosine ≈0.999983), which looked near-lossless: those are averaged across the whole corpus, whereas Recall@K depends only on correctly ranking the handful of nearest neighbors, where score gaps are small enough that even bounded per-dimension quantization error can flip the top-5 ordering. In other words, SQ8 preserves global embedding geometry well but measurably degrades fine-grained ranking precision exactly where retrieval quality is judged, so the 4× memory win is a real accuracy tradeoff rather than a free lunch, and should be reported alongside the recall number rather than the reconstruction metrics alone.
| Metric | Value |
|---|---|
| Peak RAM during build | 3072.01 MB |
| Python requirement | 64-bit only |
| Float32 store generation | 9.59 s |
| SQ8 store generation | 15.65 s |
Quantization increases preprocessing time but is performed only once.
Although the final SQ8 storage occupies only 122.07 MB, the storage construction process requires substantially more memory than the final artifact.
During build_storage.py, several temporary arrays coexist in memory, including:
- the original float32 embedding matrix,
- intermediate normalized arrays,
- the quantized matrix,
- per-dimension minimum values,
- scaling coefficients.
As a result, the measured peak resident memory during storage construction reached 3072.01 MB.
It was measured by tracemalloc
if __name__ == "__main__":
tracemalloc.start()
main()
current, peak = tracemalloc.get_traced_memory()
print(f"Current memory: {current / 10**6:.2f} MB")
print(f"Peak memory: {peak / 10**6:.2f} MB")
The sq8 .npz build RAM peak is 1400.59 Mb since it does not uploads more copies of float32 .npy.
Because a 32-bit Python process is limited to approximately 2–4 GB of addressable memory (depending on the operating system), the storage build pipeline cannot reliably execute under Python x86. Consequently, the project requires a 64-bit Python interpreter for storage generation. This limitation applies only to the build stage; loading the final SQ8 storage requires approximately 122 MB of RAM.
- Scalar Quantization provides a deterministic 4× reduction in storage size with minimal reconstruction error.
- Separating the build phase from inference allows temporary high memory usage without affecting runtime performance.
- Exposing a common storage interface simplified integration with the Indexing and Search modules.
- Peak build memory is dominated by temporary preprocessing buffers rather than the final storage artifact, making 64-bit Python a practical requirement despite the compact production output.