Skip to content

perf: allocate template caches lazily - #307

Merged
mikemiles-dev merged 1 commit into
mikemiles-dev:mainfrom
ktsaou:perf/lazy-template-cache-allocation
Jul 21, 2026
Merged

perf: allocate template caches lazily#307
mikemiles-dev merged 1 commit into
mikemiles-dev:mainfrom
ktsaou:perf/lazy-template-cache-allocation

Conversation

@ktsaou

@ktsaou ktsaou commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Problem

Each parser owns two NetFlow v9 and four IPFIX template LRU caches. LruCache::new(capacity) immediately reserves the configured hash-map capacity, so a newly admitted source reserves storage for all six caches before receiving any template.

With the default capacity of 1,000, constructing one parser requested 209,760 bytes and retained 209,680 bytes solely for mostly empty cache state. This becomes significant for multi-source collectors that keep one parser per exporter/domain.

Change

  • Add a private LazyLruCache wrapper for the six template caches.
  • Keep the configured capacity without allocating the backing LRU until first insertion.
  • Grow the backing map incrementally instead of reserving the configured maximum.
  • Release the backing storage when the final entry is removed or the cache is cleared.

The wrapper preserves the existing capacity, LRU order, eviction behavior, TTL handling, withdrawals, metrics, persistence, public APIs, and parser debug representation. Source-routing and pending-flow LRUs are unchanged.

Measured results

Parser construction now requests 192 bytes and retains 112 bytes. With a configured capacity of 100,000, inserting the first template requests 848 bytes while still reporting the configured maximum and one live entry.

For 10,000 NetFlow v9 sources with one one-field template each:

Measurement Before After
Allocator-live bytes 2,109,200,000 15,800,000
RSS growth 417,644 KiB 16,936 KiB

Eleven alternating paired trials across direct and source-scoped NetFlow v9/IPFIX parse and iterator paths retained at least 98.336% of baseline hot-path throughput. Hot allocation counts and requested bytes were identical. Creating and parsing the first template for 1,000 new v9 sources was 2.01 times faster, while requested allocation fell from 211,732,084 to 2,392,084 bytes.

Verification

  • ./scripts/check-all.sh
  • Complete --no-default-features test suite
  • Rust 1.88 all-feature test compilation
  • Strict Clippy checks
  • Allocation regressions for empty construction and first insertion
  • Focused LRU behavior tests for reads, resize, eviction, removal, and clear

@mikemiles-dev
mikemiles-dev merged commit 786b341 into mikemiles-dev:main Jul 21, 2026
6 checks passed
@ktsaou ktsaou mentioned this pull request Jul 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants