Skip to content

fix: enforce one IPFIX template owner - #304

Merged
mikemiles-dev merged 2 commits into
mikemiles-dev:mainfrom
ktsaou:fix/ipfix-template-id-ownership
Jul 22, 2026
Merged

fix: enforce one IPFIX template owner#304
mikemiles-dev merged 2 commits into
mikemiles-dev:mainfrom
ktsaou:fix/ipfix-template-id-ownership

Conversation

@ktsaou

@ktsaou ktsaou commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

What was wrong

The IPFIX parser accepts four template encodings:

  • native Data templates
  • native Options templates
  • NetFlow v9-style Data templates carried through the IPFIX parser
  • NetFlow v9-style Options templates carried through the IPFIX parser

These were stored in four independent caches. If the same Template ID was defined under two different encodings, both definitions remained cached. Data lookup used a fixed cache priority, so an older higher-priority definition could silently override the later wire definition.

The bundled IPFIX capture demonstrates the accounting symptom: it contains 15 unique Template IDs, but the parser retained 17 physical cache entries because IDs 262 and 263 were each defined as Options, then Data, then Options templates.

Protocol basis

RFC 7011 sections 3.4.1 and 3.4.2 require Templates and Options Templates to use one Template ID namespace within a Transport Session and Observation Domain. For UDP, section 8.4 says that a new different definition replaces the previous definition.

The v9-style encodings accepted by this parser follow the same logical ownership rule: one active definition per Template ID.

What this changes

  • The last valid definition in wire order becomes the sole in-memory owner of its Template ID.
  • The other three physical cache entries are removed before installation.
  • A live cross-kind replacement records one collision.
  • An expired opposite owner records expiration instead of collision.
  • Superseded keys are removed from the optional TemplateStore before the new owner is written.
  • Existing cache capacities and public APIs remain unchanged.
  • Data-record lookup performs exactly the same work as before; cleanup runs only when a template definition is received.

The store removals are intentionally unconditional when a store is configured. An empty local cache does not prove that a persisted key is absent after restart or earlier divergence.

Compatibility

This corrects observable behavior for streams that reuse one Template ID across template encodings:

  • cache size now reports one entry per logical Template ID
  • the last accepted definition determines decoding
  • collision and expiration metrics include cross-kind replacement

A store created by older code may already contain conflicting physical keys. Existing read-through priority remains unchanged until the next wire definition, which cleans the conflicting keys. General store migration and transactional store behavior are not changed here.

Tests

The regression suite covers:

  • all 12 ordered transitions across the four accepted template encodings
  • decoded record kind after each transition
  • logical cache size and collision, expiration, insertion, and eviction metrics
  • persistent-store cleanup and fresh-parser restoration for all 12 transitions
  • expired opposite-owner replacement
  • the bundled real IPFIX capture, including the corrected 15-template cache size

The focused tests pass with default and no-default features. The complete project checks pass: formatting, Clippy, build, unit and integration tests, documentation tests, README synchronization, and benchmark compilation.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Enforces a single logical owner per IPFIX Template ID across native IPFIX and V9-style template encodings, preventing stale higher-priority cache entries from silently shadowing later on-wire definitions and aligning behavior with RFC 7011’s single Template ID namespace rule.

Changes:

  • Reworked IPFIX template insertion to evict the same Template ID from the other three physical caches (and remove superseded keys from the optional TemplateStore) before installing the new definition.
  • Added a dedicated regression test suite covering cross-kind ownership transitions, store cleanup, and the “expired opposite owner” metric behavior.
  • Updated the bundled pcap integration expectations and documented the unified Template ID namespace behavior.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tests/pcap_integration.rs Updates expected cache size from the bundled IPFIX capture (17 → 15) to match the new ownership behavior.
tests/ipfix_template_id_ownership.rs New regression tests for cross-kind Template ID ownership, store cleanup, and expired-opposite-owner metrics.
src/variable_versions/ipfix/types.rs Documents the single logical Template ID namespace across accepted template representations.
src/variable_versions/ipfix/parser.rs Implements cross-cache Template ID eviction on template installation and updates lookup commentary accordingly.
README.md Documents unified Template ID namespace behavior in the cache behavior section.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tests/ipfix_template_id_ownership.rs
Comment thread src/variable_versions/ipfix/parser.rs
@ktsaou
ktsaou force-pushed the fix/ipfix-template-id-ownership branch from d5299d6 to 53b43f1 Compare July 21, 2026 08:09
@mikemiles-dev
mikemiles-dev merged commit b802c4c into mikemiles-dev:main Jul 22, 2026
6 checks passed
@ktsaou ktsaou mentioned this pull request Jul 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants