Skip to content

fix: enforce one NetFlow v9 template owner - #303

Merged
mikemiles-dev merged 2 commits into
mikemiles-dev:mainfrom
ktsaou:fix/v9-template-id-ownership
Jul 22, 2026
Merged

fix: enforce one NetFlow v9 template owner#303
mikemiles-dev merged 2 commits into
mikemiles-dev:mainfrom
ktsaou:fix/v9-template-id-ownership

Conversation

@ktsaou

@ktsaou ktsaou commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

What was wrong

NetFlow v9 uses one Template ID namespace for both data templates and Options Templates. The parser stored those two kinds in separate caches, so both could own the same ID at the same time.

Data lookup checks the data-template cache first. If an exporter changed an ID from a data template to an Options Template, records could therefore be decoded using the older data template instead of the latest definition.

Required behavior

RFC 3954 section 7 requires a collector to discard the previous definition when a Template ID is reused and use the new definition. Akvorado and GoFlow2 follow the same pattern: one stored value per version, Observation Domain, and Template ID, regardless of template kind.

What this changes

  • A valid data template removes an Options Template with the same ID.
  • A valid Options Template removes a data template with the same ID.
  • The last valid definition in wire order becomes the sole owner.
  • A live cross-kind replacement counts as a collision, not an LRU eviction.
  • An expired opposite-kind entry counts as an expiration, not a collision.
  • A successful persistent-store update removes the superseded physical key before writing the new owner.

The public API and the two physical LRU caches remain unchanged. Data-record lookup receives no additional work; the added operations run only when a template definition is received.

Persistent-store operations retain their existing best-effort error behavior. Stores populated by an older parser are not rewritten eagerly; the next valid template announcement corrects any pre-existing conflicting keys.

Tests

Three focused regression tests prove:

  • both data-to-Options and Options-to-data replacement follow wire order;
  • only one logical owner remains and records use the latest definition;
  • only the latest physical key remains in TemplateStore, and a fresh parser restores it correctly;
  • an expired opposite-kind owner is reported as expiration rather than collision.

The focused tests fail on the previous implementation and pass with this change. The complete project checks pass, including formatting, Clippy, build, unit and integration tests, documentation tests, README synchronization, benchmark compilation, and focused tests without default features.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes NetFlow v9 Template ID ownership so that ordinary (data) templates and Options templates share a single logical Template ID namespace, ensuring the most recent valid definition in wire order is the sole owner (per RFC 3954 §7). This prevents data flowsets from being decoded against a stale template when exporters reuse a Template ID across template kinds.

Changes:

  • Enforced cross-kind “single owner” semantics during template installation (cross-kind replacement removes the other kind from cache/store and is tracked as a collision when live).
  • Updated v9 parser documentation and README to describe the shared Template ID namespace behavior.
  • Added regression tests covering cross-kind replacement, persistent TemplateStore behavior, and expired-opposite-kind metrics handling.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
src/variable_versions/v9/parser.rs Centralizes template installation to enforce single-owner Template ID semantics across ordinary vs Options templates (including store key cleanup and metrics updates).
tests/v9_template_id_ownership.rs Adds focused regression tests proving wire-order ownership, persistent store key cleanup, and correct collision/expiration accounting.
README.md Documents that v9 ordinary and Options templates share one Template ID namespace and latest definition wins.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/variable_versions/v9/parser.rs Outdated
Comment thread tests/v9_template_id_ownership.rs
Comment thread tests/v9_template_id_ownership.rs
@ktsaou
ktsaou force-pushed the fix/v9-template-id-ownership branch from a6ca0e0 to 4aa5599 Compare July 21, 2026 08:12
@mikemiles-dev

Copy link
Copy Markdown
Owner

Can you
Rebase?

@ktsaou
ktsaou force-pushed the fix/v9-template-id-ownership branch from 4aa5599 to 069217b Compare July 22, 2026 07:28
@ktsaou

ktsaou commented Jul 22, 2026

Copy link
Copy Markdown
Contributor Author

rebased

@mikemiles-dev
mikemiles-dev merged commit baeb440 into mikemiles-dev:main Jul 22, 2026
6 checks passed
@ktsaou ktsaou mentioned this pull request Jul 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants