Skip to content

Older live-state event can silently overwrite a newer one in the same batch #7136

Description

@vipulpandey21

What's happening

PatchKubernetesApplicationLiveState (pkg/app/server/applicationlivestatestore/store.go) applies a batch of Kubernetes resource-state events to a snapshot. Each event carries a version, and the code is supposed to skip any event older than what's already stored:

if ev.SnapshotVersion.IsBefore(*snapshot.Version) {
    continue
}

The problem: snapshot.Version is fetched once at the top of the function and never updated while the loop applies events. So every event in the batch gets compared against that same original version, not against the version the previous event in the loop just moved the state to.

Why it matters

If a batch has two events for the same resource where an older one arrives after a newer one (a redelivered retry is a normal way this can happen), the check above doesn't catch it. The older event silently overwrites the newer state — no error, no log.

How I found it

Traced how piped assigns event versions (pkg/app/piped/livestatestore/kubernetes/appnodes.go) through to how the server applies them. Wrote a test with two out-of-order events for the same resource — the final state ends up on the stale value.

I have a fix and a test ready, opening a PR for it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions