[#12019] feat(core): make caffeine entity cache multi-node correct via change-log invalidation#12111
Draft
yuqi1129 wants to merge 6 commits into
Draft
[#12019] feat(core): make caffeine entity cache multi-node correct via change-log invalidation#12111yuqi1129 wants to merge 6 commits into
yuqi1129 wants to merge 6 commits into
Conversation
…cache in RelationalEntityStore
…ties only Relation lists, the reverse index and its rules are no longer maintained. A radix prefix index still provides hierarchical cascade invalidation. USER/GROUP/ROLE entities are excluded from caching because their materialized form embeds relation-derived data that can no longer be invalidated when referenced entities change.
…ys and reverse index
… via an allowlist
…ect via change-log invalidation Add an EntityCache.coherence() marker (LOCAL_PER_NODE | SHARED) and an entity store EntityCacheChangeLogListener that replays each entity_change_log ALTER/DROP row as invalidate(ident, type); a container drop cascades to cached children through the existing forward prefix scan. The listener is registered as a third consumer of the existing poller, next to the catalog and jcasbin caches, only for a LOCAL_PER_NODE cache (the coherence gate). Add in-transaction change-log emit points for tag, policy and job (updateTag / deleteTag, updatePolicy / deletePolicy, insertJob overwrite / deleteJob), the cached types that emitted no row before. Structural rows already exist.
Code Coverage Report
Files
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Make the default
caffeineentity store cache correct in multi-node deployments by reusing the existingentity_change_log+EntityChangeLogPoller, per the #11923 design (Phase 1 of #11737).EntityCache.coherence()→LOCAL_PER_NODE | SHARED(cache/Coherence.java);BaseEntityCachedefaults toLOCAL_PER_NODE, socaffeineis per-node. A future Redis cache reportsSHARED.EntityCacheChangeLogListenerreplays each ALTER/DROP row asinvalidate(ident, type); a container drop cascades to cached children via the existing forwardcacheIndexprefix scan. Registered as a third poller consumer (next to the catalog and jcasbin id-mapping caches) only whencoherence() == LOCAL_PER_NODE— the coherence gate.TagMetaService(update/delete),PolicyMetaService(update/delete),JobMetaService(overwrite/delete). Structural rows already exist.Why are the changes needed?
Today only the writing node invalidates its cache, so other nodes serve stale data until TTL expiry — forcing
gravitino.cache.enabled=falsein HA deployments. This reuses infrastructure already present (no schema change, no new dependency), socache.enabled=truebecomes safe across nodes with the default cache.Fix: #12019
Does this PR introduce any user-facing change?
No. Internal caching behavior only; uncached types and single-node behavior are unchanged.
How was this patch tested?
TestEntityCacheChangeLogListener— dispatch/cascade/sibling-preservation, per-type invalidation, malformed-row skipping, fault isolation, coherence values.TestEntityChangeLogService— the new tag/policy/job ALTER & DROP rows are emitted (a plain create emits none).TestEntityCacheCrossNodeInvalidation— end-to-end: a node A write emits a change-log row and node B's poller invalidates its own cache (table ALTER and schema-drop child cascade).org.apache.gravitino.cache.*,TestEntityChangeLogPoller,TestTagMetaService,TestPolicyMetaService,TestJobMetaService,TestRelationalEntityStoreall pass;spotlessCheckandjavadocare clean.