-
Notifications
You must be signed in to change notification settings - Fork 1
Cross Process Cache Sync
What this page covers: how to keep Caching Managers fresh when several
application instances share one backend. A CachingManager only sees its own writes, so when
instance A updates an entity, instances B…N serve their stale copy until TTL. CacheSync closes that
gap — the single entry point that wires a backend-native push feed (ChangeFeedStorage) where
one exists and falls back to polling (PollingCacheSync) where it doesn't, so the same wiring
works on any backend.
📌 Note — this lives in the optional
everydatabase-manageradd-on (…manager.sync), on top of a core capability (…changefeed). It is complementary to Optimistic Locking: optimistic locking resolves write conflicts (lock_version); cache sync resolves read staleness. The two compose — the lock version is also the freshness token polling rides on.
flowchart LR
A["Instance A<br/>saveAndCache(guild)"] -->|write| DB[(shared backend)]
DB -->|"push: change feed<br/>OR poll: versions()"| S["Instance B<br/>CacheSync"]
S -->|"SAVE → invalidate(key)<br/>DELETE → evict(key)"| M["B's CachingManager"]
M -->|next resolve| R["reloads authoritative state"]
Without cache sync, instance B's cache is an island: it never learns that A changed the guild. With it, A's write nudges B's cell to reload on the next access.
import br.com.finalcraft.everydatabase.manager.*;
import br.com.finalcraft.everydatabase.manager.cache.CachePolicy;
import br.com.finalcraft.everydatabase.manager.sync.CacheSync;
import java.time.Duration;
RefRegistry registry = new RefRegistry();
CachingManager<UUID, Guild> guilds = registry.manager(GUILDS, storage, CachePolicy.always());
CachingManager<UUID, Player> players = registry.manager(PLAYERS, storage, CachePolicy.ttl(Duration.ofSeconds(30)));
CacheSync sync = CacheSync.attach(storage)
.pollEvery(Duration.ofSeconds(10)) // only used if `storage` can't push (MySQL/MariaDB)
.bind(guilds)
.bind(players)
.start();
// ... on shutdown:
sync.close();That's the whole integration. A write through any instance now invalidates that entity's cache in
every other instance bound through CacheSync. You don't need to know your backend: switching
MySQL → Mongo is no code change, it just gets faster (push instead of poll).
See it end-to-end in the tests: the backend-agnostic
AbstractCacheSyncTest(a writer+reader pair on one shared DB), plusCacheSyncTest(routing/own-origin) andPollingCacheSyncTest.
CacheSync hides which mechanism a backend uses. It groups bound managers by their change
source — the storage passed to attach(storage), or (under auto()) each manager's own
CachingManager.storage() — and per group picks:
-
Push — if that source implements
ChangeFeedStorage(MongoDB, PostgreSQL, InMemory, LocalFile, GroupedFile), itsubscribes to the native change feed. Events arrive as writes happen;pollEveryis ignored. -
Poll — otherwise (MySQL/MariaDB, H2) it runs a
PollingCacheSynceverypollEvery(...)interval, version-checking the keys currently in the cache.
CacheSync.attach(storage)
.pollEvery(Duration.ofSeconds(10)) // required only if `storage` can't push
.bind(guilds)
.start();
⚠️ Gotcha — if a bound manager's backend cannot push and you set nopollEvery,start()throwsIllegalStateExceptionnaming the storage and telling you to add.pollEvery(Duration)(or back the manager with a push-capable storage). It never silently does nothing.
Push matters wherever instances genuinely share state: the networked backends, and a directory two
processes both write (a network mount, or one process holding two storages). Polling works everywhere
through Repository.versions(...).
| Backend | Push (ChangeFeedStorage) |
Transport | Poll fallback | Update detection on poll |
|---|---|---|---|---|
| MongoDB | ✅ | Change Streams (resumable; needs a replica set) | ✅ | n/a (push) |
| PostgreSQL | ✅ |
LISTEN/NOTIFY (fire-and-forget) |
✅ | n/a (push) |
| InMemory | ✅ | local-write callback (per-process reference impl) | ✅ | n/a (push) |
| MySQL / MariaDB | ❌ | Jedis pub/sub (opt-in) | ✅ (primary) | ✅ versioned only |
| H2 | ❌ | Jedis pub/sub (opt-in) | ✅ | ❌ deletes only |
| LocalFile | ✅ | OS watch service (WatchService) |
✅ (full scan) | ✅ file stamp |
| GroupedFile | ✅ | OS watch service (WatchService) |
✅ (full scan) | ✅ file stamp |
A few footnotes:
-
MongoDB is the best push: Change Streams are resumable, so an instance that briefly reconnects
misses nothing within the oplog window. Both SAVE and DELETE propagate — the entity key is the
document
_id, so a delete event carries the key with no pre-images. Requires a replica set (already needed for Transactions); the docker-compose Mongo is a 1-node replica set, so the Mongo sync tests run by default. -
PostgreSQL needs zero extra infra (
pg_notifyafter each write, a dedicated listener connection), but is fire-and-forget: an instance disconnected during aNOTIFYmisses it → pair the managers with aCachePolicy.ttl(...)safety net. - MySQL/MariaDB has no native pub/sub — polling is the intended path.
- LocalFile / GroupedFile push through the operating system's own file-watch notification, one daemon thread per storage. This is the only feed that sees a change made outside the application — an administrator editing a YAML file by hand invalidates caches exactly like a write through the API. Two caveats: the event carries no origin (a file system has nowhere to record who wrote a file), so a local write echoes back as one self-invalidation; and on macOS the JDK falls back to an internal polling watcher with second-scale latency.
-
H2 is embedded and single-node, so cross-process sync is mostly moot; polling there catches
deletes only, because it does not enforce
lock_version(every existing key reports version0, so an in-place update is invisible to the poller — see Optimistic Locking). The file backends have no lock column either, but stamp each key from its own file, so polling detects their updates too. - Jedis pub/sub (the Transport column) is an opt-in third mechanism giving the feedless backends real push (lower latency than polling, no per-key version-check load on the DB). See the pub/sub transport section below.
Push is built on a small core capability — the same "capabilities are interfaces, not flags" idiom as Transactions and Schema Migrations (see Architecture Overview). A backend that can observe its own changes implements:
public interface ChangeFeedStorage extends Storage {
String originId(); // stable id of THIS instance
ChangeSubscription subscribe(ChangeListener listener);
}A listener receives immutable ChangeEvents carrying just enough to act — never entity content
(same privacy posture as Logging & Diagnostics):
| Field | Type | Meaning |
|---|---|---|
collection() |
String |
the changed entity's collection |
key() |
String |
the key in its persisted form (key.toString()) |
op() |
ChangeOp |
SAVE or DELETE
|
version() |
long |
the lock_version after the change, or -1 when unknown/unversioned |
originId() |
String |
the producing instance's originId(), or empty when the source can't attribute it |
You normally never touch this — CacheSync subscribes for you. The raw subscription is the escape
hatch to drive something other than a cache:
if (storage instanceof ChangeFeedStorage feed) {
ChangeSubscription sub = feed.subscribe(event -> log.info("changed: {}", event));
// ... sub.close() to stop; closing the storage closes all its subscriptions.
}📌 Note — delivery is at-least-once, unordered, and (Postgres) lossy. That's safe for cache invalidation:
invalidateonly marks a cell stale, and the actual reload re-publishes through the cell's monotonic stamp, so a duplicate or out-of-order event can never regress a newer local write nor resurrect a newer delete. Don't use a change feed as a reliable event log.
Use attach(storage) when every bound manager lives on the same storage. It subscribes once
(push) or runs one poller (poll) for the whole group:
CacheSync sync = CacheSync.attach(storage)
.pollEvery(Duration.ofSeconds(10)) // ignored if `storage` pushes
.bind(guilds)
.bind(players)
.onError(t -> log.warn("cache-sync", t)) // optional: surface key-parse / poll errors
.start();When your managers are spread across different backends (the [[One Entity, Many
Databases|One-Entity-Many-Databases]] pattern), auto() routes each one by its own
CachingManager.storage() — push where the backend supports it, poll where it doesn't, in one binder:
CacheSync sync = CacheSync.auto()
.pollEvery(Duration.ofSeconds(10)) // fallback for the non-push managers
.bind(guildsOnMongo) // -> push (change stream)
.bind(walletsOnMySql) // -> poll (version polling)
.start();📌 Note —
auto()needs each manager to carry its storage, which is true for any manager built throughRefRegistry.manager(...)or theCachingManager(descriptor, storage, …)constructors.
The third mechanism: an explicit pub/sub transport, decoupled from the data backend. Where a backend
has no native feed (MySQL/MariaDB, H2), CacheSync would otherwise poll; a
transport replaces that poll with real push over a Redis/Valkey channel — lower latency, no periodic
versions(...) load on your database.
🧭 When is it worth it?
- ✅ Use it on a feedless backend (MySQL/MariaDB/H2): the sweet spot. You go from "poll every N seconds" to instant push, and the database stops answering version-check queries on a timer.
- 🤷 Already on Mongo, PostgreSQL, or a file backend? They have a native change feed (Change Streams /
LISTEN/NOTIFY/ the OS watch service) that already pushes — adding Jedis buys you little and just adds a server to run. Only reach for it to get one uniform channel across mixed backends, or to move invalidation traffic off the database. Otherwise stick with the native feed.- ❌ Single instance? You don't need cache sync at all.
An optional add-on module — declare it alongside everydatabase-manager and everydatabase-core
(it brings the Jedis client; nothing else does). Available from 1.0.5:
implementation 'br.com.finalcraft.everydatabase:everydatabase-manager-jedis:1.3.0'
implementation 'br.com.finalcraft.everydatabase:everydatabase-manager:1.3.0'
implementation 'br.com.finalcraft.everydatabase:everydatabase-core:1.3.0'import br.com.finalcraft.everydatabase.manager.sync.CacheSync;
import br.com.finalcraft.everydatabase.manager.sync.CacheSyncTransport;
import br.com.finalcraft.everydatabase.manager.sync.jedis.JedisCacheSyncConfig;
import br.com.finalcraft.everydatabase.manager.sync.jedis.JedisCacheSyncTransport;
// One Jedis client speaks to both Redis AND Valkey (identical RESP wire protocol).
CacheSyncTransport transport = JedisCacheSyncTransport.connect(
new JedisCacheSyncConfig("localhost", 6379));
CacheSync sync = CacheSync.attach(mySqlStorage) // any backend, including feedless ones
.via(transport) // push over pub/sub instead of polling
.bind(guilds)
.bind(players)
.start();
// on shutdown:
sync.close(); // stops the subscription + clears the publish hooks
transport.close(); // the transport's lifecycle is yours (close it after the syncs that use it).via(transport) takes precedence over a native feed and over polling. Each bound manager publishes
a tiny signal on every local write; the other instances invalidate/evict the matching key. The
local in-memory cache stays the source of live objects — the transport only signals "reload", so
it never touches the identity map (it is not a shared L2 cache of values).
flowchart LR
A["Instance A<br/>saveAndCache(guild)"] -->|"repository.save() (MySQL/...)"| DB[(any backend)]
A -->|"PUBLISH everydatabase:changes {coll,key,SAVE,origin}"| R[(Redis / Valkey)]
R -->|SUBSCRIBE| B["Instance B<br/>CacheSync.via(transport)"]
B -->|"SAVE → invalidate(key)"| M["B's CachingManager"]
⚠️ Only manager-mediated writes propagate. The signal is published from the manager's write path (saveAndCache/deleteAndEvict/saveAllAndCache). A rawrepository().save(...), a Moving Data Between Backends transfer, or a migration does not publish — write through the manager if you want the invalidation to fan out.
Minimal form needs only host + port; everything else defaults. For production use the builder:
JedisCacheSyncConfig cfg = JedisCacheSyncConfig.builder("redis.internal", 6380)
.ssl(true)
.username("cache-sync").password(secret) // Redis 6+ ACL
.connectTimeoutMs(3000).socketTimeoutMs(3000)
.channel("myapp:changes") // channel isolation, see below
.build();| Setting | Default | Notes |
|---|---|---|
host / port
|
— / 6379
|
required host; standard port |
password / username
|
none | password-only AUTH, or ACL user (Redis 6+) |
database |
0 |
SELECTed on connect (pub/sub itself is server-global) |
ssl |
false |
TLS |
connectTimeoutMs / socketTimeoutMs
|
2000 / 2000
|
bound a hung handshake |
channel |
everydatabase:changes |
the pub/sub channel; see below |
Channel isolation. Pub/sub channels are global per server (not scoped by database index). If two
unrelated apps (or prod + staging) share one Redis/Valkey and both keep the default channel, A's signals
reach B. Give each app its own channel with .channel("myapp:changes") / config.withChannel(...) when
you share a server.
📌 Note — a malformed or foreign payload on the channel (e.g. an unrelated app publishing to a colliding channel) is surfaced to your error handler (
onError(...)/ counted inparseFailures()), not dropped silently. So a channel collision is observable — if you see parse failures climbing, something else is publishing to your channel; give the app its own.channel(...).
Since 1.1.0, a signal is scoped by the physical store it came from, not merely the collection
name. Every Storage answers backendIdentity() — a stable identity of the underlying store that never
contains credentials (host:port/db for a server, an absolute path plus a machine discriminator for a
file/local backend). This scoping works on two levels:
-
In-memory routing (
CacheSync, for every transport and the native feed): an event carries its originbackendId, andCacheSynconly invalidates the managers that share that identity. Two apps that share the Redis channel but keep separate physical stores no longer cross-invalidate. -
Physical channel (Jedis transport specifically): the effective channel is
<channelBase>:<backendId>, and an instance subscribes only to the channels of the stores it actually reads. This is a second, orthogonal axis tochannel(): the prefix isolates applications, the **:<backendId>suffix isolates stores within one application. The base channel is always subscribed too, so a producer that sends nobackendId(a pre-1.1.0 peer) still reaches everyone — backward-compatible.
When two backends are textually different but you know they are the same physical store (two
localhost:3306 from different hosts' point of view, or two file directories a shared mount makes one),
or you simply want to declare a store shareable, set sharedIdentity("...") on the config —
when present, it is the backend identity, on every *Config.
⚠️ Gotcha —SyncParticipationand machine-local stores. A tri-statesyncParticipation(RECOMMENDEDdefault /ALWAYS/NEVER) on every*Configgates the transport's publish side only (the native feed and the receive side are untouched). Under the defaultRECOMMENDED, a machine-local backend — H2mem:, LocalFile / GroupedFile, SQL / Mongo onlocalhost— does not publish on the transport. That saves traffic for the common single-process case, but it is exactly the setup people reach for when simulating "two instances" on one box: two local processes syncing via Redis/Valkey over such a backend will silently stop seeing each other's writes. The fix is to tell the library they are the same store — declare the samesharedIdentity("...")on both configs, orsyncParticipation(SyncParticipation.ALWAYS).ALWAYSon a machine-local backend still requires asharedIdentity, or the bind fails fast withIllegalStateException— publishing to a channel nobody else subscribes to would be a silent no-op, so it is refused loudly rather than swallowed.
If the transport drops (server unreachable), CacheSync falls back to version polling until it
reconnects, then steps back to push — on by default. The fallback cadence is pollEvery(...) if set,
otherwise 30s. Disable with .transportFallback(false) to rely purely on the transport + a TTL:
CacheSync.attach(storage).via(transport)
.pollEvery(Duration.ofSeconds(15)) // fallback cadence while the transport is down
.transportFallback(true) // default; .transportFallback(false) to disable
.bind(guilds).start();📌 With the fallback enabled, use one transport per
CacheSync(a transport drives a single connectivity listener). Sharing one across several syncs would leave the earlier syncs' fallback inactive — give each its own, or disable the fallback.
A single transport is backend-agnostic — in auto() mode it syncs managers spread across different
backends through one channel, routing by collection:
CacheSync sync = CacheSync.auto()
.via(transport) // one Redis/Valkey channel for all of them
.bind(guildsOnMySql)
.bind(walletsOnH2)
.bind(profilesOnMongo)
.start();
⚠️ Because routing is purely by collection name, every bound manager must use a unique collection — two managers on the same name (even on different backends) is rejected atstart().
Valkey and Redis are interchangeable here: the same Jedis client works against both unchanged (Valkey is a RESP-compatible fork). CI exercises the transport against both.
The cache and the cache-sync layer expose always-on, dependency-free metrics (counting is ~free; only reading a snapshot allocates). For the core's event log see Logging & Diagnostics.
CacheStats s = guilds.stats();
s.hitCount(); s.missCount(); s.hitRate();
s.loadSuccessCount(); s.loadFailureCount();
s.invalidationCount(); s.evictionCount(); s.liveSize();Counts manager-mediated reads (resolve/peek/getAll); a Ref that has memoized its cell reads
lock-free and is not counted. Never carries entity content. resetStats() zeroes the counters.
The single most useful operational signal is which mechanism you're on right now:
cacheSync.mode(); // FEED | POLL | TRANSPORT_PUSH | TRANSPORT_FALLBACK_POLL | IDLE
cacheSync.transportConnected(); // is the transport up?
cacheSync.timeInFallbackMillis(); // total time the transport has spent disconnected
CacheSyncStats st = cacheSync.stats();
st.signalsReceived(); st.signalsApplied(); st.signalsSkippedOwnOrigin();
st.signalsUnmapped(); st.parseFailures();📌 Note — while a transport's push is live,
mode()(andstats()) reportTRANSPORT_PUSH— the steady state of a wired-and-connected transport.TRANSPORT_FALLBACK_POLLshows up only once the sync has actually fallen back to polling on a transport drop — not when fallback is merely disabled or the transport hasn't reported connectivity yet.
Register a CacheSyncObserver to be notified on transitions (e.g. log "transport down → fallback"):
cacheSync.observe(new CacheSyncObserver() {
@Override public void onTransportDisconnected() { log.warn("cache-sync: transport down, polling"); }
@Override public void onTransportConnected() { log.info("cache-sync: transport back, push"); }
@Override public void onModeChange(CacheSyncMode mode) { meter.gauge("cachesync.mode", mode); }
});The Jedis transport also exposes its own counters: transport.publishCount(),
transport.publishFailureCount(), transport.reconnectCount(), transport.connected().
📌 Don't instrument "drops". A dropped signal on a fire-and-forget bus is unobservable by the receiver — what's actionable is publish failures, reconnects, and time in fallback, which is exactly what's exposed. A TTL policy is the self-heal for the drops you can't see.
The push path receives the key as a String (its persisted toString()) and must turn it back into
the cache's K. The built-in KeyParsers cover the common key-contract types out of the box —
String, UUID, Long, Integer — so bind(manager) just works for those:
sync.bind(guilds); // UUID key -> UUID.fromString, resolved automaticallyFor a composite / record / wrapper key (no general inverse of toString()), pass an explicit
String -> K parser:
sync.bind(sessions, str -> Session.Id.parse(str)); // your own key parser📌 Note — the parser is only used on the push path; polling compares cached keys directly and never re-parses. An unparseable key is routed to
onError(...)and skipped — it never throws into the backend's delivery thread and never breaks the feed.
By default CacheSync skips events this instance produced — when event.originId() matches the
attached storage's originId(). Your own write already refreshed your cache write-through, so
re-invalidating it would just cause a wasted reload.
CacheSync.attach(storage).pollEvery(d).bind(guilds).includeOwnOrigin().start();Call .includeOwnOrigin() to process your own events too — for the rare topology of several caches
over one storage in one process (or an in-process test where the writer's own change must still fan
out).
📌 Note — when the source can't attribute origin (Mongo's oplog, a DB trigger), the event's origin is empty and the skip simply never fires. The instance then reloads its own just-written key — harmless (write-through already had the value; the reload re-reads the same state), just one extra read. The skip only kicks in where the library authors the payload (Postgres, InMemory).
For backends without a push feed, CacheSync's fallback delegates to PollingCacheSync. You can also
use it directly when you want polling without going through the facade:
import br.com.finalcraft.everydatabase.manager.sync.PollingCacheSync;
PollingCacheSync poller = PollingCacheSync.every(Duration.ofSeconds(10))
.bind(guilds)
.bind(players)
.start();
// ... poller.close() on shutdown.Each tick reads the current lock_version of every bound manager's currently cached keys — a
cheap key+version read, bounded by cache size, not table size — via the repository primitive:
CompletableFuture<Map<K, Long>> versions(Collection<K> keys); // key + version only, never the bodyand then:
- invalidates a key whose backend version increased since the last poll (another instance wrote it), so the next read reloads it;
- evicts a key cached here but missing from the backend (deleted elsewhere).
- Latency is one poll interval — not instant.
-
Updates need versioning. Detecting an in-place update requires a versioned descriptor
(
@OptimisticLock). A non-versioned descriptor — or H2 — reports version0for every existing key, so polling then catches only deletes. -
First-observation gap. A key is assumed as fresh as the backend the first time it is polled (it
was usually just loaded); a write in the brief window between a cache load and that key's first poll
can be missed. All later writes are caught — pair with a
CachePolicy.ttl(...)if that window matters.
💡 Tip —
pollOnce()(on bothCacheSyncandPollingCacheSync) runs one cycle synchronously. It's how the tests stay deterministic instead of waiting a tick; production relies on the scheduled interval.
-
start()is idempotent; so isclose(). Bind every manager beforestart()—bind(...)after start throws. - Push runs on the backend's listener thread (Mongo change-stream cursor / Postgres
LISTENconnection); poll runs on a single daemon thread namedeverydatabase-cache-poller. Neither keeps the JVM alive — but alwaysclose()yourCacheSyncon shutdown to stop them cleanly. See Concurrency & Threading. -
isRunning()reports whether the sync started with at least one push or poll group.
| Property | What it means |
|---|---|
| At-least-once | an event may be duplicated or reordered; the cell stamp makes that harmless |
| Lossy (Postgres) | a NOTIFY can be dropped on disconnect → keep a TTL as the safety net |
| Safe by construction |
invalidate only marks stale; the reload reads authoritative state, stamp-guarded |
| Eventually consistent | other instances converge on the next read after an event (push) or poll tick |
🧭 Decision — wire
CacheSyncwhenever more than one instance writes the same data through a shared backend. Use push backends (Mongo/Postgres) for near-instant invalidation; on a poll backend pick an interval you can tolerate as staleness, and always keep aCachePolicy.ttl(...)as the backstop for a dropped or missed event. Single-instance deployment? You don't need this at all.
-
Caching Managers — the
invalidate/evictsink that cache sync drives, and the cell/stamp model that makes a late invalidation safe. - Cache Policies & Freshness — the TTL safety net for fire-and-forget and poll backends.
-
Optimistic Locking — the
lock_versiontoken polling rides on; the complementary write-side guard. -
One Entity, Many Databases — managers across backends, the
auto()case. - MongoDB · PostgreSQL · MySQL & MariaDB — per-backend transport details.
- Concurrency & Threading — the listener/poller threads.
- The Redis/Valkey pub/sub transport ships as the
everydatabase-manager-jedismodule; see it end-to-end inAbstractJedisCacheSyncTest(Valkey + Redis) andMultiBackendJedisCacheSyncExampleTest(one transport across many backends).
EveryDatabase · Home · made by Petrus Pradella
Getting Started
Core Concepts
Working with Data
Backends
- Choosing a Backend
- MySQL & MariaDB
- PostgreSQL
- H2
- MongoDB
- Local Files
- Grouped Files
- In-Memory
- Benchmarks
Manager Module
- Caching & References
- Typed References (Ref)
- Caching Managers
- Cache Policies & Freshness
- Cross-Process Cache Sync
- Write-Back & Conflict Resolution
- Payload Schema Evolution
- One Entity, Many Databases
Operations
Advanced
Reference
Contributing