Skip to content

Latest commit

 

History

2 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

lease-fence

Two agents can both believe they hold the same lock. This module makes the resource able to prove which one is wrong.

import { createMonotonicClock, FenceGate, LeaseClient, LeaseCoordinator } from 'lease-fence';

const coordinator = new LeaseCoordinator({
  clock: createMonotonicClock('coordinator'),
  ttlMillis: 30_000,
});

const client = new LeaseClient({
  clock: createMonotonicClock('agent-a'),
  holderId: 'agent-a',
  clockSkewMillis: 500,
});

const gate = new FenceGate('deploy-slot');

const request = client.beginAcquire('deploy-slot');     // stamps the instant this was sent
const outcome = coordinator.acquire(request);           // the far side of the wire

if (!outcome.ok) {
  console.log(`held by ${outcome.heldBy}, free in ${outcome.retryAfterMillis}ms`);
} else {
  const lease = client.completeAcquire(request, outcome.grant);
  gate.commit(lease.token, () => applyTheChange());     // throws StaleFenceError if reassigned
}

The obvious implementation is wrong twice, in the same direction

Store expiresAt. Check Date.now() < expiresAt immediately before writing. On a successful renewal, set expiresAt = Date.now() + ttl. Roughly four lines, and both halves widen the same window.

The check does not cover the write

Date.now() < expiresAt is true at the moment it is evaluated. The write happens later. In an agent, "later" is a model call, and a model call can take longer than the lease.

t=0     agent A acquires, ttl 1000
t=950   A checks: 950 < 1000, valid, begin write
t=1000  the coordinator expires A and reassigns to B
t=1400  A's write lands

The check was correct when it was made and meaningless by the time it mattered. There is no way to shrink that gap from the holder's side, because the holder is exactly the participant that cannot observe its own stall.

The renewal deadline is measured from the wrong end

expiresAt = Date.now() + ttl on a renewal reply dates the lease from when the answer arrived. The coordinator dated it from when the question was processed. The difference is one round trip, and the holder always gets the longer of the two.

t=0     A sends a renewal
t=40    the coordinator extends its own copy to t=1040
t=900   the reply reaches A, which sets its deadline to t=1900

For 860 milliseconds A believes it holds a lease the coordinator has already let go. Nothing about the exchange failed; the arithmetic just leans the wrong way, and it leans that way on every renewal.

The result is a state that looks fine

In the overlap, both agents pass their own validity check, both write, and the coordinator ends up recording one coherent value with the later write winning. There is no error, no conflict, no log line. A test with a fake clock that advances only between calls never reproduces it, because the advance always lands outside the check-to-write gap rather than inside it.

What this does instead

The decision moves to the resource

FenceGate lives at the thing being written to and remembers the highest fence it has ever admitted. A write is a token plus a callback:

gate.commit(lease.token, () => applyTheChange());

The fence check and the write are the same operation, so nothing can happen between them. A token carrying a fence below the floor is refused with StaleFenceError, and the refusal is appended to gate.rejections with both holder ids, both fences, and a sequence number. That log is the only evidence anywhere in the system that two holders overlapped.

The floor is raised before the callback runs, not after. A reentrant write carrying the older fence has to be refused, and if the callback throws partway through, the resource may already carry part of the newer write, so lowering the floor back would let the older fence overwrite it.

There is deliberately no lease.assertValid(). hasExpired and wouldAdmit exist, they answer questions about the instant they are called, and they are documented as unsafe to branch a write on.

Deadlines are anchored where the request was sent

beginAcquire and beginRenewal capture the send instant and put it in the request object. completeAcquire and completeRenewal are the only functions that can turn a grant into a lease, and they read the anchor out of the request:

deadline = request.sentAt + grant.ttlMillis - clockSkewMillis;

The coordinator extends from its own reading, because that is the only moment it can actually vouch for. The holder anchors at the send instant, which is necessarily earlier. So the holder's window is always a subset of the coordinator's, no matter how slow the round trip is. A test in test/overlap.test.ts asserts that containment over two hundred randomised round trips.

A request is single use. Completing one twice throws, because an instant may anchor exactly one deadline, and reusing an anchor is how a longer lived lease inherits a timestamp that has nothing to do with its own request.

Fences move on handover and never on renewal

A fence identifies a generation of ownership, not a moment in time. Renewal returns the same fence, so writes already in flight under that lease stay valid. Acquisition after an expiry or a release mints a strictly higher one, which is what makes the previous generation refusable.

Two consequences fall out of that rule:

  • A holder that already owns the lock is denied a second acquisition rather than granted one. A second lease would mint a higher fence, and the resource would then refuse the holder's own in flight writes.
  • An expired lease is never resurrected, even when nobody else has taken it. The moment it expired, every waiting agent became entitled to acquire, and some may already have started work. Extending it would let it keep the fence a competitor is about to outrank, so renew returns { ok: false, reason: 'expired' } and the holder has to re-acquire.

A retried acquire does not burn a fence

A lost reply is indistinguishable from a lost request, so clients retry. Answering the retry with a fresh grant leaves the holder carrying fence N while the coordinator believes N+1 is live, and the holder's own writes are then refused by its own resource. The coordinator answers a repeated requestId with the grant that already exists, as long as that lease is still the current one.

Clocks carry the name of the clock they came from

Every reading is an Instant, which is a millisecond count plus a domain. Subtracting across domains throws ClockDomainError instead of returning a number. Two clocks have unrelated origins, and a monotonic clock's origin is arbitrary, so mixing them produces a value shaped exactly like a duration but off by whatever the offset happens to be. That is the failure mode that survives every type check.

grant.coordinatorExpiresAt is reported for logging and is stamped in the coordinator's domain, so measuring a lease against it throws rather than silently answering.

Clock sources are wrapped so a reading that is NaN, infinite, or lower than a previous one throws. Date.now() steps backwards whenever the host resyncs with NTP, and a backwards step quietly extends how long a holder believes it owns the lock. createMonotonicClock is backed by process.hrtime.bigint.

The test clock can advance inside a call

createManualClock(domain, { stepPerRead }) moves time forward on every single read. That is what makes the check-to-write gap reproducible: with a clock that only advances between calls, the check and the write always observe the same instant and the bug cannot be written down as a test.

Known limitations

Fencing catches a stale write only after the newer holder has touched the resource. If the stale holder writes first and the new holder has not written yet, the floor has not moved and the write is admitted. gate.admit(token) lets a coordinator push the new fence at reassignment time and close most of that window, but the push is itself a message that can be delayed or lost. Fencing bounds the damage from a stalled holder; it does not make the handover atomic.

Nothing here is persistent. The fence counter and the map of who holds what live in memory. A coordinator that restarts and resumes at fence 1 reissues fences the resource has already superseded, and every write from the new holder is refused. issuedFence and the startFence option exist so you can persist and resume, and FenceGate takes an initialFloor for the same reason, but doing the persisting is your job.

There is one coordinator and no consensus. No replication, no quorum, no leader election. If the coordinator process dies, its knowledge of who holds what dies with it. Put it behind something durable if you need it to survive.

There is no transport. Requests, grants, and outcomes are plain data. Sending them, retrying them, and timing them out is left to the caller, which is also why the send instant has to be captured explicitly.

clockSkewMillis is an assumption, not a measurement. It reserves room for drift you already estimated. It cannot detect that this host's clock is drifting faster than you assumed.

The send-time anchor assumes a request cannot be processed before it is sent. True across a network. Not true if you serialize a request and replay it much later: the resulting deadline would be anchored in the past. That direction is safe, since the lease is shorter than it should be, but the lease may be dead on arrival.

The replay cache holds the last 1024 acquire grants. A retry that arrives after its entry was evicted is treated as a fresh acquisition and does burn a fence.

hasExpired and wouldAdmit can be misused. They are advisory and documented as such, but nothing stops a caller from branching a write on them and reintroducing the check-to-write gap by hand.

Test

npm install
npm test   # 111 tests: overlap, send-time anchoring, fence ordering, clock domains, protocol misuse

test/overlap.test.ts builds a naive lease alongside the real one and shows both agents passing their own validity check, both writes landing, and no conflict recorded, then shows the same sequence refused and logged by the fence.

License

MIT

About

Fencing tokens and leases for agent locks, so two agents cannot both believe they hold one

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages