Distributed Systems Theory

Distributed Locking

A distributed lock lets multiple processes coordinate exclusive access to a shared resource across machines — but every implementation must answer the uncomfortable question: what happens if the lock holder pauses (GC, network stall) past the lock's expiry and keeps acting as if it still holds it?

The simplest form — a row/key in a database with a TTL, or a Redis SET key value NX PX ttl — is enough for advisory, best-effort locking
A lock's TTL exists to survive holder crashes, but that same TTL means a holder that pauses (GC, VM stall) past it can resume unaware it lost the lock
Fencing tokens — a monotonically increasing number issued with each lock grant — let the protected resource itself reject stale writes from an expired holder, closing the gap TTLs alone leave open
Redlock (multi-instance Redis locking) is popular but contested — Martin Kleppmann's critique argues it provides no correctness guarantee under clock/GC pauses without fencing tokens
For real correctness guarantees, a lock built on a consensus system (ZooKeeper, etcd) that issues fencing tokens is the safer foundation than a bare TTL key

A single-process synchronized lock is trivial because the JVM guarantees only one thread holds it at a time, and a crashed thread releases it automatically. Neither guarantee survives crossing a network: the "lock" is now just a fact recorded somewhere (a database row, a Redis key, a ZooKeeper znode), and there is no way to force a remote process to stop acting on the belief that it holds it.

A minimal Redis-based lock

boolean tryLock(Jedis redis, String key, String owner, long ttlMs) {
    // NX: only set if absent. PX: expire after ttlMs. Both atomic in one command.
    String result = redis.set(key, owner, SetParams.setParams().nx().px(ttlMs));
    return "OK".equals(result);
}

boolean unlock(Jedis redis, String key, String owner) {
    // Only delete if we still own it — a Lua script keeps the check+delete atomic.
    String script = "if redis.call('get', KEYS[1]) == ARGV[1] then " +
                     "return redis.call('del', KEYS[1]) else return 0 end";
    return (Long) redis.eval(script, List.of(key), List.of(owner)) == 1;
}

The gap a bare TTL lock cannot close

Without a fencing token, Storage cannot tell Process A's write is stale

A fencing token fixes this by making the protected resource the enforcement point, not the lock service: every time the lock is granted, an incrementing number is issued alongside it, and the resource rejects any write tagged with a token lower than the highest one it has already seen. Process A resuming after its pause still sends its old, now-stale token — the storage layer rejects it, no matter how confidently Process A believes it holds the lock.

Locking approaches
Approach	Guarantee	Failure mode without fencing
Database row + TTL	Advisory only	Expired holder can still write after TTL if paused
Redis SET NX PX (single instance)	Advisory, single point of failure	Same, plus the Redis instance itself can fail
Redlock (multi-instance Redis)	Advisory, majority-based	Contested — same underlying gap without tokens
ZooKeeper/etcd (consensus-backed) + fencing token	Strong, enforceable at the resource	None, if the resource actually checks the token