Why Shared-Nothing Scales on One Machine

Redis made the single-threaded event loop famous for a reason. It is simple, predictable, and very fast for work that fits in one core. That does not mean one thread is the only serious option. It means shared state is the problem, not threads.

The single-threaded event loop

A store such as Redis keeps one event loop responsible for client I/O and request execution. Socket multiplexing lets that loop watch many connections without blocking on one slow client.

• The kernel reports sockets that are ready through epoll, kqueue, or select.
• The event loop runs a bounded amount of work, then checks for more I/O.
• A read or recv call does not stall the process when no data is available. The loop moves on.

That works because in-memory operations are cheap and network waits are not. One core can process a great deal of queued work before the next request needs attention. The design gets awkward when a single core becomes the limit, or when every path starts arguing over shared data.

Shard per core

The shared-nothing alternative gives each core a thread and a private shard of data. A request goes to the core that owns its key. That core runs the operation. No global lock is required because nothing else is allowed to mutate that shard.

The important part is ownership. More threads without ownership are just more places to wait.

Network layer

A small set of I/O threads handles sockets. Each connection runs in a coroutine or fiber, so waiting for a socket yields to another connection instead of parking an OS thread.

• Non-blocking I/O keeps a waiting connection from wasting a thread.
• User-space coroutine switches cost far less than kernel thread switches.
• A lock-free queue passes work to the owning storage core.
• Network threads can run at lower priority so storage work gets the CPU first.

This layer has failure modes worth respecting. One blocking call can stall a whole I/O thread. Too few I/O threads can add latency. Async errors and cancellation also get less pleasant once the easy synchronous path is gone. None of that makes the design wrong. It just means the scheduler is part of the system now.

Storage layer

Each storage thread owns its shard and its operation queue. A network coroutine sends it work, then waits on a future or promise for the result. Cross-shard work is the exception, and it should stay that way.

• Local operations need no lock because the shard has one writer.
• Data locality improves cache behavior and gives more predictable latency.
• Adding cores adds shards, so throughput can grow without making every request fight a central mutex.
• A single-producer, single-consumer queue is enough when the ownership model stays honest.

The trade-off is real. A hot shard can still become the bottleneck. A broken SPSC assumption becomes a race. Cross-shard transactions need coordination, and coordination brings back some of the cost this model avoided. Shared-nothing is not magic. It is a way to make the expensive cases visible.

Anti-caching

Memory is finite. Anti-caching moves cold records to disk so hot records can stay in RAM. Keep indexes and enough metadata in memory to find a record, then fetch the full value asynchronously on a miss.

• Evict data based on memory pressure and access frequency.
• Keep lookup metadata resident so disk-backed records are still addressable.
• Fetch cold data without blocking the core that owns the request.
• Return a retry signal or placeholder while the record comes back.

This buys a larger working set without pretending RAM is infinite. It also adds a second latency class and more failure handling. That is acceptable when the alternative is fitting the whole dataset in memory because the architecture gave up too early.

The point

A single event loop is excellent when one core is enough. Shard per core is better when independent data can stay independent. The useful rule is simple: give data an owner, avoid shared mutation, and make coordination expensive enough that nobody adds it casually.