Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems

by Martin Kleppmann

In this chapter we looked at the issue of replication. Replication can serve several purposes: High availability Keeping the system running, even when one machine (or several machines, or an entire datacenter) goes down Disconnected operation Allowing an application to continue working when there is a network interruption Latency Placing data geographically close to users, so that users can interact with it faster Scalability Being able to handle a higher volume of reads than a single machine could handle, by performing reads on replicas Despite being a simple goal—keeping a copy of the same data on several machines—replication turns out to be a remarkably tricky problem. It requires carefully thinking about concurrency and about all the things that can go wrong, and dealing with the consequences of those faults. At a minimum, we need to deal with unavailable nodes and network interruptions (and that’s not even considering the more insidious kinds of fault, such as silent data corruption due to software bugs). We discussed three main approaches to replication: Single-leader replication Clients send all writes to a single node (the leader), which sends a stream of data change events to the other replicas (followers). Reads can be performed on any replica, but reads from followers might be stale. Multi-leader replication Clients send each write to one of several leader nodes, any of which can accept writes. The leaders send streams of data change events to each other and to ...