Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems

by Martin Kleppmann

Designing Data-Intensive Applications

Designing Data-Intensive Applications

book title

We call an application data-intensive if data is its primary challenge—the quantity of data, the complexity of data, or the speed at which it is changing—as opposed to compute-intensive, where CPU cycles are the bottleneck.

We call an application data-intensive if data is its primary challenge—the quantity of data, the complexity of data, or the speed at which it is changing—as opposed to compute-intensive, where CPU cycles are the bottleneck.

meaning of data intensive

Part I. Foundations of Data Systems

Part I. Foundations of Data Systems

part 1

Chapter 1. Reliable, Scalable, and Maintainable Applications

Chapter 1. Reliable, Scalable, and Maintainable Applications

chap 1

A data-intensive application is typically built from standard building blocks that provide commonly needed functionality.

Store data so that they, or another application, can find it again later (databases) Remember the result of an expensive operation, to speed up reads (caches) Allow users to search data by keyword or filter it in various ways (search indexes) Send a message to another process, to be handled asynchronously (stream processing) Periodically crunch a large amount of accumulated data (batch processing)

Store data so that they, or another application, can find it again later (databases) Remember the result of an expensive operation, to speed up reads (caches) Allow users to search data by keyword or filter it in various ways (search indexes) Send a message to another process, to be handled asynchronously (stream processing) Periodically crunch a large amount of accumulated data (batch processing)

building blocks of data intensive applications

Reliability The system should continue to work correctly (performing the correct function at the desired level of performance) even in the face of adversity (hardware or software faults, and even human error). See “Reliability”.

Reliability The system should continue to work correctly (performing the correct function at the desired level of performance) even in the face of adversity (hardware or software faults, and even human error). See “Reliability”.

def of reliability

Scalability As the system grows (in data volume, traffic volume, or complexity), there should be reasonable ways of dealing w...

Scalability As the system grows (in data volume, traffic volume, or complexity), there should be reasonable ways of dealing w...

def of scalability

Maintainability Over time, many different people will work on the system (engineering and operations, both maintaining current behavior and adapting the system to new use cases), and they should all be ...

Maintainability Over time, many different people will work on the system (engineering and operations, both maintaining current behavior and adapting the system to new use cases), and they should all be ...

def of maintainability

The application performs the function that the user expected. It can tolerate the user making mistakes or using the software in unexpected ways. Its performance is good enough for the required use case, under the expected load and data volume. The system prevents any unauthorized access and abuse.

The application performs the function that the user expected. It can tolerate the user making mistakes or using the software in unexpected ways. Its performance is good enough for the required use case, under the expected load and data volume. The system prevents any unauthorized access and abuse.

reliability expectations

“continuing to work correctly, even when things go wrong.”