What do you think?
Rate this book
Designing Event-Driven Systems
Many forces affect software today: larger datasets, geographical disparities, complex company structures, and the growing need to be fast and nimble in the face of change.
Proven approaches such as service-oriented and event-driven architectures are joined by newer techniques such as microservices, reactive architectures, DevOps, and stream processing. Many of these patterns are successful by themselves, but as this practical ebook demonstrates, they provide a more holistic and compelling approach when applied together.
Author Ben Stopford explains how service-based architectures and stream processing tools such as Apache Kafka® can help you build business-critical systems.
* Learn why streaming beats request-response based architectures in complex, contemporary use cases
* Understand why replayable logs such as Kafka provide a backbone for both service communication and shared datasets
* Explore how event collaboration and event sourcing patterns increase safety and recoverability with functional, event-driven approaches
* Apply patterns including Event Sourcing and CQRS, and how to build multi-team systems with microservices and SOA using patterns such as “inside out databases” and “event streams as a source of truth”
* Build service ecosystems that blend event-driven and request-driven interfaces using a replayable log and Kafka's Streams API
* Scale beyond individual teams into larger, department- and company-sized architectures, using event streams as a source of truth
Proven approaches such as service-oriented and event-driven architectures are joined by newer techniques such as microservices, reactive architectures, DevOps, and stream processing. Many of these patterns are successful by themselves, but as this practical ebook demonstrates, they provide a more holistic and compelling approach when applied together.
Author Ben Stopford explains how service-based architectures and stream processing tools such as Apache Kafka® can help you build business-critical systems.
* Learn why streaming beats request-response based architectures in complex, contemporary use cases
* Understand why replayable logs such as Kafka provide a backbone for both service communication and shared datasets
* Explore how event collaboration and event sourcing patterns increase safety and recoverability with functional, event-driven approaches
* Apply patterns including Event Sourcing and CQRS, and how to build multi-team systems with microservices and SOA using patterns such as “inside out databases” and “event streams as a source of truth”
* Build service ecosystems that blend event-driven and request-driven interfaces using a replayable log and Kafka's Streams API
* Scale beyond individual teams into larger, department- and company-sized architectures, using event streams as a source of truth
166 pages, ebook
Published January 1, 2018
Ratings & Reviews
Friends & Following
Create a free account to discover what your friends think of this book!
Community Reviews
Displaying 1 - 30 of 35 reviews
May 29, 2020
I've found exactly what I have been looking for, a very good intro to designing Event-Driven systems with described differences between Event-Driven, Event Sourced, Event Stream, and Event Stateful Stream. Also, the nice thing here is that, even if the book is Kafka focused, this knowledge can be useful with Kinesis, Event Store etc.
The biggest minus here is lack of Event design, the book is mostly focused around the entire ecosystem of application, but there Is almost no information about the smallest bricks which are events...
The biggest minus here is lack of Event design, the book is mostly focused around the entire ecosystem of application, but there Is almost no information about the smallest bricks which are events...
August 5, 2020
A really good book that explores several patterns for designing event-driven systems with Kafka.
It lays down very interesting ideas such as:
- Data in the outside
- Single writer principle
- Upcast and Downcast for Schema Evolution
- Event-sourcing and CQRS
in the context of Apache Kafka.
It is amazing how much can be build using only Kafka and Kafka Streams and how much can be plugged into it, making it easier to have auxiliary storages like Solr or Cassandra, for example.
The final chapter where a full-app is described demonstrate the power
It lays down very interesting ideas such as:
- Data in the outside
- Single writer principle
- Upcast and Downcast for Schema Evolution
- Event-sourcing and CQRS
in the context of Apache Kafka.
It is amazing how much can be build using only Kafka and Kafka Streams and how much can be plugged into it, making it easier to have auxiliary storages like Solr or Cassandra, for example.
The final chapter where a full-app is described demonstrate the power
June 5, 2021
A guide on building event-driven applications on top of Kafka. I was curious to see how it would relate to messaging as an approach to software development, as described in Enterprise Integration Patterns. It turns out this book is more about event sourcing, an advanced flavor of messaging where the message broker holds messages indefinitely and becomes the data store and the source of truth in the system.
The book is heavily biased towards Kafka and Confluent, to the point that it is more about praising Kafka specifically than designing systems in general. It does a good job of describing the features and capabilities of Kafka, but it is light on concrete examples. There is a big gap between the few illustrative lines of KSQL in the book and the complexity of the code on GitHub. Kafka has libraries for Java and native connectors for other languages, but some of the higher functions, like materialized views and state stores, are only accessible from Java and you need a sidecar setup to run them besides your non-Java code. There is no example of that.
I was surprised to learn that Kafka was invented at LinkedIn. According to the book, Amazon popularized SOA, then Netflix pushed SOA to its limits to give us microservices, and LinkedIn embraced microservices and created Kafka as a tool to help it, manage them.
Kafka was designed from the beginning for decentralized processing and very high parallelism. This explains the constant focus on partitions every time someone tries to explain Kafka. It is meant for managing streams and transforming data in flight, much more than a simple message bus used for inter-process communication. The book describes installations with 100s of brokers.
I liked the parts about non-persistent services that recreate their state from the stream when they (re)start. It is a very different way to look at services. The author refers to "the databased turned inside out."
The book introduced me to complex patterns like CRDT to manage asynchronous updates in a large system, where messages may get delayed and arrive out of order. Kafka has its own concept of transactions and the book briefly touches on sagas and dead letter topics for error handling.
The example system in the last chapter shows many concepts and how they build on each other to fulfill the application. I liked how they rekey topics to partition them on one field or another so matching partitions can be joined together in one node without the need for complex IPC. But the book doesn't show concretely how to set this up. It is left as an exercise for the reader.
You can download a free copy in PDF format, thanks to Confluent.
The book is heavily biased towards Kafka and Confluent, to the point that it is more about praising Kafka specifically than designing systems in general. It does a good job of describing the features and capabilities of Kafka, but it is light on concrete examples. There is a big gap between the few illustrative lines of KSQL in the book and the complexity of the code on GitHub. Kafka has libraries for Java and native connectors for other languages, but some of the higher functions, like materialized views and state stores, are only accessible from Java and you need a sidecar setup to run them besides your non-Java code. There is no example of that.
I was surprised to learn that Kafka was invented at LinkedIn. According to the book, Amazon popularized SOA, then Netflix pushed SOA to its limits to give us microservices, and LinkedIn embraced microservices and created Kafka as a tool to help it, manage them.
Kafka was designed from the beginning for decentralized processing and very high parallelism. This explains the constant focus on partitions every time someone tries to explain Kafka. It is meant for managing streams and transforming data in flight, much more than a simple message bus used for inter-process communication. The book describes installations with 100s of brokers.
I liked the parts about non-persistent services that recreate their state from the stream when they (re)start. It is a very different way to look at services. The author refers to "the databased turned inside out."
when a new node starts, it must load all stateful components p.52
a single repository of event data is used to feed many views, and each view can flex with the needs of that particular team. p. 91
The book introduced me to complex patterns like CRDT to manage asynchronous updates in a large system, where messages may get delayed and arrive out of order. Kafka has its own concept of transactions and the book briefly touches on sagas and dead letter topics for error handling.
The example system in the last chapter shows many concepts and how they build on each other to fulfill the application. I liked how they rekey topics to partition them on one field or another so matching partitions can be joined together in one node without the need for complex IPC. But the book doesn't show concretely how to set this up. It is left as an exercise for the reader.
You can download a free copy in PDF format, thanks to Confluent.
August 7, 2021
Книгу стоило назвать иначе - "возможности Кафки о которых вы вероятно еще не знали".
Вся книга в целом это рассказ как можно использовать Кафку и в каких сценариях. Про работу самих микросервисов и их дизайне довольно мало информации, ровно, как и о слабых сторонах Кафки и как НЕ нужно её использовать.
Поэтому хорошая книга для ознакомления с интересным инструментом, но примеры в книге довольно синтетические, поэтому при переносе решений на реальную жизнь нужно быть крайне осторожным.
Вся книга в целом это рассказ как можно использовать Кафку и в каких сценариях. Про работу самих микросервисов и их дизайне довольно мало информации, ровно, как и о слабых сторонах Кафки и как НЕ нужно её использовать.
Поэтому хорошая книга для ознакомления с интересным инструментом, но примеры в книге довольно синтетические, поэтому при переносе решений на реальную жизнь нужно быть крайне осторожным.
April 15, 2020
I found what I was looking for. Very good definitions and descriptions of patterns, of course Kafka oriented, but for being such a short book I still think is pretty concise and compact, with accurate references.
Read
December 3, 2018The book was given to me from its author in a convention (cofeve in colloquial English) which does not allow me to put a start-rating.
With this aside this book does one thing and does this really well, which is to explain different strategies and patterns and give a good insight on how to implement them using Apache Kafka. Book contains many good ideas and description of different concepts, has lots of stuff that I have already stolen, although a newcomer to the space. Diagrams are very nice diagrams with a soft choice of colours which makes them almost cute :) for a technical book (note: Confluent, the company that author has founded site uses similar hues on their blog posts). If I could praise though the book for one thing it would be the absence of "fluff" which is so common in technical books these days, second one is that it promotes the company in a nice subtle way by giving to the reader lots of value, if the reader wants then to go with the experts then ... they know what to do next.
With this aside this book does one thing and does this really well, which is to explain different strategies and patterns and give a good insight on how to implement them using Apache Kafka. Book contains many good ideas and description of different concepts, has lots of stuff that I have already stolen, although a newcomer to the space. Diagrams are very nice diagrams with a soft choice of colours which makes them almost cute :) for a technical book (note: Confluent, the company that author has founded site uses similar hues on their blog posts). If I could praise though the book for one thing it would be the absence of "fluff" which is so common in technical books these days, second one is that it promotes the company in a nice subtle way by giving to the reader lots of value, if the reader wants then to go with the experts then ... they know what to do next.
February 3, 2020
Not the best intro to the subject I'd say. There are some clearly explained parts, but in general the book is quite unstructured, explanations how great Kafka is are mixed with implementation details and counter examples, so the result is difficult for understanding mess.
August 13, 2019
Good introduction/overview to Kafka and streaming services.
May 23, 2021
I think this book is quite enlightening for me as I did not have much experience with building an event-driven system. It focuses on the Kafka framework which highlights the pros of building a streaming based system as opposed to a traditional layered based architecture. I am sure many of the software engineer were groomed to adopt a three-layers software architecture design in building a software application so this new concept might sound foreign.
One of the benefits of using Kafka would be the asynchronicity of streaming data between micro-services housed within the system. Using the concept of Kafka consumer and producer, you do not need to wait for the data to be fetched, a micro-service could subscribe to a topic and continue to stream for data. There are many benefits of using Kafka but of course it depends on your use cases of the system but reading this book does excite me to try and build a system based on Kafka. I learnt something new.
One of the benefits of using Kafka would be the asynchronicity of streaming data between micro-services housed within the system. Using the concept of Kafka consumer and producer, you do not need to wait for the data to be fetched, a micro-service could subscribe to a topic and continue to stream for data. There are many benefits of using Kafka but of course it depends on your use cases of the system but reading this book does excite me to try and build a system based on Kafka. I learnt something new.
September 6, 2021
It was a good read. At the point of entry I didn't know much about event driven architectures, streaming, tools, best practices pros or cons. The book gave a me a nice introduction to the topic in the context of Apache Kafka.
It needs to be pointed out that the book heavily invest and talks about said tool. After reading the book I looked back to the title and was surprised to noticed that it wasn't a Apache Kafka title book.
Prior knowledge of the topic is not necessary, however it is good to understand the "traditional" relational systems to see the merits of such systems.
I will definitely come back to the book when I get to use this the specific Kafka tooling as the book seems to be a good reference point for that.
It needs to be pointed out that the book heavily invest and talks about said tool. After reading the book I looked back to the title and was surprised to noticed that it wasn't a Apache Kafka title book.
Prior knowledge of the topic is not necessary, however it is good to understand the "traditional" relational systems to see the merits of such systems.
I will definitely come back to the book when I get to use this the specific Kafka tooling as the book seems to be a good reference point for that.
May 18, 2021
Pros:
* I feel more confident and excited about working with Kafka now.
* I learned the basic Kafka concepts. For example, I know that when we want to scale out a service horizontally, we have to think about the partition key. I may not know how to do that, but at least I know the nouns to look for when it comes up.
Cons:
* I wish the tradeoffs discussions had more examples. What's a case you've seen in the real world where rebuilding a service-resident dataset from scratch was too expensive? What was the workaround?
* Not everyone will learn well from this format. There are code samples, but no exercises, no quizzes, no questions to ponder. This is a lecture, not a whole course.
* I feel more confident and excited about working with Kafka now.
* I learned the basic Kafka concepts. For example, I know that when we want to scale out a service horizontally, we have to think about the partition key. I may not know how to do that, but at least I know the nouns to look for when it comes up.
Cons:
* I wish the tradeoffs discussions had more examples. What's a case you've seen in the real world where rebuilding a service-resident dataset from scratch was too expensive? What was the workaround?
* Not everyone will learn well from this format. There are code samples, but no exercises, no quizzes, no questions to ponder. This is a lecture, not a whole course.
December 22, 2022
Excellent starter book for getting started on Streaming, Asynchronous Distributed architecture on Kafka. Its heavy on Kafka which is not an issue for me.
Certain chapters are easy to follow and provide valuable knowledge, some are tough to follow. The chapters on Schema, Partitioning are great. Good introduction on Event Sourcing, CQRS etc but will need to reread these chapters or use other sources.
I think the next thing is to actually implement the Inventory Service example used in the book to better understand the power of Kafka.
I cant complain since the book is valuable and free!!
Certain chapters are easy to follow and provide valuable knowledge, some are tough to follow. The chapters on Schema, Partitioning are great. Good introduction on Event Sourcing, CQRS etc but will need to reread these chapters or use other sources.
I think the next thing is to actually implement the Inventory Service example used in the book to better understand the power of Kafka.
I cant complain since the book is valuable and free!!
October 11, 2021
This is a very short and “to-the-point” book. I loved it because it doesn’t go down the rabbit hole into the nitty gritty details of how to implement the patterns they mention with Kafka. It focuses more on the architecture and design thinking and mentions which tools can be used, if you need more information on “how to do it” you’ll have to go to a Kafka technical guide and read the corresponding topics (no pun intended), although some code samples are provided, I think having a Kafka technical book would help in implementation.
Read
January 7, 2023I'm not giving this a star rating; I was kind of skimming this for some conceptual ideas and am probably not the real intended audience.
Anyhow, to me this book did a pretty good job of laying out when and why a Apache Kafka-style event stream architecture is helpful. You can't really pick this up by reading a description of Kafka itself. The author walked through some meaningful examples to help the concepts hit home.
There is a bit of jargon here and there that is kind of specific to distributed online systems development, but that's not unexpected.
Anyhow, to me this book did a pretty good job of laying out when and why a Apache Kafka-style event stream architecture is helpful. You can't really pick this up by reading a description of Kafka itself. The author walked through some meaningful examples to help the concepts hit home.
There is a bit of jargon here and there that is kind of specific to distributed online systems development, but that's not unexpected.
August 5, 2021
A very interesting topic and a lot of great ideas. I have decided to read the ebook to dive deeper after enjoying the author's blog post. Unfortunately I found the post to be more detailed and better structured than the book, which could benefit from more effort to become a good book. For now I suggest to read the series of the blog posts instead.
January 5, 2019
I deal with Kafka and related ecosystem in production everyday and hence this book didn't really have anything new to offer to me.
Could be useful if you are starting up with event driven systems. However, it feels like the book was compiled in a haste as they seem to escape from each concept rather quickly.
Could be useful if you are starting up with event driven systems. However, it feels like the book was compiled in a haste as they seem to escape from each concept rather quickly.
February 2, 2020
A good starting point if you haven't built event-driven systems before or plan on using Kafka. It does a good job of getting you ready by exposing you to various possibilities and how they could be used for different workflows.
One drawback I observed is that some topics were a bit shallow than I would have expected.
One drawback I observed is that some topics were a bit shallow than I would have expected.
March 9, 2022
There are some good ideas in these book but it felt a bit disorganised. That might be my impression due to my limited knowledge of Kafka. Also it is very focused on Kafka for a book with such a generic title.
I got it to get some ideas of potential use cases for Kafka and the book is good for this. Also the references are pretty good and supplement the book well to get a more holistic view.
I got it to get some ideas of potential use cases for Kafka and the book is good for this. Also the references are pretty good and supplement the book well to get a more holistic view.
December 3, 2022
The book has an easy-to-understand language and comprehensible flow. It covers most of the aspects that you should consider, if you are planning to adopt the Event-Driven Systems methodology. If you have some thoughts on the topic, but can't find the technical foundation to ground on, this book can definitely help you. I totally recommend it.
May 27, 2021
I'd title it "Designing Event driven systems with Kafka". Do not expect to get any structured knowledge on event system design, but if you already have one, then you will clearly see how to apply Kafka.
September 13, 2021
Well, the book is quite enjoyable with man ideas how to use Kafka and how to tackle some problems that are known from my own experience with messaging or rather eventing.
What I usually don't understand why people want to solve every problem with Kafka.
What I usually don't understand why people want to solve every problem with Kafka.
February 15, 2022
This book gives you a great overview (overview being the keyword here) of what could be possible, how to implement different use-cases and how to handle different edge cases in an event-driven system using Apache Kafka.
December 26, 2019
A very good overview of how to conceive event driven systems using Apache Kaka. It is a short book with high-level architecture discussions, not including code samples or more detailed use cases
October 16, 2020
Loved this book
October 24, 2020
The subtitle of Kafka needs to be in the title, but the generic parts were interesting.
May 25, 2021
Very worthwhile I'd you're coming from a microservice perspective. Only knock is that it is very Confluent-centric.
July 27, 2021
Overall a pleasant read and nice introduction to a lot of the fundamentals of event driven systems, a 3.5/5 stars. I put 4/5 stars because it was much better than I expected.
January 3, 2022
Awesome. A lot of patterns and insights
Read
April 20, 2022guy really likes kafka, fair enough
May 15, 2022
Pretty good book for those who want to use kaka understand how a system works with it. I read it too early on, might have to revisit once I'm using it in a real project.
Displaying 1 - 30 of 35 reviews