What do you think?
Rate this book
Building Big Data Pipelines with Apache Beam: Use a single programming model for both batch and stream data processing
Implement, run, operate, and test data processing pipelines using Apache Beam
Key FeaturesUnderstand how to improve usability and productivity when implementing Beam pipelinesLearn how to use stateful processing to implement complex use cases using Apache BeamImplement, test, and run Apache Beam pipelines with the help of expert tips and techniquesBook DescriptionApache Beam is an open source unified programming model for implementing and executing data processing pipelines, including Extract, Transform, and Load (ETL), batch, and stream processing.
This book will help you to confidently build data processing pipelines with Apache Beam. You'll start with an overview of Apache Beam and understand how to use it to implement basic pipelines. You'll also learn how to test and run the pipelines efficiently. As you progress, you'll explore how to structure your code for reusability and also use various Domain Specific Languages (DSLs). Later chapters will show you how to use schemas and query your data using (streaming) SQL. Finally, you'll understand advanced Apache Beam concepts, such as implementing your own I/O connectors.
By the end of this book, you'll have gained a deep understanding of the Apache Beam model and be able to apply it to solve problems.
What you will learnUnderstand the core concepts and architecture of Apache BeamImplement stateless and stateful data processing pipelinesUse state and timers for processing real-time event processingStructure your code for reusabilityUse streaming SQL to process real-time data for increasing productivity and data accessibilityRun a pipeline using a portable runner and implement data processing using the Apache Beam Python SDKImplement Apache Beam I/O connectors using the Splittable DoFn APIWho this book is forThis book is for data engineers, data scientists, and data analysts who want to learn how Apache Beam works. Intermediate-level knowledge of the Java programming language is assumed.
Table of ContentsIntroduction to Data Processing with Apache BeamImplementing, Testing, and Deploying Basic PipelinesImplementing Pipelines Using Stateful ProcessingStructuring Code for ReusabilityUsing SQL for Pipeline ImplementationUsing Your Preferred Language with PortabilityExtending Apache Beam's I/O Connectors Understanding How Runners Execute Pipelines
Key FeaturesUnderstand how to improve usability and productivity when implementing Beam pipelinesLearn how to use stateful processing to implement complex use cases using Apache BeamImplement, test, and run Apache Beam pipelines with the help of expert tips and techniquesBook DescriptionApache Beam is an open source unified programming model for implementing and executing data processing pipelines, including Extract, Transform, and Load (ETL), batch, and stream processing.
This book will help you to confidently build data processing pipelines with Apache Beam. You'll start with an overview of Apache Beam and understand how to use it to implement basic pipelines. You'll also learn how to test and run the pipelines efficiently. As you progress, you'll explore how to structure your code for reusability and also use various Domain Specific Languages (DSLs). Later chapters will show you how to use schemas and query your data using (streaming) SQL. Finally, you'll understand advanced Apache Beam concepts, such as implementing your own I/O connectors.
By the end of this book, you'll have gained a deep understanding of the Apache Beam model and be able to apply it to solve problems.
What you will learnUnderstand the core concepts and architecture of Apache BeamImplement stateless and stateful data processing pipelinesUse state and timers for processing real-time event processingStructure your code for reusabilityUse streaming SQL to process real-time data for increasing productivity and data accessibilityRun a pipeline using a portable runner and implement data processing using the Apache Beam Python SDKImplement Apache Beam I/O connectors using the Splittable DoFn APIWho this book is forThis book is for data engineers, data scientists, and data analysts who want to learn how Apache Beam works. Intermediate-level knowledge of the Java programming language is assumed.
Table of ContentsIntroduction to Data Processing with Apache BeamImplementing, Testing, and Deploying Basic PipelinesImplementing Pipelines Using Stateful ProcessingStructuring Code for ReusabilityUsing SQL for Pipeline ImplementationUsing Your Preferred Language with PortabilityExtending Apache Beam's I/O Connectors Understanding How Runners Execute Pipelines
342 pages, Kindle Edition
Published January 21, 2022
6 people are currently reading
11 people want to read
Ratings & Reviews
Friends & Following
Create a free account to discover what your friends think of this book!
Community Reviews
Displaying 1 of 1 review
February 22, 2025
Beginning was decent, formatting not good
I got this on Kindle format and in the beginning I learned enough to get me started on my project. When it got into the windows and stuff I got pretty confused and didn’t really dive into it since I’m not planning to use windows, joins, or Python. Also I didn’t find the writing especially clear. The formatting was also not ideal on kindle, neither on my phone, tablet, nor desktop. I don’t really recommend this one. Disappointed because I haven’t found better books on the topic either, and there is so much to teach about it. Luckily learning bean online is pretty easy. Somehow it seems the explanations of how to use Spark are much better, bot for realtime use cases beam is much more appropriate.
I got this on Kindle format and in the beginning I learned enough to get me started on my project. When it got into the windows and stuff I got pretty confused and didn’t really dive into it since I’m not planning to use windows, joins, or Python. Also I didn’t find the writing especially clear. The formatting was also not ideal on kindle, neither on my phone, tablet, nor desktop. I don’t really recommend this one. Disappointed because I haven’t found better books on the topic either, and there is so much to teach about it. Luckily learning bean online is pretty easy. Somehow it seems the explanations of how to use Spark are much better, bot for realtime use cases beam is much more appropriate.
Displaying 1 of 1 review