What do you think?
Rate this book
GANs in Action: Deep learning with Generative Adversarial Networks
Summary
GANs in Action teaches you how to build and train your own Generative Adversarial Networks, one of the most important innovations in deep learning. In this book, you'll learn how to start building your own simple adversarial system as you explore the foundation of GAN architecture: the generator and discriminator networks.
Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications.
About the Technology
Generative Adversarial Networks, GANs, are an incredible AI technology capable of creating images, sound, and videos that are indistinguishable from the "real thing." By pitting two neural networks against each other--one to generate fakes and one to spot them--GANs rapidly learn to produce photo-realistic faces and other media objects. With the potential to produce stunningly realistic animations or shocking deepfakes, GANs are a huge step forward in deep learning systems.
About the Book
GANs in Action teaches you to build and train your own Generative Adversarial Networks. You'll start by creating simple generator and discriminator networks that are the foundation of GAN architecture. Then, following numerous hands-on examples, you'll train GANs to generate high-resolution images, image-to-image translation, and targeted data generation. Along the way, you'll find pro tips for making your system smart, effective, and fast.
What's inside
About the Reader
For data professionals with intermediate Python skills, and the basics of deep learning-based image processing.
About the Author
Jakub Langr is working on ML tooling and was a Computer Vision Lead at Founders Factory. Vladimir Bok is a Senior Product Manager overseeing machine learning infrastructure and research teams at a New York-based startup.
Table of Contents
PART 1 - INTRODUCTION TO GANS AND GENERATIVE MODELING PART 2 - ADVANCED TOPICS IN GANS
GANs in Action teaches you how to build and train your own Generative Adversarial Networks, one of the most important innovations in deep learning. In this book, you'll learn how to start building your own simple adversarial system as you explore the foundation of GAN architecture: the generator and discriminator networks.
Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications.
About the Technology
Generative Adversarial Networks, GANs, are an incredible AI technology capable of creating images, sound, and videos that are indistinguishable from the "real thing." By pitting two neural networks against each other--one to generate fakes and one to spot them--GANs rapidly learn to produce photo-realistic faces and other media objects. With the potential to produce stunningly realistic animations or shocking deepfakes, GANs are a huge step forward in deep learning systems.
About the Book
GANs in Action teaches you to build and train your own Generative Adversarial Networks. You'll start by creating simple generator and discriminator networks that are the foundation of GAN architecture. Then, following numerous hands-on examples, you'll train GANs to generate high-resolution images, image-to-image translation, and targeted data generation. Along the way, you'll find pro tips for making your system smart, effective, and fast.
What's inside
About the Reader
For data professionals with intermediate Python skills, and the basics of deep learning-based image processing.
About the Author
Jakub Langr is working on ML tooling and was a Computer Vision Lead at Founders Factory. Vladimir Bok is a Senior Product Manager overseeing machine learning infrastructure and research teams at a New York-based startup.
Table of Contents
PART 1 - INTRODUCTION TO GANS AND GENERATIVE MODELING PART 2 - ADVANCED TOPICS IN GANS
276 pages, Paperback
Published October 8, 2019
5 people are currently reading
120 people want to read
Ratings & Reviews
Friends & Following
Create a free account to discover what your friends think of this book!
Community Reviews
Displaying 1 - 7 of 7 reviews
August 9, 2022
I was having a problem in understanding GANS. This book with artistic analogies made me understand and also give me an outlook for practical implementation.
Interesting read!
It takes time to pick Gans, But if you want to understand how basic GANS work before diving into advanced ones. This book could be your great friend
Interesting read!
It takes time to pick Gans, But if you want to understand how basic GANS work before diving into advanced ones. This book could be your great friend
May 27, 2019
A timely and well-written book on GANs. Does a great job being both practical and linking to current literature. Absolutely worth reading.
October 27, 2020
I think I first heard (read) the term "Generative Adversarial Network" in this article on Deep Fakes that my friend Cameron sent me a few months back.
https://www.forbes.com/sites/robtoews...
At the time I was beginning to learn basic deep learning on Coursera.com, and I hoped to have a chance to learn about GANs - but the topic was beyond the scope of what Coursera had available at the time. Then over the summer, I decided to try simultaneously going to Chinese school five days a week while also working, which exhausted me and made me stop learning AI.
But in late September, Coursera released a new series of courses on exactly this topic, Chinese School was over and so after getting some paper revisions out of the way, I devoured the first two courses in a week. The third course goes live tomorrow (October 28, 2020) - and while I was waiting for it to come out I had this book priority shipped from the states and read it cover to cover. I have a second, more advanced book arriving from England on Friday.
I've learnt a lot of cool things in my life. Here is a non-exhaustive list of what I consider to be the most interesting things I have ever learnt.
1. It is possible to break a sphere into five pieces and rearrange the pieces via rigid motions into two spheres, each with the same volume as the original. Moreover, this can also be done for hyperspheres in dimension four or higher, but cannot be done with circles in dimension two or line segments in dimension one.
2. Assuming an n-dimensional universe with physics of the form F = ma, and where gravity exists, is radially symmetric and obeys a 1/r^{n-1} law, solar systems can only form if n = 3. If n>=4, the planets fly off into outer space, while for n<=2, they fall into the sun. Although we know physics is *not* of form F=ma, an analogous statement can be proven for Einstien's geodesic equations which replaced F=ma.
3. There exist events for which it is not possible to assign a probability.
4. It is always possible to "add" points in n-dimensional space by adding their individual components. However, it is only possible to *multiply* points in n-dimensional space (in such a way that multiplication has most of the properties we expect and interacts with addition in the way we expect) if n is a power of 2. Moreover, while for n=2 we keep *all* the usual properties of multiplication and addition, for n=4 we lose the property a*b=b*a, while for n=8 we additionally lose a*(b*c) = (a*b)*c.
5. It is possible to define, by way of simple mathematical formulas, shapes that have finite area but infinite perimeter, and which contain within themselves infinitely many copies of themselves. Moreover, we can easily calculate the appearance of these shapes using a computer program, and zoom in on them until they are larger than the universe. We can keep zooming in forever, and never stop finding new details. Moreover, this can also be done in three dimensions (infinite surface area but finite volume).
6. There exist different sizes of infinity. In particular, while the infinity of integers, natural numbers, and rational numbers is all the same infinity, the infinity of the real numbers is strictly bigger. Moreover, there are infinitely many infinities - so many so that the number of infinities is a bigger infinity than any infinity in the set of all infinities, creating a paradox.
7. While time travel into the past appears to be impossible, it is possible to travel into the distant future by putting yourself into a situation where time flows at a faster rate. Moreover, by tuning things appropriately you can travel forward as many years as you like - ten years, a hundred years, a thousand years, 10^10 years, you name it. The simplest way of doing this is to sit near the event horizon of a black hole - because gravity speeds up the flow of time - and then leave after an hour or so. You can tune how far you go into the future by tuning how long you hang out near the event horizon and how far above it you are.
8. The notion of determinism - i.e. that the entire future of the universe is uniquely determined by the position and velocity of every particle in the universe - actually depends on which specific differential equations govern physics. In particular, there exist second order differential equations for which a unique solution is NOT specified by the position and velocity at a given instant. Moreover, this type of equation has appeared in fluid mechanics - but as an approximate equation, rather than a true law of the universe. An experiment was done, and the experiments observed that the real behaviour of the system seemed to "hop around" between the different solutions of the differential equations.
9. Special relativity - which says that time flows at different rates depending on your situation - can be seen as inevitable once we decide that the laws of physics should obey certain symmetries which seem reasonable (i.e. that they don't vary from place to place, don't depend on your orientation or your speed) and that Maxwell's equations of electromagnetism exist. Therefore, if special-relativity seems counter-intuitive, one must ponder what a universe without the above symmetries and without Maxwell's equations would be like, and whether or not it would be more strange than special relativity.
I think that GANs are as interesting as anything in the list above. Allow me to briefly explain what they are:
To be honest, the genius of the idea astonishes me. We have this concept of an arms race - of which the most obvious examples (to me at least) have to do with geopolitics - obviously the USA/Soviet Nuclear Arms Race, but also the Naval arms race between Britain and Germany leading up to World War I etc - then you realize the concept can also be applied to biology. It's been a while since I read Dawkins so I forget the specific examples, but it doesn't matter - you have prey that develops some defence, so the predator has to evolve a way around the defence, so the prey has to make the defence stronger, and so on until you end up in a situation where both are investing this absurd amount of resources just to keep up with one another.
But the common component about all these situations is that we're using arms races as a theoretical tool to explain what we observe in nature. The genius of GANs is to turn this around and instead treat arms races as a mathematical phenomenon - the power of which we want to harness. Effectively you're saying "we want an ability X of an agent A to develop very quickly. What should we do?" and answering "let's introduce a second agent B with ability Y that counterbalances X, and induce an arms race. X and Y will both develop extremely quickly, and when we're done, we throw away agent B".
In more concrete terms, we're doing this. We have two neural networks - a generator network and a discriminator network. The job of the generator is to make fake pictures of something. In an example I have coded myself, they are handwritten digits from a giant database of handwritten digits.
The job of the discriminator is to take in an image and say whether its a real image from the database or a fake created by the discriminator. Every round, we feed the discriminator a bunch of fake images made by the generator, as well as real images from the database, but we don't tell it which is which. It makes its judgement, and then we tell it the answer. It looks at what it did wrong, and improves itself in order to do better next time. Next, we get the generator to send a bunch of fake images to the discriminator, who judges them as real or fake, and we send the judgements back to the generator (this time NOT telling the discriminator whether or not it was correct). The generator looks at which images fooled the discriminator and which ones didn't, and tries to learn from its mistakes and do better in the next round. An arms race ensues.
To me, looking at the output is positively magical. At the start, the generator is just making noise,
but by step 2000, we already have something vaguely numeral-like. By step 5000, they are definitely starting to look like numbers, and by step 23000, they are practically indistinguishable from the original data-set (it's unfortunate I can't include pictures in my review).
But here's where things get interesting. It turns out that while GANs have incredible potential, they are notoriously difficult to make work, because they are only useful if you can maintain - throughout the entire arms race - a balance of power between the discriminator and the generator. The biggest problem turns out to be the discriminator learning too quickly, being like "yeah these are obviously fake WTF it's just some random noise I'm not retarded", thereby depriving the generator of clues that it can use to improve. So you have to deliberately make sure you start off with a discriminator that is as dumb at recognizing fake images as a generator that is just making random noise. You have to make sure the two improve together at roughly the same rate, without one of them ever gaining a decisive edge. The moment this happens, the process breaks down and both of them stop improving.
This suggests that arms races have in inherent instability to them, and that while we deliberately create an artificial arms race in order to extract its power, this inherent instability makes the situation difficult to control. I find this interesting because another book I am reading - "The Narrow Corridor" - essentially argues that "liberty" is the result of an arms race between society and government and only exists when there is a balance of power between the two. If arms races are inherently unstable, this suggests that making society persist in a state of "liberty" should be difficult - societies should naturally "want to" diverge towards authoritarianism or anarchy (I'm not saying this book is correct - I don't know - but it's an interesting idea).
GANs are even more interesting in the context of "image to image translation". I haven't had a chance to code this up myself yet, but you can see a few examples here: https://phillipi.github.io/pix2pix/
https://www.forbes.com/sites/robtoews...
At the time I was beginning to learn basic deep learning on Coursera.com, and I hoped to have a chance to learn about GANs - but the topic was beyond the scope of what Coursera had available at the time. Then over the summer, I decided to try simultaneously going to Chinese school five days a week while also working, which exhausted me and made me stop learning AI.
But in late September, Coursera released a new series of courses on exactly this topic, Chinese School was over and so after getting some paper revisions out of the way, I devoured the first two courses in a week. The third course goes live tomorrow (October 28, 2020) - and while I was waiting for it to come out I had this book priority shipped from the states and read it cover to cover. I have a second, more advanced book arriving from England on Friday.
I've learnt a lot of cool things in my life. Here is a non-exhaustive list of what I consider to be the most interesting things I have ever learnt.
1. It is possible to break a sphere into five pieces and rearrange the pieces via rigid motions into two spheres, each with the same volume as the original. Moreover, this can also be done for hyperspheres in dimension four or higher, but cannot be done with circles in dimension two or line segments in dimension one.
2. Assuming an n-dimensional universe with physics of the form F = ma, and where gravity exists, is radially symmetric and obeys a 1/r^{n-1} law, solar systems can only form if n = 3. If n>=4, the planets fly off into outer space, while for n<=2, they fall into the sun. Although we know physics is *not* of form F=ma, an analogous statement can be proven for Einstien's geodesic equations which replaced F=ma.
3. There exist events for which it is not possible to assign a probability.
4. It is always possible to "add" points in n-dimensional space by adding their individual components. However, it is only possible to *multiply* points in n-dimensional space (in such a way that multiplication has most of the properties we expect and interacts with addition in the way we expect) if n is a power of 2. Moreover, while for n=2 we keep *all* the usual properties of multiplication and addition, for n=4 we lose the property a*b=b*a, while for n=8 we additionally lose a*(b*c) = (a*b)*c.
5. It is possible to define, by way of simple mathematical formulas, shapes that have finite area but infinite perimeter, and which contain within themselves infinitely many copies of themselves. Moreover, we can easily calculate the appearance of these shapes using a computer program, and zoom in on them until they are larger than the universe. We can keep zooming in forever, and never stop finding new details. Moreover, this can also be done in three dimensions (infinite surface area but finite volume).
6. There exist different sizes of infinity. In particular, while the infinity of integers, natural numbers, and rational numbers is all the same infinity, the infinity of the real numbers is strictly bigger. Moreover, there are infinitely many infinities - so many so that the number of infinities is a bigger infinity than any infinity in the set of all infinities, creating a paradox.
7. While time travel into the past appears to be impossible, it is possible to travel into the distant future by putting yourself into a situation where time flows at a faster rate. Moreover, by tuning things appropriately you can travel forward as many years as you like - ten years, a hundred years, a thousand years, 10^10 years, you name it. The simplest way of doing this is to sit near the event horizon of a black hole - because gravity speeds up the flow of time - and then leave after an hour or so. You can tune how far you go into the future by tuning how long you hang out near the event horizon and how far above it you are.
8. The notion of determinism - i.e. that the entire future of the universe is uniquely determined by the position and velocity of every particle in the universe - actually depends on which specific differential equations govern physics. In particular, there exist second order differential equations for which a unique solution is NOT specified by the position and velocity at a given instant. Moreover, this type of equation has appeared in fluid mechanics - but as an approximate equation, rather than a true law of the universe. An experiment was done, and the experiments observed that the real behaviour of the system seemed to "hop around" between the different solutions of the differential equations.
9. Special relativity - which says that time flows at different rates depending on your situation - can be seen as inevitable once we decide that the laws of physics should obey certain symmetries which seem reasonable (i.e. that they don't vary from place to place, don't depend on your orientation or your speed) and that Maxwell's equations of electromagnetism exist. Therefore, if special-relativity seems counter-intuitive, one must ponder what a universe without the above symmetries and without Maxwell's equations would be like, and whether or not it would be more strange than special relativity.
I think that GANs are as interesting as anything in the list above. Allow me to briefly explain what they are:
To be honest, the genius of the idea astonishes me. We have this concept of an arms race - of which the most obvious examples (to me at least) have to do with geopolitics - obviously the USA/Soviet Nuclear Arms Race, but also the Naval arms race between Britain and Germany leading up to World War I etc - then you realize the concept can also be applied to biology. It's been a while since I read Dawkins so I forget the specific examples, but it doesn't matter - you have prey that develops some defence, so the predator has to evolve a way around the defence, so the prey has to make the defence stronger, and so on until you end up in a situation where both are investing this absurd amount of resources just to keep up with one another.
But the common component about all these situations is that we're using arms races as a theoretical tool to explain what we observe in nature. The genius of GANs is to turn this around and instead treat arms races as a mathematical phenomenon - the power of which we want to harness. Effectively you're saying "we want an ability X of an agent A to develop very quickly. What should we do?" and answering "let's introduce a second agent B with ability Y that counterbalances X, and induce an arms race. X and Y will both develop extremely quickly, and when we're done, we throw away agent B".
In more concrete terms, we're doing this. We have two neural networks - a generator network and a discriminator network. The job of the generator is to make fake pictures of something. In an example I have coded myself, they are handwritten digits from a giant database of handwritten digits.
The job of the discriminator is to take in an image and say whether its a real image from the database or a fake created by the discriminator. Every round, we feed the discriminator a bunch of fake images made by the generator, as well as real images from the database, but we don't tell it which is which. It makes its judgement, and then we tell it the answer. It looks at what it did wrong, and improves itself in order to do better next time. Next, we get the generator to send a bunch of fake images to the discriminator, who judges them as real or fake, and we send the judgements back to the generator (this time NOT telling the discriminator whether or not it was correct). The generator looks at which images fooled the discriminator and which ones didn't, and tries to learn from its mistakes and do better in the next round. An arms race ensues.
To me, looking at the output is positively magical. At the start, the generator is just making noise,
but by step 2000, we already have something vaguely numeral-like. By step 5000, they are definitely starting to look like numbers, and by step 23000, they are practically indistinguishable from the original data-set (it's unfortunate I can't include pictures in my review).
But here's where things get interesting. It turns out that while GANs have incredible potential, they are notoriously difficult to make work, because they are only useful if you can maintain - throughout the entire arms race - a balance of power between the discriminator and the generator. The biggest problem turns out to be the discriminator learning too quickly, being like "yeah these are obviously fake WTF it's just some random noise I'm not retarded", thereby depriving the generator of clues that it can use to improve. So you have to deliberately make sure you start off with a discriminator that is as dumb at recognizing fake images as a generator that is just making random noise. You have to make sure the two improve together at roughly the same rate, without one of them ever gaining a decisive edge. The moment this happens, the process breaks down and both of them stop improving.
This suggests that arms races have in inherent instability to them, and that while we deliberately create an artificial arms race in order to extract its power, this inherent instability makes the situation difficult to control. I find this interesting because another book I am reading - "The Narrow Corridor" - essentially argues that "liberty" is the result of an arms race between society and government and only exists when there is a balance of power between the two. If arms races are inherently unstable, this suggests that making society persist in a state of "liberty" should be difficult - societies should naturally "want to" diverge towards authoritarianism or anarchy (I'm not saying this book is correct - I don't know - but it's an interesting idea).
GANs are even more interesting in the context of "image to image translation". I haven't had a chance to code this up myself yet, but you can see a few examples here: https://phillipi.github.io/pix2pix/
August 15, 2022
The matter is quite new but is super interesting, Gans are a bit like other machine learning models but are trained to distinguish test data from random data, I would resume like that.
So a model creates a discriminator and a generator. The discriminator tries to distinguish data that is really test data and the generator tries to fool the discriminator generating random data that looks the most possibly statistically close to the test data. And it's quite crazy but this can actually create stuff like new images, new songs and resemble some creativity, super interesting topic
So a model creates a discriminator and a generator. The discriminator tries to distinguish data that is really test data and the generator tries to fool the discriminator generating random data that looks the most possibly statistically close to the test data. And it's quite crazy but this can actually create stuff like new images, new songs and resemble some creativity, super interesting topic
June 19, 2025
A nice book to have a brief unified summary of what's going on with generative AI. Balanced between practicality and theoretical aspects, with slight nuance for applications. I found it nice to be the starting point towards understanding other models like diffusion and flow matching.
The structure makes it easy to follow and compare the improvement with each architecture. Especially fond of the introduction of adversarial examples that can extend more than just testing GANs, but also other ML models.
The structure makes it easy to follow and compare the improvement with each architecture. Especially fond of the introduction of adversarial examples that can extend more than just testing GANs, but also other ML models.
October 12, 2023
So much out-of-date code! Even though you can check corrected code on the author's github, it was corrected 4 years ago and there are plenty of errors that are not resolved. If you want to learn about GANs in 2023, better find a newer book.
December 20, 2019
Requires decent knowledge about Deep learning and ConvNets
Displaying 1 - 7 of 7 reviews