Understanding Deep Learning

by Simon J.D. Prince

An authoritative, accessible, and up-to-date treatment of deep learning that strikes a pragmatic middle ground between theory and practice.

Deep learning is a fast-moving field with sweeping relevance in today’s increasingly digital world. Understanding Deep Learning provides an authoritative, accessible, and up-to-date treatment of the subject, covering all the key topics along with recent advances and cutting-edge concepts. Many deep learning texts are crowded with technical details that obscure fundamentals, but Simon Prince ruthlessly curates only the most important ideas to provide a high density of critical information in an intuitive and digestible form. From machine learning basics to advanced models, each concept is presented in lay terms and then detailed precisely in mathematical form and illustrated visually. The result is a lucid, self-contained textbook suitable for anyone with a basic background in applied mathematics.

Genres: Artificial Intelligence, Computer Science, Technology, Textbooks, Technical, Science, Academic

527 pages, Hardcover

Published December 5, 2023

Reviews

[Gavin · Rating 5 out of 5]

Currently the best book on the topic: thoughtful, friendly, and comprehensive.

no-one really understands deep learning at the time of writing... Modern deep networks learn piecewise linear functions with more regions than there are atoms in the universe and can be trained with fewer data examples than model parameters. It is neither obvious that we should be able to fit these functions reliably nor that they should generalize well to new data... It is probably hard to imagine equations with these properties, and the reader should endeavor to suspend disbelief for now.

These theoretical results are intriguing but usually make unrealistic assumptions about the network structure... Overparameterization seems to be important, but theory cannot yet explain empirical fitting performance

However, sharpness is not a good criterion to predict generalization between datasets; when the labels in the CIFAR dataset are randomized (making generalization impossible), there is no commensurate decrease in the flatness of the minimum.

Current evidence suggests that overparameterization is needed for generalization — at least for the size and complexity of datasets that are currently used. There are no demonstrations of state-of-the-art performance on complex datasets where there are significantly fewer parameters than training examples. Attempts to reduce model size by pruning or distilling trained networks have not changed this picture.

Moreover, recent theory shows that there is a trade-off between the model’s Lipschitz constant and overparameterization; Bubeck & Sellke (2021) proved that in D dimensions, smooth interpolation requires D times more parameters than mere interpolation. They argue that current models for large datasets (e.g., ImageNet) aren’t overparameterized enough; increasing model capacity further may be key to improving performance...

there have been efforts to use shallower networks. Zagoruyko & Komodakis (2016) constructed shallower but wider residual neural networks and achieved similar performance to ResNet. More recently, Goyal et al. (2021) constructed a network that used parallel convolutional channels and achieved performance similar to deeper networks with only 12 layers... Nonetheless, the balance of evidence suggests that depth is critical; even the shallowest networks with good image classification performance require >10 layers. However, there is no definitive explanation for why. Three possible explanations are that (i) deep networks can represent more complex functions than shallow ones, (ii) deep networks are easier to train, and (iii) deep networks impose better inductive biases

We do not currently have any prescriptive theory that will allow us to predict the circumstances in which training and generalization will succeed or fail. We do not know the limits of learning in deep networks or whether much more efficient models are possible. We do not know if there are parameters that would generalize better within the same model.

It is oddly humble for a textbook: it presents the field as a confusing wonder. I like this man; he is trying to help.

[Free! here]

[Brian Powell · Rating 5 out of 5]

This book is about you, reader. It's about teaching you how neural networks work, and how they're used in the many wondrous modern applications that we know and love, from GPTs to computer vision. The text incudes your standard introductory fare, then develops through convolutional, residual, and graph neural nets; transformers, GANs, VAEs, and diffusion models round out the generative offerings. This text offers one of the first and few contemporary treatments of transformers and diffusion models, which are both hot out of the oven.

I say this book is about *you*, because lots of deep learning books are instead about the authors. Just because you know all about neural networks, Dr. Goodfellow, does *not* mean you need to write a book to prove it to us. Prince's text is pedagogical, and I don't mean to use that word lightly. He actually sat down and thought: yes, I know all this shit. But how do I get *you* to know all this shit? Seems like a basic consideration for anyone intent on writing a textbook, but the world ain't perfect. Prince's approach is unpretentious, never unnecessarily sophisticated, and strikingly visual. In fact, my favorite parts might just be the introductory material, the stuff many authors race over to get to the meat. After working through each chapter, we are treated to interesting end notes with tangents, elaborations, and tons of references. It's a bit like a palate cleanser after all the meat.

In summary, this book is a delight and a welcome addition to the hitherto depressing landscape of deep learning textbooks. I hope it becomes a standard.

[Nikhil Kapila · Rating 5 out of 5]

great book and illustrations to understanding deep learning :-)

Dr Prince does a great job.

[Maya Ravichandran · Rating 5 out of 5]

This book has greatly solidified my understanding of machine learning and deep learning. The explanations and figures are clear and intuitive. I particularly enjoyed chapters 5 (loss functions) and 12 (transformers). After finishing this book, I feel confident and ready to tackle anything within the field!

[رامي الرفوع · Rating 5 out of 5]

The best book on deep learning out there. Great visualizatio and emphasis on intuition.

[Alex Kash · Rating 5 out of 5]

Great explanation of complex material and the figures were very elucidating.

[Jules Arntz-Gray · Rating 5 out of 5]

For someone like myself who is in no way trained in this field the book was laid out in an amazingly coherent method allowing a general reader like myself to peek behind the curtain to see how deep learning works (at least as much as anyone can at this time).
The book is far ranging and wraps up with an excellent analysis of the ethical issues that AI raises.

[Diego Cánez · Rating 5 out of 5]

Very clear and intuitive explanations accompanied by deeply insightful and powerful figures/visualizations. So happy to see a comprehensive up-to-date DL textbook. Totally recommend it :)
PS: Reading it as the main textbook for a first course on Deep Learning at an AI MSc.

[Brenden B · Rating 5 out of 5]

I don't know how any one could give this clear and concise explanation of so many methods anything other than 5 stars!