Introduction To Conformal Prediction With Python: A Short Guide For Quantifying Uncertainty Of Machine Learning Models

by Christoph Molnar

Introduction To Conformal Prediction With Python is the quickest way to learn an easy-to-use and very general technique for uncertainty quantification. " This concise book is accessible, lucid, and full of helpful code snippets. It explains the mathematical ideas with clarity and provides the reader with practical examples that illustrate the essence of conformal prediction, a powerful idea for uncertainty quantification. "
– Junaid Butt, Research Software Engineer, IBM Research "Modern statistics can be a difficult topic, but Christoph has managed to make it feel easy, practical, and fun! Reading this book is a great first step towards gaining mastery of conformal prediction and related topics."
– Anastasios Angelopoulos, Researcher at the University of California, Berkeley A prerequisite for trust in machine learning is uncertainty quantification. Without it, an accurate prediction and a wild guess look the same. Yet many machine learning models come without uncertainty quantification. And while there are many approaches to uncertainty – from Bayesian posteriors to bootstrapping – we have no guarantees that these approaches will perform well on new data.

"I really enjoyed reading the book. The data science and machine learning community needs more people like Christoph Molnar who are able to translate emerging breakthrough research into digestible concepts. I can see this book becoming a key piece in accelerating the rate of adoption of conformal ML."
– Guilherme Del Nero Maia, Principal Data Science at Jabil At first glance conformal prediction seems like yet another contender. But conformal prediction can work in combination with any other uncertainty approach and has many advantages that make it stand Sound good? Then this is the right book for you to learn about this versatile, easy-to-use yet powerful tool for taming the uncertainty of your models. " Great practical examples, easy explanations, and highly entertaining. If you want to learn about the best Uncertainty Quantification framework for the 21st century, don't miss out on this book. "
– Valeriy Manokhin, Managing Director at Open Predictive Technologies & Creator of Awesome Conformal Prediction This With the knowledge in this book, you'll be ready to quantify the uncertainty of any model.

"This book is a comprehensive guide and resource for anyone who wants to learn how to quantify uncertainty with conformal prediction by using python. Christoph's writing is clear and engaging. He provides practical examples that help readers understand how to apply conformal prediction techniques/concepts to real-world problems."
– Tony Zhang, Data Scientist at Munich Re

Genres: Nonfiction

104 pages, Paperback

Published February 27, 2023

Reviews

[Hossein Ahmadi · Rating 5 out of 5]

This book gave me a better understanding of what conformal prediction is and how to implement it by providing hands-on example. It is a brief yet helpful summary of conformal prediction. I’d recommend!

[molly · Rating 5 out of 5]

Fantastic applied and intuitive introduction to a difficult topic. This was perfect for me as a non-statistician to quickly be able to actually use conformal prediction without wading through all the math. The introduction was brief but comprehensive, and focused mainly on fostering an intuitive understanding of why conformal prediction is useful and the basic principle of what a non-conformity score is. From there, it is relatively straightforward to understand the extensions to different types of models, given a basic understanding of machine learning model evaluation. It's not suitable for the total lay reader (although what total lay reader would pick this up, let's be honest), but if you know the basic models and metrics you should be fine. I would say you can be building your own conformal predictions within half an hour of picking up the book. Amazing.

Don't skip the Q&A at the end! I love how he straightforwardly says what the sample size for calibration should be, and whether the calibration and evaluation datasets can be the same (these are the types of things i usually end up googling fruitlessly). He also gives a toolset for developing your own conformal prediction, but in my case it was also useful for testing my comprehension.

[Hayden · Rating 4 out of 5]

About 85% of the book's content felt like an exact replica of material available online. Which was fairly disappointing at first. Since you are just paying $20 to get a localized version of content you could track down in about 8 hours total. I know this since a few of months ago around the time he was writing the book, I scoured the internet for anything on conformal prediction.

Though, after rereading the book for a second time, many of the underlying principles really began to click - and I started to change my opinion on it's utility - thus the four stars from a person who can be fairly stingy with higher ratings.

Given the book's title and content, the target audience is truly a person completely new to the topic, while others may be disappointed or underwhelmed.

[Kevin · Rating 5 out of 5]

An excellent overview of the basic theory and logic behind conformal prediction, a great guide to help you figure out where you might use it and how to go deeper. Also written in very approachable language, highly recommend!

[Josua Naiborhu · Rating 5 out of 5]

great book with intuitive way of explaining intricate concept of conformal prediction. top-notch!

[Amir V · Rating 4 out of 5]

* Very useful code snippets, practical tips and rules of thumb.
* The intuition behind conformal prediction could use a bit more elaboration, but maybe that's not what the book is intended for.
* They top-level survey of the relevant papers on applying CP to various machine learning problems is invaluable.