What do you think?
Rate this book
Statistical Methods in Online A/B Testing
"Statistical Methods in Online A/B Testing" is a comprehensive guide to statistics in online controlled experiments, a.k.a. A/B tests, that tackles the difficult matter of statistical inference in a way accessible to readers with little to no prior experience with it. Each concept is built from the ground up, explained thoroughly, and illustrated with practical examples from website testing. The presentation is straight to the point and practically oriented so you can apply the takeaways in your daily work.
It is a must-read for anyone looking for a deep understanding of how to make data-driven business decisions through conversion rate optimizers, product managers, growth experts, data analysts, marketing managers, experts in user experience and design. The new research presented and the fresh perspective on how to apply statistics and experimentation to achieve business goals make for an interesting read even for experienced statisticians.
The book deals with scientific methods, but their introductions and explanations are grounded in the business goals they help achieve, such as innovating under controlled risk, and estimating the effect of proposed business actions before committing to them. While the book doesn't shy away from math and formulas, it is to the extent to which these are essential for understanding and applying the underlying concepts. The presentation is friendly to readers with little to no prior knowledge in statistics. Artificial and impractical examples like dice rolling and betting are absent, instead statistical concepts are illustrated through scenarios which might well be mistaken with the last couple of A/B tests you managed.
This book also doesn't shy away from the fact that much of the current statistical theory and practice in online A/B testing is misguided, misinterpreted, or misapplied. It also addresses the issue of blind copying of scientific applications without due consideration of the unique features of online business, which is widespread. The book will help you avoid these malpractices by explicitly pointing out frequent mistakes, while also helping you align your usage of statistics and experimentation with any business goals you might want to pursue.
It is a must-read for anyone looking for a deep understanding of how to make data-driven business decisions through conversion rate optimizers, product managers, growth experts, data analysts, marketing managers, experts in user experience and design. The new research presented and the fresh perspective on how to apply statistics and experimentation to achieve business goals make for an interesting read even for experienced statisticians.
The book deals with scientific methods, but their introductions and explanations are grounded in the business goals they help achieve, such as innovating under controlled risk, and estimating the effect of proposed business actions before committing to them. While the book doesn't shy away from math and formulas, it is to the extent to which these are essential for understanding and applying the underlying concepts. The presentation is friendly to readers with little to no prior knowledge in statistics. Artificial and impractical examples like dice rolling and betting are absent, instead statistical concepts are illustrated through scenarios which might well be mistaken with the last couple of A/B tests you managed.
This book also doesn't shy away from the fact that much of the current statistical theory and practice in online A/B testing is misguided, misinterpreted, or misapplied. It also addresses the issue of blind copying of scientific applications without due consideration of the unique features of online business, which is widespread. The book will help you avoid these malpractices by explicitly pointing out frequent mistakes, while also helping you align your usage of statistics and experimentation with any business goals you might want to pursue.
- GenresBusiness
304 pages, Kindle Edition
Published September 27, 2019
14 people are currently reading
167 people want to read
Ratings & Reviews
Friends & Following
Create a free account to discover what your friends think of this book!
Community Reviews
Displaying 1 - 6 of 6 reviews
December 1, 2019
A very practical treatise on how to correctly + effectively perform AB testing in a digital site/app setting.
A must read for anyone finding themselves performing tests but not feeling 100% confident in the process.
A must read for anyone finding themselves performing tests but not feeling 100% confident in the process.
June 11, 2020
Lots of knowledge on the nuts and bolts behind many concepts of A/B testing that are not looked into in so many blog posts about this subject. I got a lot of value out of it.
September 19, 2021
Essential for any practitioner of AB testing in the online space. Generalisable to all fields, not just e-commerce.
May 23, 2022
Very good intro to ab testing
June 1, 2020
This book is an asset. A must read for any business practicioner who deals with testing in a daily basis but is not a data/ statistics geek.
It's very practical and provides a lot of useful insights whose implementation is straightforward.
Didn't give 5 stars because in various parts of the book the author refer to a "proprietary implementation of a given solution" that is available in his software, without providing further explanation. Despite of that, it's a good book.
It's very practical and provides a lot of useful insights whose implementation is straightforward.
Didn't give 5 stars because in various parts of the book the author refer to a "proprietary implementation of a given solution" that is available in his software, without providing further explanation. Despite of that, it's a good book.
January 28, 2020
THE textbook for online A/B testing and the often complicated statistics behind a good optimization program.
Even for someone in the field for well over 5 years, I was able to fill in some of the holes left from my university Statistics 101 course.
If you or someone you care about is in the field of digital marketing this would be a must-read. You can't run a proper optimization program unless you have the basics covered and this book goes well beyond that.
It would give you a really solid understanding of how to read results from your A/B tests, how to properly draw hypothesis, how to pick the right kind of statistical tests and so much more.
Even for someone in the field for well over 5 years, I was able to fill in some of the holes left from my university Statistics 101 course.
If you or someone you care about is in the field of digital marketing this would be a must-read. You can't run a proper optimization program unless you have the basics covered and this book goes well beyond that.
It would give you a really solid understanding of how to read results from your A/B tests, how to properly draw hypothesis, how to pick the right kind of statistical tests and so much more.
Displaying 1 - 6 of 6 reviews