Bandit Algorithms for Website Optimization: Developing, Deploying, and Debugging

by John Myles White

When looking for ways to improve your website, how do you decide which changes to make? And which changes to keep? This concise book shows you how to use Multiarmed Bandit algorithms to measure the real-world value of any modifications you make to your site. Author John Myles White shows you how this powerful class of algorithms can help you boost website traffic, convert visitors to customers, and increase many other measures of success. This is the first developer-focused book on bandit algorithms, which were previously described only in research papers. You’ll quickly learn the benefits of several simple algorithms―including the epsilon-Greedy, Softmax, and Upper Confidence Bound (UCB) algorithms―by working through code examples written in Python, which you can easily adapt for deployment on your own website.

Genres: Programming, Technology, Software, Algorithms, Web, Computer Science, Nonfiction

85 pages, Paperback

First published January 1, 2012

Reviews

[Alan Kang · Rating 4 out of 5]

Concise, accessible and practical.

A few more words on the accessibility:

* Clearly defines critical jargons related to the Multi-armed bandit problem.
* Starts from simple algorithm and gradually develops to sophisticated one.
* Everything is explained two or three times: in plain English, simple arithmatics, and finally, in Python code.

[Hugh Rawlinson · Rating 4 out of 5]

The prose describes the concepts and the code very well. I started not knowing much about bandit algorithms, and now feel that I know the basics, and have a good idea of where to go next for further study.

Unfortunately, I found the code to be distracting due to its lack of clarity, and “unpythonic” tendencies. There were a few instances where domain specific terms were introduced, and then different names were used in code. The provided code is in R and Julia by default, and while python is also provided, the R code has to be modified. If there’s ever talk of another edition, I’d almost suggest Typescript and an accompanying website playground.

On the whole, a good quick introduction to an interesting topic.

[Quy Doan · Rating 4 out of 5]

I thought the definition of AI in 2000 has been replaced and disappeared in the deep learning era, but the A* algorithm, the annealing algorithm were still there. They was just jumping to another fields and solving another problems.
And I thought there is no alternative for A/B testing until I read this book.

[Vahe Hakobyan · Rating 3 out of 5]

Even though it is nice and intuitive, I did not find much value in this book except the series of references at the very end.
The rest is basically a paraphrasing of Sutton, which has a draft-free version of his book on the web.

[Rachel Childers · Rating 4 out of 5]

A very concise, practical introduction to bandit algorithms, with working (Python) code, for people who want to implement them and understand how they are implemented and evaluated. I picked this up because I wanted to introduce an undergrad RA to the field of bandit algorithms in the gentlest way that will actually result in being able to use and understand them, and after reading it I feel it's ideal for that task.

This isn't a complete or up-to-date overview of the field, and it's missing any formal theory, but it's not for that. It gives a qualitative introduction to the setup of the problem and evaluation of the problem, and walks you through building up each step, in an object-oriented manner that gives you a framework that can accommodate many of the more advanced or modern procedures. Use this as a supplement to a theoretical treatment (say, the relevant chapters of the Lattimore and Szepesvari giant brick of a book) to develop an intuitive understanding of what it means to explore or exploit, and play around with simulated data to see what different algorithms, hyperparameters, etc do, and develop an understanding of performance through plots that can then be mapped to the more formal guarantees provided by theory.

I do think theory is an important supplement because each simulation can only cover a limited part of the space, and so might give a misleading perspective on actual performance unless you know what to look for; eg, knowing which algorithms are instance optimal and that the bound depends on the gap between arms, vs worst case or even adversarially optimal and so with performance that degrades less in changing environments. The simulation approach is still valuable, since the simulations are effectively the object theory is about, and it's useful to visualize and experiment with it once you know what to look for. But it will be helpful to start with a super short intro to just the practicalities before getting into that.

[Leonardo]

In this book, we covered how to evaluate algorithmic predictions based on a test set that we collected beforehand. This is known as offline evaluation. If your machine learning system is user-facing, this is only the first step in evaluating an algorithm, though. The next step is usually online testing or live testing, where the consequences of employing the algorithm in the overall system are evaluated. Changing the recommendations or search results users are shown by a website can drastically change their behavior and lead to unexpected consequences. To protect against these surprises, most user-facing services employ A/B testing, a form of blind user study. In A/B testing, without their knowledge a selected portion of users will be provided with a website or service using algorithm A, while the rest of the users will be provided with algorithm B. For both groups, relevant success metrics will be recorded for a set period of time. Then, the metrics of algorithm A and algorithm B will be compared, and a selection between the two approaches will be made according to these metrics. Using A/B testing enables us to evaluate the algorithms “in the wild,” which might help us to discover unexpected consequences when users are interacting with our model. Often A is a new model, while B is the established system. There are more elaborate mechanisms for online testing that go beyond A/B testing, such as bandit algorithms. A great introduction to this subject can be found in the book Bandit Algorithms for Website Optimization by John Myles White (O’Reilly).

Introduction to Machine Learning with Python Pág.360

[Abhay · Rating 4 out of 5]

A nice, short and intuitive introduction to the topic with directions on where to go after reading the book.

[Marcin · Rating 3 out of 5]

Good introduction to one-arm bandit algorithms, definitely lacking on details and math but still clear and precise.

[Justine · Rating 5 out of 5]

One of the more accessible and easy to read CS textbooks

[Mohamed · Rating 3 out of 5]

Good introductory book. Reads like a blog post honestly. Probably needs a new edition as it does not cover new concepts such as contextual bandits and thomson sampling.

[Debasish Ghosh · Rating 3 out of 5]

Very readable approach to the subject. Personally though I would have liked a somewhat more mathematical treatment. But it sets up the stage nicely to go for more advanced papers on the subject.

[Shakir Sharfraz · Rating 5 out of 5]

If you think that A/B testing is the only way to do controlled experiments. This books gives a good primer to a better alternative - multi armed bandits.

[Alex · Rating 5 out of 5]

Awesome - clearly written and easy to understand.

[Marco · Rating 4 out of 5]

Very clear and hands-on introduction to the Multi-Armed Bandit.

[huydx · Rating 4 out of 5]

short and concise book

[Olga · Rating 4 out of 5]

Good, intuitive introduction to MAB theory.