ZeV’s Reviews > Algorithms to Live By: The Computer Science of Human Decisions > Status Update

"If changing strategies doesn't help, you can try to change the game. And if that's not possible, you can at least exercise some control about which games you choose to play. The road to hell is paved with intractable recursions, bad equilibria, and information cascades. Seek out games where honesty is the dominant strategy. Then just be yourself."

In building network of people, the young explores more, the old exploits more. But the social preference is not about age as such -- they're about where people perceive themselves to be on the *interval* relevant to their decision. Meaning, if the old knows they live much longer, their preference becomes exploration as opposed to exploitation.

Multi-armed bandit problems are everywhere. How well do people solve these problems? In general, it seems that people tent to over explore (i.e., under exploit as a result).

Clinical trial and the selection of treatments as a multi-armed bandit problem, trying to get the better treatments to people even while an experiment is in progress. (History of development of such methods.)

Algorithms that offer the guarantee of minimal regret ... of the ones they've discovered, the most popular are known as Upper Confidence Bound algorithms.

Jeff Bezos's "regret minimization framework": So I wanted to project myself forward to age 80 and say, "Okay, now I'm looking back on my life. I want to have minimized the number of regrets I have. ... I knew the one thing I might regret is not ever having tried."

The Gittins index provides a formal, rigorous justification for preferring the unknown, provided we have some opportunity to exploit the results of what we learn from exploring ... the unknown has a chance of being better, even if we actually expect it to be no different, or if it's just as likely to be worse. The untested rookie is worth more than the veteran of seemingly equal ability, since we know less about him.

Exploration vs. exploitation: A sobering property of trying new things is that the value of exploration, of finding a new favorite, can only go down over time, as the remaining opportunities to savor it dwindle. Discovering an enchanting cafe on your last night in town doesn't give you the opportunity to return. ... The flip side is that the value of exploitation can only go *up* over time.