More on this book
Community
Kindle Notes & Highlights
Started reading
April 2, 2025
Simply put, exploration is gathering information, and exploitation is using the information you have to get a known good result.
“multi-armed bandit problem.”
So explore when you will have time to use the resulting knowledge, exploit when you’re ready to cash in. The interval makes the strategy.
Win-Stay, Lose-Shift algorithm: choose an arm at random, and keep pulling it as long as it keeps paying off. If the arm doesn’t pay off after a particular pull, then switch to the other one.
valuing the present more highly than the future, as “discounting.”
Gittins index values
Unkown potential boosts value. Less data means more room for discovery making exploration worthwhile.
“regret minimization framework.”
Choosing actions that limit how much you will regret not picking the best option once you know the full results.
“optimism in the face of uncertainty.”
Assume the best about unknowns to encourage exploration.
Data scientist Jeff Hammerbacher, former manager of the Data group at Facebook, once told Bloomberg Businessweek that “the best minds of my generation are thinking about how to make people click ads.”
Tuskegee experiment
Conducted By US public health 1932-1972 Lied to black men with syphilis into thinking they were being treated while researchers withheld care. Researchers wanted to see the effects of the disease. This lead to many deaths despite penacilin being a cure. Makes my heart hurt thinking of this. Four presidents allowed this to occur: Hoover, Roosevelt, Truman, Nixon
In Zelen’s procedure, you start with a hat that contains one ball for each of the two treatment options being studied. The treatment for the first patient is selected by drawing a ball at random from the hat (the ball is put back afterward). If the chosen treatment is a success, you put another ball for that treatment into the hat—now you have three balls, two of which are for the successful treatment. If it fails, then you put another ball for the other treatment into the hat, making it more likely you’ll choose the alternative.
Zelen’s algorithm was first used in a clinical trial sixteen years later, for a study of extracorporeal membrane oxygenation, or “ECMO”—an audacious approach to treating respiratory failure in infants.
2015, the FDA released a pair of draft “guidance” documents on “Adaptive Design” clinical trials for drugs and medical devices,
If the fda is moving towards innovative trial designs, it suggests that drs should consider new drugs rather than relying soley on traditional options potentially leading to better outcomes for patients.
