Chaos Monkeys: Obscene Fortune and Random Failure in Silicon Valley

by Antonio García Martínez

Here’s something you may not know: every time you go to Facebook or ESPN.com or wherever, you’re unleashing a mad scramble of money, data, and pixels that involves undersea fiber-optic cables, the world’s best database technologies, and everything that is known about you by greedy strangers. Every. Single. Time. The magic of how this happens is called “real-time bidding” (RTB) exchanges, and we’ll get into the technical details before long. For now, imagine that every time you go to CNN.com, it’s as though a new sell order for one share in your brain is transmitted to a stock exchange. Picture it: individual quanta of human attention sold, bit by bit, like so many million shares of General Motors stock, billions of times a day. Remember Spear, Leeds & Kellogg, Goldman Sachs’s old-school brokerage acquisition, and its disappearing (or disappeared) traders? The company went from hundreds of traders and two programmers to twenty programmers and two traders in a few years. That same process was just starting in the media world circa 2009, and is right now, in 2016, kicking into high gear. As part of that shift, one of the final paroxysms of wasted effort at Adchemy was taking place precisely in the RTB space. An engineer named Matthew McEachen, one of Adchemy’s best, and I built an RTB bidding engine that talked to Google’s huge ad exchange, the figurative New York Stock Exchange of media, and submitted bids and ads at speeds of upwards of one hundred thousand requests per sec...