Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

by Cathy O'Neil

Following the market crash of 2008, two financial engineers, Emanuel Derman and Paul Wilmott, drew up such an oath. It reads: ~ I will remember that I didn’t make the world, and it doesn’t satisfy my equations. ~ Though I will use models boldly to estimate value, I will not be overly impressed by mathematics. ~ I will never sacrifice reality for elegance without explaining why I have done so. ~ Nor will I give the people who use my model false comfort about its accuracy. Instead, I will make explicit its assumptions and oversights. ~ I understand that my work may have enormous effects on society and the economy, many of them beyond my comprehension. That’s a good philosophical grounding. But solid values and self-regulation rein in only the scrupulous. What’s more, the Hippocratic Oath ignores the on-the-ground pressure that data scientists often confront when bosses push for specific answers. To eliminate WMDs, we must advance beyond establishing best practices in our data guild. Our laws need to change, too. And to make that happen we must reevaluate our metric of success.