Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

by Cathy O'Neil

In WMDs, many poisonous assumptions are camouflaged by math and go largely untested and unquestioned.

“How do you justify evaluating people by a measure for which you are unable to provide explanation?”

Ill-conceived mathematical models now micromanage the economy, from advertising to prisons.

But as a rule, the people running the WMDs don’t dwell on those errors. Their feedback is money, which is also their incentive.

A model’s blind spots reflect the judgments and priorities of its creators.

racism is the most slovenly of predictive models. It is powered by haphazard data gathering and spurious correlations, reinforced by institutional inequities, and polluted by confirmation bias.

The result is that we criminalize poverty, believing all the while that our tools are not only scientific but fair.

Bogus is the word for it. In fact, misinterpreted statistics run through the history of teacher evaluation.

The problem was that the administrators lost track of accuracy in their quest to be fair. They understood that it wasn’t right for teachers in rich schools to get too much credit when the sons and daughters of doctors and lawyers marched off toward elite universities. Nor should teachers in poor districts be held to the same standards of achievement. We cannot expect them to perform miracles.

By their sixties, whites are eleven times richer than African Americans.

Each of us should have the right to receive an alert when a credit score is being used to judge or vet us.

Math deserves much better than WMDs, and democracy does too.