Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

by Cathy O'Neil

cling

hellish.

lot.

appalling

haphazard

It’s almost as if the software were designed expressly to punish low-wage workers and to keep them down.

bleary

The root of the trouble, as with so many other WMDs, is the modelers’ choice of objectives. The model is optimized for efficiency and profitability, not for justice or the good of the “team.”

flummoxes

burrowed

cull

blithely

realm.

lurid

plummeting

tenure,

elated,

bogus,

boo-boo

ballooned

In statistics, this phenomenon is known as Simpson’s Paradox: when a whole body of data displays one trend, yet when broken into subgroups, the opposite trend comes into view for each of those subgroups.

botched

mishmash

Unlike the FICO scores they resemble, e-scores are arbitrary, unaccountable, unregulated, and often unfair—in short, they’re WMDs.

Now that it might be time to buy a house or car, it’s only natural that financial models would mine the same trove of data to size us up.

make do

Fair and Isaac’s great advance was to ditch the proxies in favor of the relevant financial data, like past behavior with respect to paying bills. They focused their analysis on the individual in question—and not on other people with similar attributes.

veritable

deadbeats

murky

nourished

Another all-too-common approach is to consider the applicant’s credit score. If people pay their bills on time and avoid debt, employers ask, wouldn’t that signal trustworthiness and dependability?

turf.

blemished

The practice of using credit scores in hirings and promotions creates a dangerous poverty cycle. After all, if you can’t get a job because of your credit record, that record will likely get worse, making it even harder to land work.

plight

Even with the Affordable Care Act, which reduced the ranks of the uninsured, medical expenses remain the single biggest cause of bankruptcies in America.

Given these numbers, it is not hard to argue that the poverty trap created by employer credit checks affects society unequally and along racial lines.

rife

ordeal

waltz

sweepstakes

cornucopia

scrape

altercations

tenants.

besieged

tarring

riddled

forgery,