Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

by Cathy O'Neil

They don’t need to know anything about statistics or the weig...

The software itself, much like an online travel site, creates individual ...

Think of it: transparent, controlled by the use...

You might call it the opposit...

The weakest part of his argument, it seemed to me, was its justification.

He argued that the coming avalanche of personalized advertising would be so useful and timely that customers would welcome it.

They would beg f...

As he saw it, most people objected to advertisements because they wer...

In the future, they wo...

Presumably, folks in his exclusive demo would welcome pitches tailored to them, perhaps featuring cottages in the Bahamas, jars of hand-pressed virgin o...

And he joked that he would never have to see another ad for the University of Phoenix—a for-profit education factory that appeals largely to the strivi...

While spending more than $50 million on Google ads alone, the University of Phoenix targeted poor people with the bait of upward mobility.

Its come-on carried the underlying criticism that the struggling classes weren’t doing enough to improve their lives.

And it w...

Between 2004 and 2014, for-profit enrollment tripled, and the industry now accounts for 11 percent of the country’s...

If it was true during the early dot-com days that “nobody knows you’re a dog,” it’s the exact opposite today.

We are ranked, categorized, and scored in hundreds of models, on the basis of our revealed preferences and patterns.

This establishes a powerful basis for legitimate ad campaigns, but it also fuels ...

ads that pinpoint people in great need and sell them false or ...

They find inequality and f...

The result is that they perpetuate our existing social stratification, with...

The greatest divide is between the winners in our system, like our venture capitalist, and the ...

Anywhere you find the combination of great need and ignorance, you’ll lik...

the boom in for-profit colleges is fueled by predatory ads.

Vatterott College, a career-training institute, is a particularly nasty example.

A 2012 Senate committee report on for-profit colleges described Vatterott’s recruiting manual, which sounds diabolical.

It directs recruiters to target “Welfare Mom w/Kids. Pregnant Ladies. Recent Divorce. Low Self-Esteem. Low Income Jobs. Experienced a Recent Death. Physically/Mentally Abused. Recent Incarcera...

Why, specifically, were they targetin...

Vulnerability is wo...

It always ha...

She covers her mouth when she smiles, indicating that she’s sensitive about her bad teeth.

She anxiously twirls her old wedding ring, which from the looks of her swollen knuckle will be stuck there till the end of her days. Arthritis.

So when he pitches his products to her, he focuses on the ugliness of her tee...

He can promise to restore the beauty of her smile and wash away the ...

With this knowledge, he knows he’s halfway to a sale before even clearin...

The playbook for predatory advertisers is similar, but they carry it out at massive scale, targeting ...

The customers’ ignorance, of course, is a crucial pie...

Once the ignorance is established, the key for the recruiter, just as for the snake-oil merchant, is to locate the most vulnerable people and then use their private information against them.

This involves finding where they suffer the most, which is known as the “pain point.”

It might be low self-esteem, the stress of raising kids in a neighborhood of warring gangs, o...

Many people unwittingly disclose their pain points when they look ...

or, later, when they fill out college q...

With that valuable nugget in hand, recruiters simply promise that an expensive education at their university will provide t...

“We deal with people that live in the moment and for the moment,” Vatterott’s tra...

“Their decision to start, stay in school or quit school is based more ...

Pain is the greater motivator in the...

To optimize recruiting—and revenue—they need to know whom their messages reached and, if possible, what impact they had.

Only with this data can they go on to optimize the operation.

The key for any optimization program, naturally, is to ...