Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

by Cathy O'Neil

They’re n...

Yet a crucial component of a WMD is that it is damaging to many people’s lives. And with these types of predatory ads, the damage doesn’t begin until students start ta...

To many of the students, the loans sound like free money, and the school doesn’t take pains to correct this misconception.

But it is debt, and many of them quickly find themselves up to their necks in it.

Some people no doubt attend for-profit colleges and emerge with knowledge and skills that serve them well.

But do they fare better than graduates from community colleges, whose degrees cost a fraction as much?

Their assets are negative: the average household in this enormous and struggling underclass has a net debt of $14,800, much of it in extortionate credit card accounts.

What these people need is money. And the key to earning more money, they hear again and again, is education.

Along come the for-profit colleges with their highly refined WMDs to target and fleece th...

They sell them the promise of an education and a tantalizing glimpse of upward mobility—while plu...

They take advantage of the pressing need in poor households, along with their ignorance and their aspi...

And they do this at gr...

This leads to hopelessness and despair, along with skepticism about the value of education more broadly, and it exacerba...

Now regulators are pushing for new laws governing the market for personal data—a crucial input for all sorts of WMDs.

date, a couple of federal laws, such as the Fair Credit Reporting Act and the Health Insurance Portability and Accountability Act, or HIPAA, establish some limits on health and credit data.

we’ll see in coming chapters, some of the most effective and nefarious WMDs manage to engineer work-arounds.

They study everything from neighborhoods to Facebook friends to predict our behavior—and even lock us up.

My point is that police make choices about where they direct their attention. Today they focus almost exclusively on the poor.

That’s their heritage, and their mission, as they understand it.

And now data scientists are stitching this status quo of the social order into models, like PredPol, that hold e...

this sense, PredPol, even with the best of intentions, empowers police departments to zero in on the poor, stopping more of them, arresting a portion of those, and sending a subgroup to prison.

And the police chiefs, in many cases, if not most, think that they’re taking the only sensible route to combating crime.

That’s where it is, they say, pointing to the highlighted...

The result is that we criminalize poverty, believing all the while that our tools are not only scientific but fair.

Known simply as stop and frisk to most people, the practice had drastically increased in the data-driven age of CompStat.

The police regarded stop and frisk as a filtering device for crime.

Over the previous decade, the number of stops had risen by 600 percent, to nearly seven hundred thousand incidents.

The great majority of those stopped were innocent.

For them, these encounters were highly unpleasant, ...

Everyone knew that an outsized proportion of the people the police stopped were young, dark-skinned men.

But how many did they stop? And how often did these encounters lead to arrests or stop crimes?

What we found, to no great surprise, was that an overwhelming majority of these encounters—about 85 percent—involved young African American or Latino men.

In certain neighborhoods, many of them were stopped repeatedly.

Only 0.1 percent, or one of one thousand stopped, was linked in any w...

Yet this filter captured many others for lesser crimes, from drug possession to underage drinking, that might h...

Some of the targets, as you might expect, got angry, and a good number of those found themselves c...

The NYCLU sued the Bloomberg administration, charging that the stop-and-f...

It was an example of uneven policing, one that pushed more minorities into the criminal just...

Black men, they argued, were six times more likely to be incarcerated than white men and twenty-one times more likely to be killed by police, at least according to the av...

Stop and frisk isn’t exactly a WMD, because it relies on human judgment and is not for...

But it is built upon a simple and destructi...

If police stop one thousand people in certain neighborhoods, they’ll uncover, on average, one significant su...

Even when the hit ratio is miniscule, if you give yourself enough chances you’ll reach your target.

And that helps to explain why the program grew so dramatically under Bloomberg’s watch.

If stopping six times as many people led to six times the number of arrests, the inconvenience and harassment suffered by thousands upon thou...

Weren’t they interested in sto...

Aspects of stop and frisk were similar to WMDs, though. For example, it had...

It ensnared thousands of black and Latino men, many of them for committing the petty crimes and misdemeanors that go on in college fr...

But while the great majority of university students were free to sleep off their excesses, the victims of stop and frisk were booked, and some of them di...

What’s more, each arrest created new data, further just...