Weapons of Math Destruction Quotes

“Equally important, statistical systems require feedback—something to tell them when they’re off track. Without feedback, however, a statistical engine can continue spinning out faulty and damaging analysis while never learning from its mistakes. Many of the WMDs I’ll be discussing in this book, including the Washington school district’s value-added model, behave like that. They define their own reality and use it to justify their results. This type of model is self-perpetuating, highly destructive—and very common. If the people being evaluated are kept in the dark, the thinking goes, they’ll be less likely to attempt to game the system. Instead, they’ll simply have to work hard, follow the rules, and pray that the model registers and appreciates their efforts. But if the details are hidden, it’s also harder to question the score or to protest against it.”
― Cathy O'Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

“It’s important to note, as we endeavor to understand relative harms, that they are entirely dependent on context. For example, if a high-risk score for a given defendant qualified him for a reentry program that would help him find a job upon release from prison, we’d be much less worried about false positives. Or in the case of the child abuse algorithm, if we are sure that a high-risk score leads to a thorough and fair-minded investigation of the situation at home, we’d be less worried about children unnecessarily removed from their parents. In the end, how an algorithm will be used should affect how it is constructed and optimized.”
― Cathy O'Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

“People with savings, of course, can keep their credit intact during tough times. Those living from paycheck to paycheck are far more vulnerable. Consequently, a sterling credit rating is not just a proxy for responsibility and smart decisions. It is also a proxy for wealth. And wealth is highly correlated with race.”
― Cathy O'Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

“The practice of using credit scores in hirings and promotions creates a dangerous poverty cycle. After all, if you can’t get a job because of your credit record, that record will likely get worse, making it even harder to land work. It’s not unlike the problem young people face when they look for their first job—and are disqualified for lack of experience. Or the plight of the longtime unemployed, who find that few will hire them because they’ve been without a job for too long. It’s a spiraling and defeating feedback loop for the unlucky people caught up in it.”
― Cathy O'Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

“If it was true during the early dot-com days that “nobody knows you’re a dog,” it’s the exact opposite today. We are ranked, categorized, and scored in hundreds of models, on the basis of our revealed preferences and patterns.”
― Cathy O'Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

“We can use the scale and efficiency that make WMDs so pernicious in order to help people. It all depends on the objective we choose.”
― Cathy O'Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

“Our livelihoods increasingly depend on our ability to make our case to machines.”
― Cathy O'Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

“Note that there’s no option to answer “all of the above.” Prospective workers must pick one option, without a clue as to how the program will interpret it. And some of the analysis will draw unflattering conclusions. If you go to a kindergarten class in much of the country, for example, you’ll often hear teachers emphasize to the children that they’re unique. It’s an attempt to boost their self-esteem and, of course, it’s true. Yet twelve years later, when that student chooses “unique” on a personality test while applying for a minimum-wage job, the program might read the answer as a red flag: Who wants a workforce peopled with narcissists?”
― Cathy O'Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

“Why, specifically, were they targeting these folks? Vulnerability is worth gold. It always has been.”
― Cathy O'Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

“Instead, they face heightened anxiety and sleep deprivation, which causes dramatic mood swings and is responsible for an estimated 13 percent of highway deaths. Worse yet, since the software is designed to save companies money, it often limits workers’ hours to fewer than thirty per week, so that they are not eligible for company health insurance.”
― Cathy O'Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

“Like gods, these mathematical models were opaque, their workings invisible to all but the highest priests in their domain: mathematicians and computer scientists.”
― Cathy O'Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

“Welcome to the dark side of Big Data.”
― Cathy O'Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

“If language and child care issues posed problems for otherwise solid candidates, the solution was not to reject those candidates but instead to provide them with help—whether English classes or onsite day care—to pull them through. This is a point I’ll be returning to in future chapters: we’ve seen time and again that mathematical models can sift through data to locate people who are likely to face great challenges, whether from crime, poverty, or education.”
― Cathy O'Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

“Sometimes the job of a data scientist is to know when you don't know enough.”
― Cathy O'Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

“mathematical models were opaque, their workings”
― Cathy O'Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

“Apollo Group, the parent company for the University of Phoenix, spent more than a billion dollars on marketing in 2010, almost all of it focused on recruiting. That came out to $2,225 per student on marketing and only $892 per student on instruction. Compare that to Portland Community College in Oregon, which spends $5,953 per student on instruction and about 1.2 percent of its budget, or $185 per student, on marketing.”
― Cathy O'Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

“I post a petition on my Facebook page. Which of my friends will see it on their news feed? I have no idea. As soon as I hit send, that petition belongs to Facebook, and the social network’s algorithm makes a judgment about how to best use it. It calculates the odds that it will appeal to each of my friends. Some of them, it knows, often sign petitions, and perhaps share them with their own networks. Others tend to scroll right past. At the same time, a number of my friends pay more attention to me and tend to click the articles I post. The Facebook algorithm takes all of this into account as it decides who will see my petition. For many of my friends, it will be buried so low on their news feed that they’ll never see it.”
― Cathy O'Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

“Once companies amass troves of data on employees’ health, what will stop them from developing health scores and wielding them to sift through job candidates? Much of the proxy data collected, whether step counts or sleeping patterns, is not protected by law, so it would theoretically be perfectly legal. And it would make sense. As we’ve seen, they routinely reject applicants on the basis of credit scores and personality tests. Health scores represent a natural—and frightening—next step.”
― Cathy O'Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

“And in Florida, adults with clean driving records and poor credit scores paid an average of $1,552 more than the same drivers with excellent credit and a drunk driving conviction.”
― Cathy O'Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

“Anywhere you find the combination of great need and ignorance, you’ll likely see predatory ads. If people are anxious about their sex lives, predatory advertisers will promise them Viagra or Cialis, or even penis extensions. If they are short of money, offers will pour in for high-interest payday loans. If their computer is acting sludgy, it might be a virus inserted by a predatory advertiser, who will then offer to fix it. And as we’ll see, the boom in for-profit colleges is fueled by predatory ads.”
― Cathy O'Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

“It’s easy to raise graduation rates, for example, by lowering standards. Many students struggle with math and science prerequisites and foreign languages. Water down those requirements, and more students will graduate. But if one goal of our educational system is to produce more scientists and technologists for a global economy, how smart is that? It would also be a cinch to pump up the income numbers for graduates. All colleges would have to do is shrink their liberal arts programs, and get rid of education departments and social work departments while they’re at it, since teachers and social workers make less money than engineers, chemists, and computer scientists. But they’re no less valuable to society.”
― Cathy O'Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

“By leaving cost out of the formula, it was as if U.S. News had handed college presidents a gilded checkbook. They had a commandment to maximize performance in fifteen areas, and keeping costs low wasn’t one of them.”
― Cathy O'Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

“A 2013 study by the New York Civil Liberties Union found that while black and Latino males between the ages of fourteen and twenty-four made up only 4.7 percent of the city’s population, they accounted for 40.6 percent of the stop-and-frisk checks by police. More than 90 percent of those stopped were innocent.”
― Cathy O'Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

“According to the American Civil Liberties Union, sentences imposed on black men in the federal system are nearly 20 percent longer than those for whites convicted of similar crimes.”
― Cathy O'Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

“What’s more, attempting to score a teacher’s effectiveness by analyzing the test results of only twenty-five or thirty students is statistically unsound, even laughable. The numbers are far too small given all the things that could go wrong. Indeed, if we were to analyze teachers with the statistical rigor of a search engine, we’d have to test them on thousands or even millions of randomly selected students.”
― Cathy O'Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

“The math could multiply the horseshit, but it could not decipher it.”
― Cathy O'Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

“This underscores another common feature of WMDs. They tend to punish the poor. This is, in part, because they are engineered to evaluate large numbers of people. They specialize in bulk, and they’re cheap. That’s part of their appeal. The wealthy, by contrast, often benefit from personal input. A white-shoe law firm or an exclusive prep school will lean far more on recommendations and face-to-face interviews than will a fast-food chain or a cash-strapped urban school district. The privileged, we’ll see time and again, are processed more by people, the masses by machines.”
― Cathy O'Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

“We are judged by what we do, not by who we are.”
― Cathy O'Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

“model’s blind spots reflect the judgments and priorities of its creators. While the choices in Google Maps and avionics software appear cut and dried, others are far more problematic. The value-added model in Washington, D.C., schools, to return to that example, evaluates teachers largely on the basis of students’ test scores, while ignoring how much the teachers engage the students, work on specific skills, deal with classroom management, or help students with personal and family problems. It’s overly simple, sacrificing accuracy and insight for efficiency.”
― Cathy O'Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy

“If we’re going to be equal before the law, or be treated equally as voters, we cannot stand for systems that drop us into different castes and treat us differently.”
― Cathy O'Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy