The Black Box Society Quotes

“Facebook had a “Verified Apps” program & claimed it certified the security of participating apps. It didn’t. Facebook promised users that it would not share their personal information with advertisers. It did.”
― Frank Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information

“The bulk of online privacy policies are a great example of a failed disclosure regime. They revolve around the fiction that consumers can and will bargain for privacy, or “opt out” of deals or jobs they deem too privacy invasive.”
― Frank Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information

“American regulators tend to set up elaborate monitoring regimes, but then are unable to (or fail to) impose meaningful fines for clear wrongdoing. Such regulation appears to be little more than a full employment program for compliance officers and attorneys.”
― Frank Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information

“Transactions that are too complex to explain to outsiders may well be too complex to be allowed to exist.”
― Frank Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information

“The shadowy masters of industrial data mining eviscerate personal privacy from behind a veil of corporate secrecy. We’ll see this dynamic repeatedly: corporate secrecy expands as the privacy of human beings contracts.”
― Frank Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information

“The more we rely on search engines and social networks to find what we want and need, the more influence they wield. The power to include, exclude, and rank is the power to ensure that certain public impressions become permanent, while others remain fleeting.”
― Frank Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information

“The argument that search and reputation algorithms are what they are and must be so forever appears to carry a lot of conviction in some quarters, but it is a self-serving oversimplication and no true reflection of reality.”
― Frank Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information

“This move from legitimation-via-transparency to protection-via-secrecy was the soil out of which the black box society sprang, and with it, many of the social dangers of the information age.”
― Frank Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information

“Ideology is enlisted to hide that self-serving rationale: basic, commonsense record-keeping requirements are resisted on the ostensible grounds that “big government” is strangling free enterprise.”
― Frank Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information

“The reputation, search, and finance industries profit by keeping us in the dark. And they have created a larger culture of secrecy that infects all industry.”
― Frank Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information

“If a would-be Sinclair tried to document today’s food horror stories, there’s a good chance he’d be fined, jailed, or even labeled a terrorist. In Iowa, Utah, and Missouri, undercover investigations of factory farms are illegal.”
― Frank Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information

“A “war on finance fraud and systemic risk” would be a welcome new direction for the intelligence apparatus.”
― Frank Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information

“Fortunately, building early warning systems into the finance system will not be as difficult or costly as the vast antiterror apparatus.”
― Frank Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information

“A “write once, read-many” (WORM) storage drive could record all uses of the system, since it can be “designed so that data cannot be altered once it is written to disc.”
― Frank Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information

“Unfortunately, the contemporary domestic intelligence apparatus is so vast as to render the judiciary incapable of fine-grained review of its decisions.”
― Frank Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information

“lists), activists have been considered threats, rather than citizens exercising core constitutional rights. Political dissent is a routine target for surveillance by the FBI.”
― Frank Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information

“Presently, a falsehood in a single large database can percolate into dozens of smaller ones, and it is often up to the victim to request corrections, one by one.”
― Frank Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information

“To avoid a black market in hacked data (as well as to assure proper enforcement of current fair data practices law), we should require data controllers to keep records of the original source of their data,”
― Frank Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information

“Some of the black boxes of reputation, search, and finance simply need to be pried open. For example, data brokers need to fess up about the data they are hoarding, trading, and selling.”
― Frank Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information

“regulators need to start forcing such firms to give consumers a sense of the sheer size of the data trove gathered about them—and its content.”
― Frank Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information

“The monetary fines levied against Facebook for these violations were minuscule: less than a few hours of revenue.”
― Frank Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information

“How much does the black box firm have to reveal? To whom must it reveal it? And how fast must the revelation occur?”
― Frank Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information

“The first step toward improving reputation, search, and finance firms is to learn more about what they are doing. But that is only the beginning of reform.”
― Frank Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information

“philosopher Langdon Winner called “technological somnambulism,”217 we have given the search sector an almost unimaginable power to determine what we see, where we spend, how we perceive.”
― Frank Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information

“Google or Facebook were once in the right place at the right time. It’s not clear whether they are still better than anyone else at online data science, or whether their prominence is such that they’ve become the permanent “default.”
― Frank Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information

“The power to include, exclude, and rank is the power to ensure which public impressions become permanent and which remain fleeting.9 That is why search services, social and not, are “must-have” properties for advertisers”
― Frank Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information

“It is time to reclaim our right to the presumption of innocence, and to the security of the light. It may be that we cannot stop the collection of information, but we can regulate how it is used.”
― Frank Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information

“That, to me, is the core harm of surveillance: that it freezes into place an inefficient (or worse) politico-economic regime by cowing its critics into silence.”
― Frank Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information

“It would be easier to give tech companies the benefit of the doubt if Silicon Valley’s own diversity record weren’t so dismal.”
― Frank Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information

“They are programmed by human beings, whose values are embedded into their software.106 And they must often use data laced with all-too-human prejudice.”
― Frank Pasquale, The Black Box Society: The Secret Algorithms That Control Money and Information