More on this book
Community
Kindle Notes & Highlights
by
Cathy O'Neil
Read between
July 21 - August 8, 2018
Conditions change, and so must the model.
A model’s blind spots reflect the judgments and priorities of its creators.
So to sum up, these are the three elements of a WMD: Opacity, Scale, and Damage.
Justice cannot just be something that one part of society inflicts upon the other.
In other words, the modelers for e-scores have to make do with trying to answer the question “How have people like you behaved in the past?” when ideally they would ask, “How have you behaved in the past?”
In the end, how an algorithm will be used should affect how it is constructed and optimized.
We need to ensure that data effectively and comprehensively represents the world—even, we hope, bears witness to the world and its suffering, rather than shaping the world—especially in ways that exacerbate misery.
Right now, mammoth companies like Google, Amazon, and Facebook exert incredible control over society because they control the data.
Let’s start building a framework now to hold algorithms accountable for the long term. Let’s base it on evidence that the algorithms are legal, fair, and grounded in fact. And let’s keep evolving what those things mean, depending on the context. It will be a group effort, and we’ll need as many lawyers and philosophers as engineers, but we can do it if we focus our efforts.