More on this book
Community
Kindle Notes & Highlights
freedom to
The digital poorhouse also impairs the ability of poor and working-class people to exert self-determination and autonomy, undermining freedom to. The complexity of the digital poorhouse erodes targets’ feelings of competence and proficiency. Too often, these tools simply grind down a person’s resolve until she gives up things that are rightfully hers: resources, autonomy, respect, and dignity.
Equity requires the ability to develop and evolve. But as Cathy O’Neil has written, “Mathematical models, by their nature, are based on the past, and on the assumption that patterns will repeat.”9
Link here with anticipatory systems
The digital poorhouse undermines inclusion as assimilation.
The digital poorhouse also limits the ability of its targets to achieve inclusion as their whole selves.
But the moral calculus of the digital poorhouse individualizes risk and shreds social commitment.
If there is to be an alternative, we must build it on purpose, brick by brick and byte by byte.
“cybernation,”
internecine
But there is nothing inevitable about this outcome. We can dismantle the digital poorhouse.
The most important step in dismantling the digital poorhouse is changing how we think, talk, and feel about poverty.
Does the tool increase the self-determination and agency of the poor? Would the tool be tolerated if it was targeted at non-poor people?
Link here to Cory Doctorow's Gradient of Privilege
Think of the principles of non-harm, below, as a first draft of a Hippocratic oath for the data scientists, systems engineers, hackers, and administrative officials of the new millennium.
And the state doesn’t require a cop to kill a person.
The God of history is still saying, “That is not enough!”
Aren’t tools neutral? Isn’t it the intention of their users that matter?
smoothing viscous material. The shape of a bucket trowel harmonizes with its purpose seamlessly; it pulls every grain of sand off the side of a bucket with a motion so natural it feels inevitable, like dancing with a talented partner.
Masonry tools evolved over the past six thousand years to do particular jobs. They are shaped by their purpose and by the minds, hands, and cultures of people.
As feminist technology scholar Corlann Gee Bush wrote in the early 1980s, technologies are best understood as having valence, “a bias or ‘charge’ analogous to that of atoms that have lost or gained electrons through ionization.”
Bush is not arguing that a technology’s use is absolutely determined by its design, but that the history and context it carries within its form gives it a kind of mass, a gravity that draws specific modes of behavior and interaction into its orbit.
Jenny L. Davis would call this an affordance.
Automated decision-making technologies used in public services cannot escape the history that spawned them. The legacy of the poorhouse influences which technologies we choose to develop, affects their design, and shapes their impacts on people and communities still.
Automated eligibility systems and predictive analytics are best understood as political decision-making machines. They do not remove bias, they launder it, performing a high-tech sleight of hand that encourages us to perceive deeply political decisions as natural and inevitable. They reinforce some values: efficiency, cost savings, adherence to the rules. They obscure or displace others: self-determination, dignity, autonomy, mutual obligation, trust, due process, equity. They embody very particular ways of understanding the world, and foreclose more promising visions.
