Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor

by Virginia Eubanks

New Deal legislation undoubtedly saved thousands of lives and prevented destitution for millions. New labor laws led to a flourishing of unions and built a strong white middle class. The Social Security Act of 1935 established the principle of cash payments in cases of unemployment,

New Deal programs also enshrined the male breadwinner as the primary vehicle for economic support of women and families. Federal protections were tied to wages, union membership, unemployment insurance, and pensions. But by incentivizing long-term wage-earning and full-time,

year-round work, the protections privileged men’s employment patterns over women’s. Another signature program of the New Deal, Aid to Dependent Children (ADC, called Aid to Families with Dependent Children, or AFDC, after 1962), was structured to support a tiny number of widows with children after the death of a male wage earner. Women’s economic security was thus tied securely to their roles as wives, mothers, or widows, guaranteeing their continued economic dependence.

If we just gather all the facts, systems engineers assume, the correct answers to intractable policy problems like homelessness will be simple, uncontroversial, and widely shared. But, for better or worse, this is not how politics work.

Data scientist Cathy O’Neil has written that “models are opinions embedded in mathematics.”

Once the big blue button is clicked and the AFST runs, it manifests a thousand invisible human choices. But it does so under a cloak of evidence-based objectivity and infallibility.

I find the philosophy that sees human beings as unknowable black boxes and machines as transparent deeply troubling. It seems to me a worldview that surrenders any attempt at empathy and forecloses the possibility of ethical development. The presumption that human decision-making is opaque and inaccessible is an admission that we have abandoned a social commitment to try to understand each other.

Our relationship to poverty in the United States has always been characterized by what sociologist Stanley Cohen calls “cultural denial.” Cultural denial is the process that allows us to know about cruelty, discrimination, and repression, but never openly acknowledge

This myopic focus on what’s new leads us to miss the important ways that digital tools are embedded in old systems of power and privilege.

It is mere fantasy to think that a statistical model or a ranking algorithm will magically upend culture, policies, and institutions built over centuries.

Oath of Non-Harm for an Age of Big Data I swear to fulfill, to the best of my ability, the following covenant: I will respect all people for their integrity and wisdom, understanding that they are experts in their own lives, and will gladly share with them all the benefits of my knowledge. I will use my skills and resources to create bridges for human potential, not barriers. I will create tools that remove obstacles between resources and the people who need them. I will not use my technical knowledge to compound the disadvantage created by historic patterns of racism, classism, able-ism, sexism, homophobia, xenophobia, transphobia, religious intolerance, and other forms of oppression. I will design with history in mind. To ignore a four-century-long pattern of punishing the poor is to be complicit in the “unintended” but terribly predictable consequences that arise when equity and good intentions are assumed as initial conditions. I will integrate systems for the needs of people, not data. I will choose system integration as a mechanism to attain human needs, not to facilitate ubiquitous surveillance.

I will not collect data for data’s sake, nor keep it just because I can. When informed consent and design convenience come into conflict, informed consent will always prevail. I will design no data-based system that overturns an established legal right of the poor. I will remember that the technologies I design are not aimed at data points, probabilities, or patterns, but at human beings.