More on this book
Community
Kindle Notes & Highlights
With black-box decision-makers, we are no longer facing the sexist hiring manager or xenophobic guard but a seemingly neutral device that nonetheless reflects the biases of the society it’s embedded in. Unless we demand not only a choice in deciding whether and how these systems are used but also pathways to challenge decisions, we will change the form of the gatekeeper, but the prejudice will remain.
“I am the lead author. It is based on my MIT master’s thesis.” “Well, the paper showed these systems performed very well for white men but there were some notable gaps, especially with Black women.” I wasn’t sure why he felt the need to explain the paper to me after I told him I had authored it. I was out of energy to keep challenging other people’s perceptions of me. “There is certainly work to be done,” I replied.
“That is not really our concern. We don’t have those types of African American problems in Rwanda.” I was reminded that some issues do not translate across a global context. Hearing my American accent, they probably didn’t realize I was also African.
I told them that my research used images of African parliament members, including those in Rwanda, to show issues of bias in technical systems. Still no interest. Instead, one of the men suggested, “Why not focus your energy on building something instead of critiquing what’s not working?” They then shared an initiative they were leading that was focused on supporting the development of more scientists and innovators in Rwanda.
In 2019 Google came under fire after it was reported that a subcontractor coerced homeless individuals in Atlanta to surrender the valuable biometric data of their faces.[1]
The promise of access to powerful decision-makers was undeniable. Yet there were other power dynamics to consider. Would my participation really make a difference, or was my inclusion solely for virtue signaling? In the end, Ethan advised that you don’t make change by talking only to the people who agree with you. He cautioned me to adjust my expectations, since there were many different tiers of access.
attachable snow spikes for shoes.
My overconfidence with my snowboarding abilities given my prior experience skateboarding was not unlike how some tech companies were launching AI products. Success in one area does not guarantee success in another. Using skateboarding techniques to attempt to snowboard left both my pride and my bottom sore. Launching immature and inappropriate products also affects the long-term bottom line of tech companies.
Optimizing a fairly constrained environment is not quite the same as working with an unconstrained environment. Early explorers of face detection techniques soon ran into the limit that the training data of people photographed in well-lit environments with set poses did not equip systems for unconstrained environments, also known as the real world, where most people do not walk around with studio-quality lighting. When researchers began collecting images of faces “in the wild,” candid images of people posted online to improve face detection systems, this was like moving from a nice indoor
...more
“Many people can gain tech skills. Your ability to communicate and tell a story about your technical work is what will separate you from your peers. It’s what makes people see why change is necessary. I don’t want to hurt your feelings. I want you to be ready for the headwinds.”
We also had our photos in the New York Times article that came out: “Amazon Is Pushing Facial Technology That a Study Says Could Be Biased,” by Natasha Singer.[5]
“I cannot pay like a tech start-up, but what I can teach you will set you up to go beyond me. Learn from all of us and be better.”
Unlike the reception I received from IBM, which made me feel change was possible, Amazon’s approach made me question the extent to which large corporations could be trusted.
They posted spirited messages on Twitter to rally behind both the research and me personally. We needed more.
So while Amazon was right that my studies had looked at gender classification and not facial verification or facial identification, I was correct in my assertion that the bias observed in my studies was cause for concern in other areas, given shared technical approaches used with a whole range of facial recognition technologies. Unlike Microsoft, Amazon did not submit its systems to be tested for this landmark study in 2019. By the time NIST’s vindication came in December, my illusion of safety had already dissolved. I had assumed basing my work at an academic institution like MIT would offer
...more
excoded
legalese.
amicus
Their system had been trained on Canadians who spoke English as their first language. When the system was tested on Canadians who spoke French as their first language, these speakers did not match the training data. The context collapse occurred because the signs being used to infer Alzheimer’s disease may also overlap with signals conveyed by someone searching for the right word or stringing together words in unusual ways in a second language.
Barbican.
For example, the accuracy of near-infrared facial recognition may be affected by the emotional and physical condition of the individual, which can be influenced by illness, alcohol, or exercise.[5] StoneLock’s corporate training grounds, workplace environments where alcohol consumption is limited if not banned, and which employees tend not to frequent when ill, may prove ill-matched to the variability introduced in a residential community.
At the same time, large errors in classification, using contestable labels, demonstrate how fallible AI systems—even those produced by leading companies—can be. Such errors disrupt facile assumptions of machine neutrality and technological objectivity.
We need artists who use their creativity to craft evocative pieces that humanize the impact of AI-driven mistakes. We need tenants who speak up against the installation of intrusive systems and take the initiative to learn more about the technologies we are often encouraged to accept without questioning. We need researchers who take the time to make work accessible and digestible so the greater public knows what is at stake and advocacy groups can take up that work to support successful campaigns. We also need to acknowledge the risks associated with speaking up against powerful entities. In
...more
I glanced over at Max, Megan Smith’s black-and-white cat. He looked uninterested but was nonetheless involved in my practice run before my first congressional testimony. Max was quite focused on licking his right paw as he unfurled near the swimming pool.
“Keep it bipartisan,” she advised. “Make sure everyone on the committee understands how it impacts their constituents.”
moot
Alvaro explained that moot hearing was the jargon they used to describe a practice hearing.
The privileged few were designing for the many with little regard for the harmful impact of their creations. The consequences continue to ripple. Michael Oliver, Nijeer Parks, Robert Williams, and Randall Reid were all arrested due to misidentification aided by automated facial recognition. Williams was wrongfully arrested in front of his two young daughters and held by law enforcement overnight.[1]
“Where are my keys?” “Not here,” said my glasses case. “Try later,” said my coat pocket. I still had not mended the hole in the lining.
With some hesitation, I thought about what she might show of me outside the lab. I settled on having her film the process of having my hair styled at a salon full of Black women: Simply Erinn’s. I wanted her to see who I spoke to outside the lab about my research and also where I drew a sense of broader community beyond academic circles.
sheepishly
My step to being more open on-screen started in the stylist’s chair, my hair as unfinished as the story Shalini was spinning. In that chair, I had no laptops or screens to hide behind. If the aim was for people to see me as more than a researcher, we would start at the roots.
Under the roar of a hair dryer, I shouted over to Shalini, “I hope this is intimate enough for you.”
What was apparent to me was not so apparent to others. I had to believe in myself in the face of doubt.
Unfortunately, if we wait to see immediate impact before we invest time, talent, treasure, and networks into supporting creative endeavors, they may never have the support they need to thrive.
The television station that had carried my nine-year-old imagination into MIT now showed me at MIT discussing artificial intelligence. Maybe there was another child seeing me and imagining new possibilities. Shortly after the PBS debut, Coded Bias was released on Netflix to the more than 200 million subscribers of the platform at the time and translated into thirty languages. The film received critical acclaim and numerous awards.
I wanted to say “thank you” for uncovering the truth of this while Algorithmic Bias is still in relatively early stages.
Letting my guard down on camera if even just partially is a risk I now see as having been worth it. I keep the messages I receive to motivate me when I find myself in moments I want to give up. The response to the film reminds me of the impact of Pete Souza’s photo of the young boy touching the head of President Obama.
“But all the time I’m spending on the PhD is time I am not spending on other responsibilities, and we just hired three new staff members at AJL and are currently in the process of hiring two more people. Then, I have media obligations in September. It’s too much!”
Generating Outstanding Awareness Tenaciously!
Dropping out of MIT in the long run might not impact me significantly in terms of job opportunities, as many thrive in the tech field and other spaces without graduate degrees, let alone doctoral degrees. Still, the stereotypes associated with dropping out as a young woman in STEM felt unfair given all the work I had already done.
I was annoyed by the notion that not being validated by an academic institution would be a potential scarlet letter.
The ironic stigma of being a dropout hung in my mind, but more important, the legacies of so many people who made this choice possible permeated my thoughts.
“The Look,” like the coded gaze, is a reflection of societal cues of who was deemed suspicious, who was assumed to have authority, and whom our children were taught to shun. The ad follows a day in the life of a Black man who is facing everyday yet no less painful discrimination as he is followed in a department store as if he could be a thief and distanced by workplace colleagues as an unwelcome presence. The final scene reveals he is in fact a judge, but even that standing in society when he leaves the bench of authority is no shield for microaggressions.
I thought about short sound bites to say during the interview to make it easier for everyday people to understand why the demonstrations mattered.
Though I was tired, if I did not speak up for myself I would be quietly complicit in this kind of treatment.
Kosiesem.
My ego was also bruised. I had gotten used to the spotlight and being among the leading voices when it came to issues around facial recognition technologies. Fast Company’s choice to put me on the cover of the “2020 Most Creative People in Business” issue under editor Stephanie Mehta was, I felt, a significant step in media representation, given how few Black people who were not professional athletes or entertainers were allowed such a spotlight. Looking at it also made me feel validated as I stared defiantly in a yellow jacket.
If news coverage depicts Black people only as victims, it perpetuates a harmful trope that we lack agency to make meaningful change.
Complete erasure is one way to “invisibilize” a group, yet inclusion that builds on stereotypical representation can also be harmful. It is not enough just to be seen if you and people like you are rendered in disempowering terms or through disempowering frames. By continuing to show up even when it hurts and even at the risk of erasure, I was combating symbolic annihilation. Now I knew I would need to be more strategic about where I committed my energy.
