Unmasking AI: My Mission to Protect What Is Human in a World of Machines

by Joy Buolamwini

Though the dataset was labeled by six skin types, to account for off-by-one errors, I grouped skin types I–III, which covered mainly the parliament members from Iceland, Finland, and Sweden, as the lighter-skinned group. I categorized skin types IV–VI as the darker-skinned group, which covered many of the parliament members in Senegal, Rwanda, and South Africa. Taking this step of phenotypic analysis at the time was a significant contribution to the field of computer vision. Phenotypic analysis demonstrated not just additional ways of understanding gender classifiers like the ones I was testing from these commercial products, but also how we might understand other areas of computer vision. For example, subsequent studies showed that pedestrian tracking systems were more likely to miss detecting people with darker skin than those with lighter skin. When applied to self-driving cars, the result would be that people with darker skin were at increased risk of being hit by a car with automated driving capabilities fully engaged.