Unmasking AI: My Mission to Protect What Is Human in a World of Machines

by Joy Buolamwini

Looking at how machine learning models are being developed, we see the impact of what I first heard Kate Crawford term the “politics of classification.” Having the ability to define classification systems is in itself a power. The choices that are made are influenced by cultural, political, and economic factors, and while these classifications don’t have to be based on definite distinctions, they still have an impact on individual lives and societal attitudes. Despite the power shadows inherent in classification systems, often those systems go unchallenged. Instead, they are used as shorthand. Morgan Klaus Scheuerman and their colleagues have done tremendous scholarship on how gender classification systems lead to erasure, reification of social constructs, and the reinforcement of gender stereotypes.[5] In my research, I had to contend with understanding the limitations of classification systems we use as well as acknowledging those systems’ utility in showing discrimination, unfairness, or inequitable treatment. Yes, the classification systems were problematic, but in using them I could show their limitations as well as the limitations of the other systems I examined. To contend with those limitations, it is important to call out the assumptions of the classifications that are being used so there isn’t a universal acceptance that these are the ones that must be used or that they do not have problems.