When we look at proclamations about the performance of AI systems, we must have insight into the types of tests that were used to reach the conclusion, the data collection methods, the labels used, and who was involved in decision-making. Yet as I went through each face in my Pilot Parliaments Benchmark to determine gender and skin type, the subjectivity of the exercise became more apparent. We cannot ignore this subjectivity as more AI systems are introduced into our lives, and we need to push for high standards for AI deployers to support the claims they make about what their systems can do.
...more
