I'm Starting to Worry About This Black Box of Doom

by Jason Pargin

That’s because they’d stumbled across a world-changing concept: applying a numerical value to human behavior that had never before been quantifiable. Up until then, for all human history, any individual could lie to themselves, could secretly believe they were more attractive than they are, or smarter, or more creative, or nicer, or richer. Or, and this is the big one, that their beliefs were popular. The Forbidden Numbers strip all that away; your true spot on the social hierarchy is revealed for all to see, in the form of likes and followers. It doesn’t matter how comfortable or well-fed somebody is; if you humiliate them in front of their peers, they’ll want to burn the system to the ground. Well, social media algorithms are a twenty-four-seven humiliation machine. That, Phil believed, is how a population is primed for authoritarian rule. And that’s just one example; we’re essentially teaching machines how to hack human insecurity.”