Technically Wrong: Sexist Apps, Biased Algorithms, and Other Threats of Toxic Tech

by Sara Wachter-Boettcher

When you remember how few people change the default settings in the software they use, Facebook’s motivations become a lot clearer: Facebook needs advertisers. Advertisers want to target by gender. Most users will never go back to futz with custom settings. So, Facebook effectively designs its onboarding process to gather the data it wants, in the format advertisers expect. Then it creates its customizable settings and ensures it gets glowing reviews from the tech press, appeasing groups that feel marginalized—all the while knowing that very few people, statistically, will actually bother to adjust anything. Thus, it gets a feel-good story about inclusivity, while maintaining as large an audience as possible for advertisers. It’s a win-win . . . if you’re Facebook or an advertiser, that is.

You know when you try to shop online or read an article, but a little window pops up before you can even see what you’re looking at, trying to get you to sign up for the site’s email list? I’m sure you do, because you can barely use the internet without this happening. In tech, we call these kinds of pop-ups opt-ins, meaning you’re being asked to choose to receive marketing messages from a company. If you click the “no thanks” option, that’s an opt-out. But not only is the design intrusive and frustrating—Can I at least see your products before being badgered into daily emails?—there’s a new design trend that’s making them even worse: rather than tapping a button that says “no thanks,” sites are now making users click condescending, passive-aggressive statements to get the intrusive window to close: No thanks, I hate saving money. No thanks, this deal is just too good for me. I’m not interested in being awesome. I’d rather stay uninformed.

Huge disconnect between company design & what users want. Just because something is status quo doesn't mean it's effective.

they manipulate our emotions so that companies can collect our information, without actually doing any of the work of creating a relationship or building a loyal following.

time and again, the conversation turned away from how to make the program useful, and toward that word I find so empty: “delight.”

That’s why these companies are so shameless: They’re not really trying to build loyalty. All they want is data.

All these cutesy copy strings and celebratory features create a false intimacy between us and the products we use. We’re not actually friends with our digital products, no matter how great their personalities might seem at first. Real friends don’t create metrics to gauge whether people think they care. They don’t try to tell you jokes when you’re in the middle of a crisis. They don’t force you to relive trauma, or write off hate speech, or any of the things tech products routinely do in the name of engagement. They simply care. Tech companies, on the other hand, use “personality” to manipulate us—to keep us clicking and tapping and liking and reading and saving and faving, while, just outside the screen, they flout regulations, steal data, and keep oppressive systems intact. It’s time we see through them.