A Hacker's Mind: How the Powerful Bend Society's Rules, and How to Bend them Back

by Bruce Schneier

One more hack of our attention circuits that occurs on modern social networks: manufacturing outrage. Facebook uses algorithms to optimize your feed. Its goal is to keep you on the platform—the more you’re on Facebook, the more ads you see, and the more money the company makes—so it tries to show you content that engages you on which to display ads. (Don’t forget this: the whole point of these systems is to sell advertising, whose whole point is to manipulate you into making purchases.) Similarly, Google wants to keep you watching YouTube videos. (YouTube is a Google subsidiary.) The YouTube algorithm figured out that increasingly polarizing and fringe content engages users. This is an unanticipated hack and very profitable for Google. Facebook and YouTube are polarizing not because they were originally intended to be, but because (1) the algorithm optimized itself to serve up ever more specialized content on the basis of a user’s interests, and (2) management chose to disregard the potential problems that this created. Applied to politics, such schemes tend to polarize users by making it easier for them to occupy an ideological bubble with others who share their worldview, and by prioritizing material that provokes the most excitement. By accelerating and fine-tuning the task of finding and displaying specialized, polarizing content, automated content recommendation systems reduce the less filtered interactions that force us to consider and reconsider our beliefs and opin...