What do you think?
Rate this book
Behind the Screen: Content Moderation in the Shadows of Social Media
An eye-opening look at the invisible workers who protect us from seeing humanity’s worst on today’s commercial internet
Social media on the internet can be a nightmarish place. A primary shield against hateful language, violent videos, and online cruelty uploaded by users is not an algorithm. It is people. Mostly invisible by design, more than 100,000 commercial content moderators evaluate posts on mainstream social media enforcing internal policies, training artificial intelligence systems, and actively screening and removing offensive material—sometimes thousands of items per day.
Sarah T. Roberts, an award-winning social media scholar, offers the first extensive ethnographic study of the commercial content moderation industry. Based on interviews with workers from Silicon Valley to the Philippines, at boutique firms and at major social media companies, she contextualizes this hidden industry and examines the emotional toll it takes on its workers. This revealing investigation of the people “behind the screen” offers insights into not only the reality of our commercial internet but the future of globalized labor in the digital age.
Social media on the internet can be a nightmarish place. A primary shield against hateful language, violent videos, and online cruelty uploaded by users is not an algorithm. It is people. Mostly invisible by design, more than 100,000 commercial content moderators evaluate posts on mainstream social media enforcing internal policies, training artificial intelligence systems, and actively screening and removing offensive material—sometimes thousands of items per day.
Sarah T. Roberts, an award-winning social media scholar, offers the first extensive ethnographic study of the commercial content moderation industry. Based on interviews with workers from Silicon Valley to the Philippines, at boutique firms and at major social media companies, she contextualizes this hidden industry and examines the emotional toll it takes on its workers. This revealing investigation of the people “behind the screen” offers insights into not only the reality of our commercial internet but the future of globalized labor in the digital age.
280 pages, Hardcover
First published June 25, 2019
Ratings & Reviews
Friends & Following
Create a free account to discover what your friends think of this book!
Community Reviews
Displaying 1 - 11 of 11 reviews
June 23, 2019
Behind The Screen is a very superficial attempt to portray the lives of internet content moderators, those unfortunates who look at all the porn and the hate on social media, and give them a thumbs up or down. They are the maintenance crew of the internet. Browsing social media would be a horrific experience without them.
There are three levels of moderators, according to Sarah T. Roberts. One is to work directly at a company that does it. Or, work for a contractor that moderates for others, which is called outsourcing. Lastly, there is home work, where moderators are paid to stay away. Not examined are the lives of freelancers, who surf the internet on their own, looking for gigs that last mere seconds, and pay one cent or so. They add tags to photos, check or write translations, caption images, categorize and classify, and determine what is porn or hate. They are the bottom of the heap.
Even at the top of the heap, those working directly at a social platform, tend to have no benefits, can’t use the facilities everyone else can, and have zero chance of this being a foot in the door, even if they take on side projects. They are second class workers with no hope of advancement. When they burn out, they are disposed of.
Despite the emphasis on the moderators, it takes Roberts 75 pages to introduce one and even begin to describe the grind. She interviewed three moderators at one Silicon Valley firm. And while she documents conversations with them about the trivial hiring process and the lack of benefits, she never bothers to ask senior management why it has policies against moderators staying more than two years or having to take three months off before applying for a second one year stint, why this path is a dead end, or how they came to implement such a system.
There follows a pointlessly bland trip to Manila to see outsourcing in action, but Roberts doesn’t compare working conditions (offices, teams, managers, hours) or highlight differences in job functions. She doesn’t speak to top management there either, and doesn’t follow through to see the effects of it all. It’s mostly about adopting American language and culture so they don’t screw up on behalf of American users. She also speaks at length with a Canadian expat in Mexico who runs an outsourcing operation. His concerns are mostly cultural too.
The one home worker she interviewed in depth was justifiably fed up with all the hate she had to read all day.
Behind The Screen leaves no impression of the life of a moderator. The most impressive thing anyone says in the book is that a completed shift is “like you spent eight hours just in this hole of filth.” Max Breen was the moderator who said it, and he also told Roberts not to bother trying the work herself. So she didn’t.
Neither does she examine the state of the freelancer, who, paid one cent per decision, can make up to $40 for a 12 hour day if they’re really good. She spoke to no doctors about the PTSD-like effects of doing this work. Instead, there is a long discussion of bogus therapy sessions for all who work at the Silicon Valley firm, though workers cannot request direct attention to their own plight, since they have no healthcare. She also doesn’t tie the moderators to the greater disease of the gig economy of precarity, no benefits and repetitive assembly-line jobs, where workers are not even considered employees, even though they’re in the same building. (This is far from unique to content moderation.) Or how totally isolated home workers pathetically try to recreate an office atmosphere with chat functions the company provides. Or how they plot to create unions for better pay and conditions. At the very end of the book she manages to mention that some moderators at Microsoft and Facebook are making claims in court over their treatment by the companies.
The book was an exercise in frustration. Long descriptions of Manila and Silicon Valley were space fillers that added nothing to understanding what goes on behind the screen. The lack of depth and follow through was surprising, and not in a good way. Roberts likes to speak of herself as an expert at the center of the issue, but she never demonstrates that expertise. There are no recommendations other than studies are needed, and her five or six interviews don’t count. Roberts has no answers and nothing new for me to tell you here.
islavery is a very hot topic. If it seems I know as much as Roberts does on this topic, it’s not true. I don’t. I have simply read far better books on it.
David Wineberg
There are three levels of moderators, according to Sarah T. Roberts. One is to work directly at a company that does it. Or, work for a contractor that moderates for others, which is called outsourcing. Lastly, there is home work, where moderators are paid to stay away. Not examined are the lives of freelancers, who surf the internet on their own, looking for gigs that last mere seconds, and pay one cent or so. They add tags to photos, check or write translations, caption images, categorize and classify, and determine what is porn or hate. They are the bottom of the heap.
Even at the top of the heap, those working directly at a social platform, tend to have no benefits, can’t use the facilities everyone else can, and have zero chance of this being a foot in the door, even if they take on side projects. They are second class workers with no hope of advancement. When they burn out, they are disposed of.
Despite the emphasis on the moderators, it takes Roberts 75 pages to introduce one and even begin to describe the grind. She interviewed three moderators at one Silicon Valley firm. And while she documents conversations with them about the trivial hiring process and the lack of benefits, she never bothers to ask senior management why it has policies against moderators staying more than two years or having to take three months off before applying for a second one year stint, why this path is a dead end, or how they came to implement such a system.
There follows a pointlessly bland trip to Manila to see outsourcing in action, but Roberts doesn’t compare working conditions (offices, teams, managers, hours) or highlight differences in job functions. She doesn’t speak to top management there either, and doesn’t follow through to see the effects of it all. It’s mostly about adopting American language and culture so they don’t screw up on behalf of American users. She also speaks at length with a Canadian expat in Mexico who runs an outsourcing operation. His concerns are mostly cultural too.
The one home worker she interviewed in depth was justifiably fed up with all the hate she had to read all day.
Behind The Screen leaves no impression of the life of a moderator. The most impressive thing anyone says in the book is that a completed shift is “like you spent eight hours just in this hole of filth.” Max Breen was the moderator who said it, and he also told Roberts not to bother trying the work herself. So she didn’t.
Neither does she examine the state of the freelancer, who, paid one cent per decision, can make up to $40 for a 12 hour day if they’re really good. She spoke to no doctors about the PTSD-like effects of doing this work. Instead, there is a long discussion of bogus therapy sessions for all who work at the Silicon Valley firm, though workers cannot request direct attention to their own plight, since they have no healthcare. She also doesn’t tie the moderators to the greater disease of the gig economy of precarity, no benefits and repetitive assembly-line jobs, where workers are not even considered employees, even though they’re in the same building. (This is far from unique to content moderation.) Or how totally isolated home workers pathetically try to recreate an office atmosphere with chat functions the company provides. Or how they plot to create unions for better pay and conditions. At the very end of the book she manages to mention that some moderators at Microsoft and Facebook are making claims in court over their treatment by the companies.
The book was an exercise in frustration. Long descriptions of Manila and Silicon Valley were space fillers that added nothing to understanding what goes on behind the screen. The lack of depth and follow through was surprising, and not in a good way. Roberts likes to speak of herself as an expert at the center of the issue, but she never demonstrates that expertise. There are no recommendations other than studies are needed, and her five or six interviews don’t count. Roberts has no answers and nothing new for me to tell you here.
islavery is a very hot topic. If it seems I know as much as Roberts does on this topic, it’s not true. I don’t. I have simply read far better books on it.
David Wineberg
February 21, 2020
This book is valuable for being informative about an emerging occupational structure developing along with the spread of social media on the web. It concerns the job of commercial content moderation (CCM), which comprises in some variety of forms the individuals whose job it is to review videos and other posts that have been flagged as being objectionable and determine whether the posts comply with firm policies and standards or not, in which case they are removed. It is a job largely done by people rather than AI algorithms. These workers are most frequently not a part of the main workforce of the firm for whom the moderation occurs, but are rather temp and contract stringers, often working at home and often working at a great geographic distance of the focal firm. How many people do this? Who knows, but by the end of the book, it is clear that the lower bounds of this workforce is in the tens of thousands of workers.
Who knew this was a thing? It is clear that this work has to exist if one thinks about it at all. Ever been blasted in a chat room? Think about what it would be with no moderation. Ever try to post comments to newspaper and newsfeed stories? The discourse you see there has already passed muster - what would it be like without moderation? How many times have I taken the wrong turn on YouTube and found videos that were gross or downright creepy?
So far, so good. Now what is the book about? Thankfully, it is not filled with disgusting images or links. The first story line in the book is just what the job entails and what type of people work it. This is really interesting, such as the first firm, which hires recent elite liberal arts grads to work as contractors for a major Silicon Valley firm. Not only is the gross nature of the work a part of the job, but this is also clearly a new installment of the digital sweat shop work that is popping up all over the globe that pays next to nothing for micro tasks required on a 24/7 basis and which gravitate to the areas with the lowest pay scales. Welcome to the new economy - same as the old economy in terms of treating the least of us.
A second line of argument in the book is about the potential harm and long term distress (sort of a digital PTSD) that may befall these workers. The author states this potential and then repeats the claim at regular intervals in the book. There are two issues though. First, most of the people interviewed claim that they adjust to the difficult work just fine but have heard about problems. Second, there is no evidence cited to support the existence of such damages to workers. It is not surprising that interviewees would say they were OK. Since the occupation is a new one, it is reasonable to think that the long term effects have yet to be documented. Fine, but all we are left with are the suspicions that damage is being done and not much more.
A third line of development in the book is the range of contexts in which this work occurs. Some firms are contractors to major cites. Others are off-shored call-center like operations in Manilla. Another is a boutique firm tied in with PR functions for a firm seeking CCM to defend its brand. All of these overlap, of course, but it is really interesting the variety of business models that seem possible. I am still wondering why the CCM function is nearly universally outsourced in some form even though it is so important for the public persona of the firm. The author tries to address this towards the end of the book but I remain puzzled. Perhaps it is me.
I had some issues with the construction of the book. First, this is clearly a trade book, but it reads as if it is trying to be an academic book, in which all the relevant prior research must be mentioned. (I know, it is her dissertation project, but the publisher can also use developmental editors.) I would have preferred the author telling the clearest and most compelling story and leaving some of the references to notes or perhaps and appendix - or even to refereed journal articles. Along with this, the book in spots is a bit repetitious, repetitive, redundant, and reiterative. Finally, it is clear that the author has some affinities with economic sociology, which is fine, but after the nth reference to the oppressive nature of post-colonial globalizing capitalism, my senses started to dull. It wasn’t necessary. The extra syllables do not add that much to the impact of the book.
Overall, the information value of the book makes it work reading. I was not asked to moderate its content further.
Who knew this was a thing? It is clear that this work has to exist if one thinks about it at all. Ever been blasted in a chat room? Think about what it would be with no moderation. Ever try to post comments to newspaper and newsfeed stories? The discourse you see there has already passed muster - what would it be like without moderation? How many times have I taken the wrong turn on YouTube and found videos that were gross or downright creepy?
So far, so good. Now what is the book about? Thankfully, it is not filled with disgusting images or links. The first story line in the book is just what the job entails and what type of people work it. This is really interesting, such as the first firm, which hires recent elite liberal arts grads to work as contractors for a major Silicon Valley firm. Not only is the gross nature of the work a part of the job, but this is also clearly a new installment of the digital sweat shop work that is popping up all over the globe that pays next to nothing for micro tasks required on a 24/7 basis and which gravitate to the areas with the lowest pay scales. Welcome to the new economy - same as the old economy in terms of treating the least of us.
A second line of argument in the book is about the potential harm and long term distress (sort of a digital PTSD) that may befall these workers. The author states this potential and then repeats the claim at regular intervals in the book. There are two issues though. First, most of the people interviewed claim that they adjust to the difficult work just fine but have heard about problems. Second, there is no evidence cited to support the existence of such damages to workers. It is not surprising that interviewees would say they were OK. Since the occupation is a new one, it is reasonable to think that the long term effects have yet to be documented. Fine, but all we are left with are the suspicions that damage is being done and not much more.
A third line of development in the book is the range of contexts in which this work occurs. Some firms are contractors to major cites. Others are off-shored call-center like operations in Manilla. Another is a boutique firm tied in with PR functions for a firm seeking CCM to defend its brand. All of these overlap, of course, but it is really interesting the variety of business models that seem possible. I am still wondering why the CCM function is nearly universally outsourced in some form even though it is so important for the public persona of the firm. The author tries to address this towards the end of the book but I remain puzzled. Perhaps it is me.
I had some issues with the construction of the book. First, this is clearly a trade book, but it reads as if it is trying to be an academic book, in which all the relevant prior research must be mentioned. (I know, it is her dissertation project, but the publisher can also use developmental editors.) I would have preferred the author telling the clearest and most compelling story and leaving some of the references to notes or perhaps and appendix - or even to refereed journal articles. Along with this, the book in spots is a bit repetitious, repetitive, redundant, and reiterative. Finally, it is clear that the author has some affinities with economic sociology, which is fine, but after the nth reference to the oppressive nature of post-colonial globalizing capitalism, my senses started to dull. It wasn’t necessary. The extra syllables do not add that much to the impact of the book.
Overall, the information value of the book makes it work reading. I was not asked to moderate its content further.
April 25, 2023
This book is informative and mostly interesting. Three out of five because I wanted more of the interview material with the moderators. And, maybe I wanted more detail of the horrors they face in their work. I’m not sure I actually want that though.
The portions of the book focused on the business structures around moderation are slow to read. I do appreciate being able to compare those that worked directly on “Megatech’s” campus as contractors to those in remote locations like call centers. Despite both sets of workers being undervalued contractors and their work being similar, they have different perspectives. I can’t imagine those moderating U.S. posts from another culture can be as effective as those who live in the U.S. That said, moderating centers have flourished in the Philippines because they are steeped in U.S. pop culture.
The interviews with the workers, taking risks because of NDAs, are the most interesting parts of the book, including Roberts’s analysis of their discussions. The work seems horrible, and I think “sin eater” is an outstanding description. I imagine PTSD is a pretty common result of this work, which says a lot about humanity. I was surprised to learn that the moderators often feel helpless. They follow their protocols and report to law enforcement as necessary, but they usually don’t learn the result of their reporting, and if they don’t hear anything, it’s easy to assume nothing happened.
The fact that “Megatech” limited their contractors to one year stints with a mandatory break in between makes it clear that the horror of the work is acknowledged, and yet mental health is a secondary thought. I agree with the moderator who suggested required sessions with a therapist for all moderators.
Finally, the book’s conclusion is outstanding. Content moderation is finally acknowledged as a reality of the web, and as one moderator said, without the work, the internet would be a cesspool. Now that moderation is acknowledged, we need to appreciate their necessary, dangerous, and invisible work by providing them the physical health care, mental health care, pay, and recognition they deserve. Companies are making billions from the work of free content creators; the least we can do is move some of that money to those making it as safe as possible.
The portions of the book focused on the business structures around moderation are slow to read. I do appreciate being able to compare those that worked directly on “Megatech’s” campus as contractors to those in remote locations like call centers. Despite both sets of workers being undervalued contractors and their work being similar, they have different perspectives. I can’t imagine those moderating U.S. posts from another culture can be as effective as those who live in the U.S. That said, moderating centers have flourished in the Philippines because they are steeped in U.S. pop culture.
The interviews with the workers, taking risks because of NDAs, are the most interesting parts of the book, including Roberts’s analysis of their discussions. The work seems horrible, and I think “sin eater” is an outstanding description. I imagine PTSD is a pretty common result of this work, which says a lot about humanity. I was surprised to learn that the moderators often feel helpless. They follow their protocols and report to law enforcement as necessary, but they usually don’t learn the result of their reporting, and if they don’t hear anything, it’s easy to assume nothing happened.
The fact that “Megatech” limited their contractors to one year stints with a mandatory break in between makes it clear that the horror of the work is acknowledged, and yet mental health is a secondary thought. I agree with the moderator who suggested required sessions with a therapist for all moderators.
Finally, the book’s conclusion is outstanding. Content moderation is finally acknowledged as a reality of the web, and as one moderator said, without the work, the internet would be a cesspool. Now that moderation is acknowledged, we need to appreciate their necessary, dangerous, and invisible work by providing them the physical health care, mental health care, pay, and recognition they deserve. Companies are making billions from the work of free content creators; the least we can do is move some of that money to those making it as safe as possible.
June 18, 2021
I’ve read this book as research material for a play I’ve been commissioned to write by a Milan theatre company.
It is a bit wordy and repetitive but still rather revealing in that it sheds a light on a hidden aspect of our everyday interactions with the web and the social media.
It’s like the elephant in the room: every time we engage with a platform that is based on or includes user-generated content (anything from Facebook to the comment section of your newspaper of choice) we are really interacting with a universe made of algorithms and actual people (usually entry-level, low-paid outsourced staff working for the big companies but officially kept under the radar) who are in charge of viewing, filtering and taking down content deemed disturbing or inappropriate.
The psychological impacts of a job dealing with the darkest side of human instincts, wallowing in violence and perversions day in day out and their unfavourable contractual status are discussed in detail with a variety of examples taken from real interviews.
Another interesting aspect which is though given secondary relevance is the invisible hand of corporations controlling the kind of information that is allowed to seep through onto our screens, i.e. a sort of undeclared censorship policy depending mostly on the personal tastes and political view of social media platforms executives.
One important and often forgotten lesson to take home after reading this book is that every time we create an account on any site which includes our involvement as content creators, we are not in fact building a new home for our self-expression but renting a room in someone else’s mansion and therefore bound by their guidelines as rules of the house.
It is a bit wordy and repetitive but still rather revealing in that it sheds a light on a hidden aspect of our everyday interactions with the web and the social media.
It’s like the elephant in the room: every time we engage with a platform that is based on or includes user-generated content (anything from Facebook to the comment section of your newspaper of choice) we are really interacting with a universe made of algorithms and actual people (usually entry-level, low-paid outsourced staff working for the big companies but officially kept under the radar) who are in charge of viewing, filtering and taking down content deemed disturbing or inappropriate.
The psychological impacts of a job dealing with the darkest side of human instincts, wallowing in violence and perversions day in day out and their unfavourable contractual status are discussed in detail with a variety of examples taken from real interviews.
Another interesting aspect which is though given secondary relevance is the invisible hand of corporations controlling the kind of information that is allowed to seep through onto our screens, i.e. a sort of undeclared censorship policy depending mostly on the personal tastes and political view of social media platforms executives.
One important and often forgotten lesson to take home after reading this book is that every time we create an account on any site which includes our involvement as content creators, we are not in fact building a new home for our self-expression but renting a room in someone else’s mansion and therefore bound by their guidelines as rules of the house.
May 6, 2021
I read this for my dissertation, which is on a similar area but not quite as labour focused. As much as I enjoyed the content and research, and really liked the ethnographic descriptions of the workspace (I wanted to know even more, like what their offices looked like etc.) I felt like the writing was repetitive and required a good edit. It felt almost like there was a word count to achieve. However, the style of writing didn’t distract from the useful data and I did enjoy that side.
November 8, 2022
Read this one while I was buried in the sand. My brother always competes in the Coin Finders Championships which is essentially the Olympics for metal detector enthusiasts is what he told me and I'm always pissed because even though they allow doubles teams to compete, he won't do it with me just cause one time I stuck his detector on my ween area and said "beep beep beep must be my balls of steel." Even though he used to love our crude "for the boys" humor not even two or three years before that. Anyway this year, I figured I would ruin his chances in the contest by gluing a bunch of coins to myself and hiding in the beach where he was detecting and when he found me I would jump out and go "I am Coin Dad! What have you down with my babies?" I didn't get to though cause this guy made a shark robot that was running up and down the beach biting people so they cancelled the contest. The shark didn't bite me though, maybe cause I was so metaly it thought I was a fellow machine. This book was good.
December 23, 2019
Content moderators are the digital janitors and waste haulers of the 21st century, and this book has a tinge of Upton Sinclair's "The Slaughterhouse" in how it reveals how these low-paid workers are often traumatized and exploited. Hopefully it will help drive the public and policy-makers to make some needed changes in how we regulate Big Tech.
January 11, 2021
En bok om det dagliga arbetet som sker bakom kulisserna på sociala medier, där människor i mindre välbärgade länder arbetar hårt för att filtrera bort oönskade element från nätverken så att de inte riskerar att nå slutanvändarna. Människor som offrar sin mentala hälsa för att möjliggöra det kraftigt friserade internet vi har idag.
February 21, 2020
The writer explores the lives of those who address online content issues behind the scenes. The unseen hands (and eyes) that do, indeed, keep on top of content matters make them the invisible heroes -- or villains -- of the new age of communications.
October 29, 2021
Extremely important book imo
January 31, 2022
week 3 tute - good introductory read!
but wishing for more expansion of outsourced Filipino work
but wishing for more expansion of outsourced Filipino work
Displaying 1 - 11 of 11 reviews