Content Moderators and the Risk of Creating Local Bridges
https://www.nytimes.com/2021/08/31/technology/facebook-accenture-content-moderation.html
Content moderation on Facebook is essential for protecting many of its users. While the specifics of Facebook’s methods may be hotly debated (e.g. the 2021 U.S. Capitol attack), the people involved in carrying out the content moderation are often overlooked.
Accenture is one of the key companies Facebook relies on to handle the violent and/or hateful posts that slip past the AI filters. Workers then have to make a call – deleting posts that violate Facebook’s (rapidly changing) guidelines, or keeping posts and labeling them as “disturbing.”
But the nature of the work has created toxic working conditions for its employees. They must watch, read, and sift through incredibly horrible and distressing posts from all over the world (you can find more details in the article linked above). Some Accenture workers report starting to experience depression, anxiety, paranoia, and suicidal thoughts, all for typically low wages.
—
It’s appalling yet interesting to see how these Accenture workers have been impacted emotionally and mentally after being introduced to violent posts. Assuming that they are shocked because similar things don’t occur frequently in their own lives, I ended up thinking about the potential risk of novel sources of information.
When the employee is connected to the malicious Facebook user and their content, that connection becomes a local bridge over the structural hole between their communities. The moderator is a social gatekeeper now, beholding this disturbing information and able to reject the information from entering their community or able to introduce the information, perhaps by venting after a long day at work.
Considering how clusters tend to share values and norms, it makes sense that the worker could be shocked and possibly traumatized by content if it comes from another cluster with very different ones. Being part of a tightly knit community offers a sort of protection in that this allows for greater accountability within the group to keep to those social values and norms. However, when a person forms a local bridge to another person, they are putting themselves out there, and could be exposed to deleterious information.
This also raises questions about the impact of social media, especially on more vulnerable demographics. Do the benefits of social media outlets outweigh higher risks of exposing children and teens to unhelpful content?