“By November 2022, Snap employees discussed 10,000 user reports of sextortion each month and acknowledged that, given the shame and other barriers to reporting, these reports “likely represent only a small portion of this abuse,” a recently unsealed version of the lawsuit says filed by the New Mexico Attorney General against Snap. This less redacted version of the filing, which we first saw a month ago, contains new details about what Snap employees allegedly knew about the extent of the sextortion problem the company is said to have promoted on its platform.
In one alleged case, employees pointed to a case with 75 reports that “mentioned nudity, minors and blackmail, but the account was still active.” And in 2022, an internal investigation by Snap reportedly found that over a third of teenage girls and 30 percent of teenage boys on its app were exposed to “unwanted contact on its platform.” the complaint says.
The new details paint a picture of a company that is aware of its alleged shortcomings in protecting children on its services, but is not sufficiently focused on addressing them. “Former Snap trust and security employees complained that they had little contact with upper management compared to their work at other social media companies and that attempts to add in-app security mechanisms were met with resistance [Snap CEO] “Evan Spiegel prioritized design,” the complaint states.
“…the attempt to add in-app security mechanisms was met with resistance because [Snap CEO] Evan Spiegel prioritized design”
In a statement published in the newsroom, Snap said its app was “designed as a place to communicate with a close circle of friends, with safety guardrails built in, and has made conscious design decisions to make it harder for strangers to discover minors on our service.” We are constantly evolving our security mechanisms and policies, from using advanced technology to detect and block certain activities, to banning friendships with suspicious accounts, to cooperating with law enforcement and government agencies, and more.”
According to the complaint, in 2021, Snap employees distributed an external report that included examples of alleged predators engaging with children as young as eight through Snapchat and obtaining child sexual abuse material. However, they feared that measures to detect this type of behavior would place undue burden on users' privacy and “incur disproportionate administrative costs,” the complaint said.
Employees also reportedly identified risks with certain Snapchat features, such as Quick Add, which suggests other users connect with them. “We need to develop new approaches that shield our most vulnerable users (minors) and make it harder for predatory users to find them via quick add, search, etc.,” a manager wrote, according to the complaint. “We believe we can achieve this without significantly impacting the product experience for these users as we pursue new strategies in inventory generation/restrictions and other techniques to more effectively separate minors from those outside their networks,” Snap later added caused the Quick Add feature to only appear for accounts of 13- to 17-year-olds if they had “a certain number of mutual friends with that person.” But internally, the complaint says, employees recognized that the approach still had significant flaws.
The unsealed complaint also provides further details about how Snap allegedly facilitated the illegal sale of guns. In an undated presentation, the company acknowledged that its platform sees “50 posts per day related to illegal gun sales and 9,000 views of these marketed weapons per day.” And even when content is reported: “[r]Reported content is typically viewed hundreds of times before it is reported.”
“Stripes make it impossible to unplug for even a day.”
This includes internal communications that recognize Snapstreaks' addictive nature and let users know how many days they have been communicating with another user. “Wow, we should have more addictive features like this,” one employee allegedly wrote, according to a January 2017 email. “Most streakers are our core group,” wrote another. An October 2019 presentation reportedly said: “Streaks make it impossible to unplug for even a day.”