New law will punish social media companies for users’ violent content

In the wake of last month’s massacre at two mosques in New Zealand, Australia has passed sweeping legislation (PDF) that threatens huge fines for social media companies and jail time for their executives if they don’t promptly remove “abhorrent violent material” from their platforms.

Such content includes videos depicting terrorist acts, murders, attempted murders, torture, rape or kidnap.

The Guardian reports that Labor opposition joined forces with the ruling Liberal-National Coalition to pass the law on Thursday, despite warning that, in spite of what the government is promising, the legislation – the Sharing of Abhorrent Violent Material Act – won’t allow for prosecution of social media execs.

The legislation stipulates that criminal penalties apply to content service providers that have “reasonable grounds to believe” they host such prohibited content if they don’t give police a heads-up about it “within a reasonable time after becoming aware of the existence of the material” or if they fail to “ensure the expeditious removal of the material from the content service.”

The penalties for individuals who “provide a hosting service” and fail to remove material can be up to three years in jail, a AUD $2.1m fine, or both. Labor believes these penalties won’t apply to social media execs because their companies, and not the individuals themselves, provide the service.

Corporate penalties range up to AUD $10.5m or 10% of annual turnover. But again, Labor says there don’t appear to be provisions for the law to punish executives for the actions of others in their social media companies.

The tech industry is strongly opposed to the law: they believe it may criminalize anybody in their companies for failing to promptly remove violent content.

Australia’s attorney general Christian Porter considers the act “most likely a world first.” It was rapidly drafted – not a good thing, according to critics, particularly given that it was written without much input from technology companies or experts – following the Christchurch terrorist attack.

The New York Times quoted Porter during a House of Representatives debate on the bill on Thursday:

These platforms should not be weaponized for these purposes. Internet platforms must take the spread of abhorrent violent material online seriously.

Edited versions helped it spread like wildfire

The alleged gunman in that attack—in which 50 Muslim worshippers were murdered and 39 were wounded—is believed to be an Australian white nationalist. He allegedly live-streamed what’s thought to be the single worst terrorist attack carried out by an Australian, according to ABC.

Video of his alleged attack spread faster on social media than it could be taken down.

On Friday, Facebook finally broke its silence on the killings when the New Zealand Herald published a letter from Facebook Chief Operating Officer Sheryl Sandberg.

In her letter, the COO said that in spite of being shared Live, the video spread mainly by people who shared it. The difficulty of stopping the viral spread was compounded by the fact that so many people re-edited it, which makes it harder for Facebook’s systems to identify and block content. Sandberg said that Facebook has identified more than 900 different videos showing portions of “those horrifying 17 minutes.”

“Your silence is an insult to our grief.”

Facebook, for one, stayed mute for two weeks after the attack, while politicians fumed at the failure of such platforms to keep themselves from being used to disseminate manifestos of hate and videos of horror.

From a letter written by New Zealand privacy commissioner John Edwards to Facebook executives following the massacre, as reported by the New Zealand Herald:

It would be very difficult for you and your colleagues to overestimate the growing frustration and anger here at Facebook’s facilitation of and inability to mitigate the deep, deep pain and harm from the live-streamed massacre of our colleagues, family members and countrymen broadcast over your network.

Your silence is an insult to our grief.

Sandberg said in her letter that Facebook is exploring live-stream video restrictions that could factor in things such as an individual’s history of Community Standard violations. As well, Facebook is investing in research to build better technology that could quickly identify edited versions of violent videos and images and prevent people from re-sharing them.

Tech strenuously objects

The Digital Industry Group, an advocacy group representing Google, Facebook, Twitter, Amazon and Verizon Media in Australia, has warned that the bill was passed without adequate industry input and that it threatens penalties against tech companies for content created by users. Managing director Sunita Bose:

This law, which was conceived and passed in five days without any meaningful consultation, does nothing to address hate speech, which was the fundamental motivation for the tragic Christchurch terrorist attacks.

With the vast volumes of content uploaded to the internet every second, this is a highly complex problem that requires discussion with the technology industry, legal experts, the media and civil society to get the solution right. That didn’t happen this week.

Richard Di Natale, a senator in the Australian Greens party, blamed both Australia’s conservative government and the opposition Labor Party for “ramming through” legislation that encompasses “some of the most significant changes to social media online regulation that we have ever seen.”

The law is also being criticized for the fuzziness of the standard of removing content “expeditiously” – a vague time table that the law says would be decided by a jury …which could well lead to protracted legal battles.

Scott Farquhar, the chief executive of Atlassian – an Australian enterprise software company:

If the material in question is uploaded and you don’t take it down “expeditiously”, you can go to jail. What is expeditiously? Not defined! “Who” in a company? Not defined!

— Scott Farquhar (@scottfarkas) April 3, 2019

Farquhar also said that as written, the law could send anybody to jail for three years if they work at a company that allows user-generated content, including news sites, social media sites, dating sites, and job sites.

Despite the law’s making an exception for journalists whose jobs include reporting on abhorrent violence, even mainstream media are up in arms over the legislation, Farquhar pointed out. The Australian on Wednesday reported that a spokeswoman for its publisher, News Corp Australia, said that holding digital platforms liable for what they disseminate was overdue, but this bill goes well beyond that and could lead to censorship and criminalization of journalism:

While we have worked with the government to try to minimize the impact, this law risks criminalizing news reporting and provides significant powers to the eSafety Commissioner to take down news content.

She called for an immediate review of the legislation following the election:

In the rush to pass this law it will not be subjected to any review. Given the known consequences of the legislation on news reporting we strongly recommend the government and the opposition agree that the bipartisan Parliamentary Joint Security on Intelligence and Security review this law immediately after the election.