Email

But none of those constraints apply when it comes to platforms’ content moderation. 

Because of how broad the First Amendment is in scope, and the fact that it only protects against government censorship, it’s incumbent upon TikTok to develop robust community standards backed by an enforceable terms of service agreement with its users, the experts say. 

Even so, thinking about content moderation as a simple pass/fail test misses the point, Haupt says, adding that it’s about “calibration.” Wihbey agreed, adding that there are layers of nuance associated with successful content moderation policy. But above all, transparency is the key value, he says. 

Here is more from our conversation with Wihbey, who has written extensively about social media policy and content moderation.

What do you make of these calls for greater oversight of TikTok in this case? What current regulations are there?

For some time now there have been folks in Congress and elsewhere who have been advocating for subjecting TikTok to a lot more critical scrutiny — maybe even banning the app altogether. And we’ve discussed that before — the fact it has sort of gone nowhere, in part because it’s not clear such a ban would be legal, but also whether it would be sensible. Just given TikTok’s enormous importance in the media ecosystem, it would be extremely unwise to try to ban it outright under some flimsy pretext.

That said, I do think that a number of companies — TikTok included, but also Twitter/X, which kind of helped to inflame the whole bin Laden letter situation — should be releasing a lot more data about how they do business and how they curate and boost content, and what kinds of content is curated and boosted. And there has been some reporting that shows that companies like TikTok have some idiosyncratic or subjective ways of boosting content. In my view, there’s a good public interest rationale that they should be releasing a lot more data, they should be a lot more transparent about how they do that.

The United States remains an outlier in that we have no real mechanisms on the books to force any kind of transparency or to provide any policy guidance on how they should run these platforms. That stands in contrast to our allies in Europe who have passed the Digital Services Act, which is now on the books and is increasingly going to be influential and how technology platforms do business. It stands in contrast to India, which has a bunch of rules under their information technology laws that are quite influential, and then a whole bunch of other countries. So the United States continues to take a very laissez faire approach.

How do you see the reaction to this letter? Should the public be concerned about it spreading?

In terms of this particular letter, it was widely read a year after the 9/11 terrorist attacks. It was a piece of terrorist propaganda; it was widely discussed and dissected and was known to most news consumers. In some ways, it is a historical document.

There’s a risk here of confusing a couple different things. One is TikTok has Chinese ownership in that the parent company is Chinese, and we should bring lots of critical scrutiny to companies that are foreign-owned and have wide influence in the United States. That’s perfectly appropriate under U.S. law.

But we should do the same thing with our own U.S.-headquartered companies, such as X and YouTube and Instagram and the rest of them. But I think the fact that the Osama bin Laden letter blew up on TikTok does create an optics problem for TikTok — and it’s good that they sort of swung into action and tried to mitigate some of the harms there.

But at the same time, it’s not clear to me what the harms are, given that this letter is well known and has been part of public discourse for a long time. Of course, Hamas’s attack on Israel has created a heightened situation of tension and sensitivity around anything that is antisemitic in nature; and so what’s kind of interesting to me, from a company policy perspective, is the way in which context feeds into content moderation and decision-making. We tend to think of content moderation as having clear rules and strict enforcement. But it’s true that context matters, and the same piece of information in one context may have much greater weight or much less weight.

In your opinion, how should TikTok have handled this?

This has to be a judgment call on behalf of the human beings running TikTok. And I think it’s entirely appropriate for them to think about the various soft tools of moderation that can be used here, such as down-ranking or circuit breaking on virality — that kind of thing.

I’m not quite sure about an outright ban of a historical document. You could say, maybe we are in an extreme moment where because of the rise and dangers of anti-semitism, this particular incident requires special care. I think that’s a plausible argument and this is a private company, so it’s certainly within their discretion to act. And so I think that the appropriate tools here are the algorithmic tools: demotion, down-ranking, circuit-breaking of virality — sort of minimizing the reach of what some call ‘lawful but awful’ speech.

Tanner Stening is a Northeastern Global News reporter. Email him at t.stening@northeastern.edu. Follow him on X/Twitter @tstening90.