Surely TikTok isn't that bad.. is it? (2024)

If you don’t know the name of Eugene Wei, then I have a treat for you. (If you already do, you know why it’s a treat.) Wei gives his own CV on his Substack as “a former product executive at companies like Amazon, Hulu, Flipboard, and Oculus, with a stint in film school in between.” His blog Remains Of The Day has a number of fascinating posts, including why Amazon introduce Prime, giving you free shipping for an annual fee that might be less than you’d spend on individual shipping for all the items you bought (spoiler: because when transactions were abandoned, it was most frequently at the point where people were confronted with the shipping costs).

In August 2020, he wrote a blogpost there titled TikTok and the Sorting Hat:

I’ve been fascinated with TikTok. Here in 2020, TikTok is, for many, including myself, the most entertaining short video app going. The U.S. government is considering banning the app as a national security risk, and while that’s the topic du jour for just about everyone right now, I’m much more interested in tracing how it got a foothold in markets outside of China, especially the U.S. with its powerful incumbents.

They say you learn the most from failure, and in the same way I learn the most about my mental models from the exceptions. How did an app designed by two guys in Shanghai managed to run circles around U.S. video apps from YouTube to Facebook to Instagram to Snapchat, becoming the most fertile source for meme origination, mutation, and dissemination in a culture so different from the one in which it was built?

There’s a good chunk of history in there, but then he gets to the meat of how it is that TikTok works so well:

TikTok takes content from one group of people and matches it to other people who would enjoy that content. It is trying to figure out what hundreds of millions of viewers around the world are interested in. When you frame TikTok's algorithm that way, its enormous unrealized potential snaps into focus.

The idea of using a social graph to build out an interest-based network has always been a sort of approximation, a hack. You follow some people in an app, and it serves you some subset of the content from those people under the assumption that you’ll find much of what they post of interest to you.

…But what if there was a way to build an interest graph for you without you having to follow anyone? What if you could skip the long and painstaking intermediate step of assembling a social graph and just jump directly to the interest graph? And what if that could be done really quickly and cheaply at scale, across millions of users? And what if the algorithm that pulled this off could also adjust to your evolving tastes in near real-time, without you having to actively tune it?

Wei’s point is that that’s what TikTok does: it carefully watches your interests and rapidly matches you to what the algorithm decides you’ll find rewarding. Wei wrote a followup post, Seeing Like An Algorithm, which goes into this in greater detail; and then a third, American Idle, on the effect on creators. They’re all worth your time.

But the other question around TikTok now is the one Wei alluded to back in 2020: is this social network (which mostly dispenses with the “social” element in favour of “ignore who it comes from, just enjoy it”) a national security risk?

The facts are simple:

• TikTok is owned by a Chinese company, and Chinese companies have to do what they’re told by the ruling Chinese Community Party (CCP), if they’re told to do something. (And of course they’re not allowed to say they’ve been told do so something. In that respect, it’s the same as most Western companies being required to do something by intelligence agencies or the police.)

• TikTok is built on an algorithm, and algorithms are tuned to show some things and not others. Ben Thompson demonstrated back in July 2020 that TikTok was censoring content around the Hong Kong protests at the time, and nothing has changed since then except that TikTok has got bigger. So it’s obvious: TikTok’s algorithm is run to protect the CCP.

But is that a problem for the US? Well, TikTok made a big misstep the other day when the prospect of a law forcing the sale of the US entity to a US-owned company abruptly came on to the agenda. TikTok responded not with subtle lobbying, but by putting a modal on its American users’ screen:

“Congress is planning a total ban of TikTok”, it began. Which is straight-up untrue. As mentioned above, the proposal is that the TikTok Americans use wouldn’t be under the potential control of the CCP. “Speak up now—before your government strips 170 million Americans of their Constitutional right to free expression.” Again, not correct (though helpful to have the claimed user numbers: half of America?). Nobody’s stopping free expression, just stipulating the ownership of one of its channels. As to the final part, it’s hard to know how millions of businesses would be affected. “Countless creators” is, of course, an uncounted number. Could be millions! Could be five!

And finally, the tour de force: just enter your zip code and you can harass your Representative. (The bill started in the House of Representatives.) This led to a lot of puzzled Congress staff getting calls from high schoolers taking a break between lessons to complain about TikTok being—oh, that’s the bell, I gotta go.

Imagine being that Representative: in five years’ time, they might vote against you.

This is the most spectacular own goal for a company which doesn’t want people to believe that it could manipulate the population into doing things—in other words, that it doesn’t pose any sort of national security risk through its ability to encourage colossal numbers of people to take a particular action during an election year.

Yet it got worse: the next day, TikTok did another version of the same modal, but targeted (quite how wasn’t clear, because not everyone got it, and we don’t know what was chosen). String together the pieces: unknown algorithm from company ultimately controlled by the CCP urging citizens to take a specific political action favouring the company. If you’re a politician seeing all that, you’d be a bit foolish not to think it needs reining in.

And indeed, the strong (362 for, 65 against) support for the bill seems to have come after a national security briefing to Congress last week. What did they learn? We don’t know, and it’s not clear if we ever will.

By now you’re wondering: what’s the social warming aspect of this? I think it’s pretty clear: TikTok is incredibly good at sorting people into niches of interest, and finding material to feed those niches. It amplifies those interests, pulls the user in, keeps them inside the app. The algorithm is under someone else’s control; you can be sure it is tweaked and twiddled all the time. And it can be used against political interests the company—or nation state that ultimate controls it—doesn’t like. That’s not even mild social warming; that is an absolutely clear effect. The number of people who say they get their news from TikTok (the most popular source in the UK for 12-15 year-olds), the amount of time spent—nearly an hour per day on average, more than other social media apps—adds up to something that anyone sensible is going to look at sidelong. Look at the fact that India banned it long before it was big. That’s a country which knows the wildfire effects of social media, and doesn’t trust China as a neighbour.

When Facebook was the app that people fretted about, you did at least know that it was controlled by someone whose instincts were knowable; Mark Zuckerberg is the product of a comprehensible libertarian strand of American thinking, where the First Amendment is the big thing. When it’s TikTok, and China is eyeing up Taiwan, and would want Trump rather than Biden to be president, then you want to know the motivations of the people behind an app that demonstrates it can motivate people to act politically. The bill to force TikTok to sell its US interests makes sense; the question now is whether the US Senate will back it, though as I say, I’d think a politician who has seen its effect would be a fool not to. Biden, meanwhile, has said he will sign the bill if passed by both Houses.

I wonder if the UK will do the same. But given the way our politicians have only shown interest if the media coming under control of a foreign power is an old right-wing newspaper, maybe don’t get your hopes up.

Glimpses of the AI tsunami

(Of the what? Read here. And then the update.)

• A big pro-AI pre-roll at SXSW (“it’s going to disrupt you!”) was loudly and thoroughly booed.

• This video is very weird, but shows you where AI generation is going. Just give it the first minute until Greta Thunberg appears, and then it’s boosters on.

• Dammit, OpenAI’s recruiting tool shows the same bias as its training data. Can’t these machines just learn, but do it better?

• Princess Kate’s noodling on her family photo taken by her husband (though with professional lighting, according to professional photographers who have looked at it) means we are now firmly in the age of the deep fake, where we don’t trust what we see, says Jamie Bartlett.

• You can buy Social Warming in paperback, hardback or ebook via One World Publications, or order it through your friendly local bookstore. Or listen to me read it on Audible.

You could also sign up for The Overspill, a daily list of links with short extracts and brief commentary on things I find interesting in tech, science, medicine, politics and any other topic that takes my fancy.

• Back next week! Or leave a comment here, or in the Substack chat, or Substack Notes, or write it in a letter and put it in a bottle so that The Police write a song about it after it falls through a wormhole and goes back in time.