A.W. Ohlheiser | Vox

Misinformation is winning the war on misinformation

2024-06-20T15:27:35-04:00

Online misinformation about everything from elections to vaccines still finds audiences online. | Chris Delmas/AFP via Getty Images

Misinformation on the internet has never been worse. Or at least that’s my analysis of it, based on vibes.

People on TikTok are eating up videos saying a bunch of inaccurate things about the dangers of sunscreen, while the platform’s on-app Shop propels obscure books containing bogus cures for cancer onto the Amazon bestseller list. Meanwhile, the presumed Republican nominee for president is fresh off what appears to be a successful push to neuter efforts to address disinformation campaigns about the election. Also, Google’s AI Overview search results told people to put glue on pizza.

But this is all anecdotal. Can I prove my hunch with data? Sadly, no. The data I — or more accurately, researchers with actual expertise on this — would need to do that is locked behind the opaque doors of the companies that run the platforms and services on which the internet’s worst nonsense is hosted. Evaluating the reach of misinformation, in the present day, is a grueling and indirect process with imperfect results.

For my final newsletter contribution, I wanted to find a way to assess the state of misinformation online. As I’ve been covering this topic over and over for the past while, there’s one question that keeps popping into my head: Do companies like Google, Meta, and TikTok even care about meaningfully tackling this problem?

The answer to this question, too, is imperfect. But there are some things that might lead to an educated guess.

Ways to measure misinformation are disappearing

One of the most important things a journalist can do while writing about the spread of bad information online is to find a way to measure its reach. There’s a huge difference between a YouTube video with 1,000 views, and one with 16 million, for instance. But lately, some of the key metrics used to put supposedly “viral” misinformation into context have been disappearing from public view.

TikTok disabled view counts for popular hashtags earlier this year, shifting instead to simply showing the number of posts made on TikTok using the hashtag. Meta is shutting down CrowdTangle, a once-great tool for researchers and journalists looking to closely examine how information spreads across social media platforms, in August. That’s just a couple months before the 2024 election. And Elon Musk decided to make “likes” private on the platform, a decision that, to be fair, is bad for accountability but could have some benefits for normal users of X.

Between all this and declining access to platform APIs, researchers are limited in how much they can really track or speak to what’s going on.

“How do we track things over time? Apart from relying on the platform’s word,” said Ananya Sen, an assistant professor of information technology and management at Carnegie Mellon University, whose recent research looks at how companies inadvertently fund misinformation-laden sites when they use large ad tech platforms.

Disappearing metrics is basically the opposite of what a lot of experts on manipulated information recommend. Transparency and disclosure are “key” components of reform efforts like the Digital Services Act in the EU, said Yacine Jernite, machine learning and society lead for Hugging Face, an open-source data science and machine learning platform.

“We’ve seen that people who use [generative AI] services for information about elections may get misleading outputs,” Jernite added, “so it’s particularly important to accurately represent and avoid over-hyping the reliability of those services.”

It’s generally better for an information ecosystem when people know more about what they’re using and how it works. And while some aspects of this fall under media literacy and information hygiene efforts, a portion of this has to come from the platforms and their boosters. Hyping up an AI chatbot as a next-generation search tool sets expectations that aren’t fulfilled by the service itself.

Platforms don’t have much incentive to care

Platforms aren’t just amplifying bad information, they’re making money off it. From TikTok Shop purchases to ad sales, if these companies take meaningful, systemic steps to change how disinformation circulates on their platforms, they might work against their business interests.

Social media platforms are designed to show you things you want to engage with and share. AI chatbots are designed to give the illusion of knowledge and research. But neither of these models are great for evaluating veracity, and doing so often requires limiting the scope of a platform working as intended. Slowing or narrowing how a platform like this works means less engagement, which means no growth, which means less money.

“I personally can’t imagine that they would ever be as aggressively interested in addressing this as the rest of us are,” said Evan Thornburg, a bioethicist who posts on TikTok as @gaygtownbae. “The thing that they’re able to monetize is our attention, our interest, and our buying power. And why would they whittle that down to a narrow scope?“

Many platforms begrudgingly began efforts to take on misinformation after the 2016 US elections, and again at the beginning of the Covid pandemic. But since then, there’s been kind of a pullback. Meta laid off employees from teams involved with content moderation in 2023, and rolled back its Covid-era rules. Maybe they’re sick of being held responsible for this stuff at this point. Or, as technology changes, they see an opportunity to move on from it.

So do they care?

Again, it’s hard to quantify the efforts by major platforms to curb misinformation, which leaves me leaning once again on informed vibes. For me, it feels like major platforms are backing away from prioritizing the fight against misinformation and disinformation, and that there’s a general kind of fatigue out there on the topic more broadly. That doesn’t mean that nobody is doing anything.

Prebunking, which involves preemptively fact-checking rumors and lies before they gain traction, is super promising, especially when applied to election misinformation. Crowdsourced fact-checking is also an interesting approach. And to the credit of platforms themselves, they do continue to update their rules as new problems emerge.

There’s a way in which I have some sympathy for the platforms here. This is an exhausting topic, and it’s tough to be told, over and over, that you’re not doing enough. But pulling back and moving on doesn’t stop bad information from finding audiences over and over. While these companies assess how much they care about moderating and addressing their platform’s capacity to spread lies, the people targeted by those lies are getting hurt.

Your social media diet is becoming easier to exploit

2024-06-12T16:11:52-04:00

It’s weird and often a bit scary to work in journalism right now. Misinformation and disinformation can be indistinguishable from reality online, as the growing tissues of networked nonsense have ossified into “bespoke realities” that compete with factual information for your attention and trust. AI-generated content mills are successfully masquerading as real news sites. And at some of those real news organizations (for instance, my former employer) there has been an exhausting trend of internal unrest, loss of confidence in leadership, and waves of layoffs.

The effects of these changes are now coming into focus. The Pew-Knight research initiative on Wednesday released a new report on how Americans get their news online. It’s an interesting snapshot, not just of where people are seeing news — on TikTok, Instagram, Facebook, or X — but also who they’re trusting to deliver it to them.

TikTok users who say they regularly consume news on the platform are just as likely to get news there from influencers as they are from media outlets or individual journalists. But they’re even more likely to get news on TikTok from “other people they don’t know personally.”

And while most users across all four platforms say they see some form of news-related content regularly, only a tiny portion of them actually log on to social media in order to consume it. X, formerly Twitter, is now the only platform where a majority of users say they check their feeds for news, either as a major (25 percent) or minor (40 percent) reason for using it. By contrast, just 15 percent of TikTok users say that news is a major reason they’ll scroll through their For You page.

The Pew research dropped while I was puzzling through how to answer a larger question: How is generative AI going to change media? And I think the new data highlights how complicated the answer is.

There are plenty of ways that generative AI is already changing journalism and the larger information ecosystem. But AI is just one part of an interconnected series of incentives and forces that are reshaping how people get information and what they do with it. Some of the issues with journalism as an industry right now are more or less own goals that no amount of worrying about AI or fretting about subscription numbers will fix.

Here are some of the things to look out for, however:

AI can make bad information sound more legit

It’s hard to fact-check an endless river of information and commentary, and rumors tend to spread much faster than verification, especially during a rapidly developing crisis. People turn to the internet in those moments for information, for understanding, and for cues on how to help. And that frantic, charged search for the latest updates has long been easy to manipulate for bad actors who know how to do it. Generative AI can make that even easier.

Tools like ChatGPT can mimic the voice of a news article, and the technology has a history of “hallucinating” citations to articles and reference material that doesn’t exist. Now, people can use an AI-powered chatbot to essentially cloak bad information in all the trappings of verified information.

“What we’re not ready for is the fact that there are basically these machines out there that can create plausible sounding text that has no relationship to the truth,” Julia Angwin, the founder of Proof News and a longtime data and technology journalist, recently told the Journalists Resource.

“For a profession that writes words that are meant to be factual, all of a sudden you’re competing in the marketplace — essentially, the marketplace of information — with all these words that sound plausible, look plausible and have no relationship to accuracy,” she noted.

A flood of plausible-sounding text has implications beyond journalism, too. Even for people who are pretty good at determining whether an email or an article is trustworthy or not, AI-generated text might mess with nonsense radars. Phishing emails and reference books — not to mention photography and video — are already fooling people with AI-generated writing.

AI doesn’t understand jokes

It didn’t take very long for Google’s AI Overview tool, which generates automated responses to search queries right on the results page, to start creating some pretty questionable results.

Famously, Google’s AI Overview told searchers to put a little glue on pizza to make the cheese stick better, drawing from a joke answer on Reddit. Others found Overview answers instructing searchers to change their blinker fluid, referencing a joke that’s popular on car maintenance forums (blinker fluid does not exist). Another Overview answer encouraged eating rocks, apparently because of an Onion article. These errors are funny, but AI Overview isn’t just falling for joking Reddit posts.

Google’s response to the Overview issues said that the tool’s inability to parse satire from serious answers is partially due to “data voids.” That’s when a specific search term or question doesn’t have a lot of serious or informed content written about it online, meaning that the top results for a related query will probably be less reliable. (I’m familiar with data voids from writing about health misinformation, where bad results are a real problem.) One solution to data voids is for there to be more reliable content about the topic at hand, created and verified by experts, reporters, and other people and organizations who can provide informed and factual information. But as Google sweeps up more and more eyeballs to internal results, rather than external sources, the company’s also removing some incentives for people to create that content in the first place.

Why should a non-journalist care?

I worry about this stuff because I am a reporter who has covered information weaponization online for years. This means two things: I know a lot about the spread and consequences of misinformation and rumor, and I make a living by doing journalism and would very much like to continue to do that. So of course, you might say, I care. AI might be coming for my job!

I’m a little skeptical of the idea that generative AI, a tool that does not do original research and doesn’t really have a good way of verifying the information it does surface, will be able to replace a practice that is, at its best, an information-gathering method that relies on doing original work and verifying the results. When they’re used properly and that use is disclosed to readers, I don’t think these tools are useless for researchers and reporters. In the right hands, generative AI is just a tool. What generative AI can do, in the hands of bad actors and a phalanx of grifters — or when deployed to maximize profit without regard for the informational pollution it creates — is fill your feed with junky and inaccurate content that sounds like news but isn’t. Although AI-generated nonsense might be posing a threat to the media industry, journalists like me aren’t the target for it. It’s you.

Why lying on the internet keeps working

2024-06-05T16:55:19-04:00

Conservative demonstrators who allege that the government pressured or colluded with social media platforms to censor right-leaning content under the guise of fighting misinformation protest outside the US Supreme Court in Washington, DC, March 18, 2024. | Saul Loeb/AFP via Getty Images

About a month ago, I wrote about a viral book of “Lost” herbal remedies that had, at the time, sold 60,000 copies on the TikTok Shop despite appearing to violate some of the app’s policies on health misinformation. The book’s sales were boosted by popular videos from wellness influencers on the app, some of which had millions of views, who claimed inaccurately that the once obscure 2019 book contained natural cures for cancer and other ailments.

The influencers, along with TikTok, made money off the sale of this misleading book. I brought all this to the attention of TikTok. The videos I flagged to a company spokesperson were removed after a review for violating TikTok’s policies banning health misinformation.

The book remained for sale in the shop, and new influencers stepped in. Nonetheless, I haven’t stopped seeing TikTok Shop promotions for this book, The Lost Book of Herbal Remedies, since.

“This right here is the reason they’re trying to ban this book,” said one TikTok Shop seller’s video, as he pointed to the book’s list of herbal cancer treatments. Later, he urged his viewers to click through on a link to the Shop listing and buy right away because “it probably won’t be around forever because of what’s inside.”

The video got more than 2 million views in two days. Click through the link as instructed and you’ll see that sales for the book have doubled since my article came out. The Lost Book of Herbal Remedies has sold more than 125,000 copies through the TikTok Shop’s e-commerce platform on TikTok alone. The book’s popularity doesn’t stop there, though: as of June 5, it is the No. 6 bestselling book on Amazon and has been on Amazon’s bestseller list for seven weeks and counting.

The “Invisible Rulers” of online attention

I was thinking about my experience digging into the The Lost Book of Herbal Remedies while reading the forthcoming book Invisible Rulers, by Stanford Internet Observatory researcher Renee DiResta. The book examines and contextualizes how bad information and “bespoke realities” became so powerful and prominent online. She charts how the “collision of the rumor mill and the propaganda machine” on social media helped to form a trinity of influencer, algorithm, and crowd that work symbiotically to catapult pseudo-events, Twitter Main Characters, and conspiracy theories that have captured attention and shattered consensus and trust.

DiResta’s book is part history, part analysis, and part memoir, as it spans from pre-internet examinations of the psychology of rumor and propaganda to the biggest moments of online conspiracy and harassment from the social media era. In the end, DiResta applies what she’s learned in a decade of closely researching online disinformation, manipulation, and abuse, to her personal experience of being the target of a series of baseless accusations that, despite their lack of evidence, prompted Rep. Jim Jordan, as chair of the House subcommittee on Weaponization of the Federal Government, to launch an investigation.

There’s a really understandable instinct that, I think, a lot of people have when they read about online misinformation or disinformation: They want to know why it’s happening and who is to blame, and they want that answer to be easy. Hence, meme-ified arguments about “Russian bots” causing Trump to win the presidential election in 2016. Or, perhaps, pushes to deplatform one person who went viral by saying something wrong and harmful. Or the belief that we can content-moderate our way out of online harms altogether.

DiResta’s book explains why these approaches will always fall short. Blaming the “algorithm” for a dangerous viral trend might feel satisfying, but the algorithm has never worked without human choice. As DiResta writes, “virality is a collective behavior.” Algorithms can surface and nudge and entangle, but they need user data to do it effectively.

Parables, panics, and prevention

Writing about individual viral rumors, conspiracy theories, and products can sometimes feel like telling parables: The Lost Book of Herbal Remedies becomes instructive on the ability of anything to become a TikTok Shop bestseller, so long as the influencers pushing the product are good enough at it.

Most of these parables in the misinformation space do not have neat or happy endings. Disinformation reporter Ali Breland, in his final piece for Mother Jones, wrote about how QAnon became “everything.” To do so, Breland begins with the parable of Wayfair, the cheap furniture seller that became the center of a moral panic about pedophiles.

This moment in online panic history, which also features heavily in DiResta’s book, happened in the summer of 2020, after many QAnon influencers and activity hubs had been banned from mainstream social media (which, incidentally, I interviewed DiResta about at the time for a piece questioning whether such a move happened too late to have any meaningful effect on QAnon’s influence).

Here’s what happened: Somebody online noticed that Wayfair was selling expensive cabinets. The cabinets had feminine names. The person drew some mental dots and connected them: surely, these listings must be coded evidence of a child trafficking ring. The idea caught fire in QAnon spaces and quickly spread beyond the paranoia enclaves. The wild and debunked idea co-opted a real hashtag used to raise awareness about actual human trafficking, which interfered with real investigations.

Breland, in his Mother Jones piece, tracks how the central tenets of the QAnon conspiracy theory stretched way beyond its believers and stayed there. Now, “[W]e are in an era of obsessive, odd, and sprawling fear of pedophilia—one where QAnon’s paranoid thinking is no longer bound to the political fringes of middle-aged posters and boomers terminally lost in the cyber world,” he wrote.

The Wayfair moral panic didn’t become a trend simply because of bad algorithms; it was evidence that the attention QAnon had grabbed previously had worked. Ban its hashtags and its influencers, but the crowd remained, and we were, to some degree, in it.

The Lost Book of Herbal Remedies became a bestseller by flowing through some well-worn grooves. The influencers promoting it knew what they could and couldn’t say from a moderation standpoint, and when those who broke the rules were removed, new influencers stepped up to earn those commissions. My article, and my efforts to bring this trend to the attention of TikTok, didn’t really do anything to slow the demand for this inaccurate book. So, what would work?

DiResta’s ideas for this echo conversations that have been happening among misinformation experts for some time. There are some things platforms absolutely should be doing from a moderation standpoint, like removing automated trending topics, introducing friction to engaging with some online content, and generally giving users more control over what they see in their feeds and from their communities. DiResta also notes the importance of education and prebunking, which is a more preventative version of addressing false information that focuses on the tactics and tropes of online manipulation. Also, transparency.

Would people be more likely to believe that there’s not a vast conspiracy to censor conservatives on social media if there was a public database of moderation actions from platforms? Would people be less enthusiastic to buy a book of questionable natural cures if they knew more about the commissions earned by the influencers promoting it? I don’t know. Maybe!

I do know this, though: After a decade of covering online culture and information manipulation, I don’t think I’ve ever seen things as bad as they are now. It’s worth trying, at least, something.

Congress’s online child safety bill, explained

2024-05-29T16:23:33-04:00

The bill, with versions in the House and Senate, would require online platforms to take actions to protect the safety of users under the age of 17. | Nikolas Kokovlis/NurPhoto via Getty Images

It’s tough to feel urgency about something that progresses in slow motion. Bear with me, though, because it is time, once again, to care about the Kids’ Online Safety Act, otherwise known as KOSA, a federal bill that was designed to protect children from online harms.

The bill has been hanging around in Congress in some form since 2022, when Sens. Richard Blumenthal (D-CT) and Marsha Blackburn (R-TN) introduced their bipartisan response to a series of congressional hearings and investigations into online child safety. While KOSA’s specific provisions have changed in the years since, the central goal of the legislation remains the same: legislators want to make platforms more responsible for the well-being of kids who use their services, and provide tools to parents so that they can manage how younger people use the internet.

The dangers posed to minors by the internet has long been simultaneously a real threat and a moral panic. It’s a political issue that has bipartisan support, while also appearing to be extremely difficult to govern without infringing on First Amendment protections.

KOSA was born after Facebook whistleblower Frances Haugen revealed, among other things, that Meta had evidence its platforms were harming the mental health of teens, and did nothing to mitigate those harms (Facebook has previously said that they believe Haugen’s claims are misleading). The environment in which the bill’s sponsors sought support, however, is rife with evidence of how such legislation might be misused for partisan reason.

The conservative think tank Heritage Foundation has said directly that they would seek to use measures like KOSA to restrict access to content about sexual and gender identity online. And while revised versions of the bill seek to address this concern, Fight for the Future, a digital rights advocacy group, has gathered a coalition of organizations that believe the current version of the bill still leaves LGBTQ+ youth vulnerable to censorship and harm, by limiting self expression and cutting off minors from access to information.

So, what is KOSA, exactly?

Here’s why we’re talking about KOSA now: The latest Senate version of the bill has enough votes to pass. And recently, legislators in the House introduced their own version of the bill, which differs in some ways from the Senate version, but is on track to go before the full Energy and Commerce committee in June.

The House bill is progressing alongside another privacy measure that more generally addresses data security standards. The two KOSA bills have bipartisan support, and follow a successful push to pass a law that could ban the short-video platform TikTok.

Both KOSA bills aim to achieve their goals by requiring the following:

Online services covered by the bill would need to take measures to prevent harm to users under the age of 17. The House and Senate bills have different definitions of the platforms and harms to which this provision would apply. Both have language requiring platforms to mitigate harm related to certain mental health disorders, compulsive social media usage, physical violence, sexual exploitation, and drug use.

Covered sites would have to introduce limitations into the design of their platform on how minors use it. For instance, KOSA would require platforms to limit the ability of other users to communicate with minors, limit personalized recommendation features for minors, limit features that encourage minors to spend more time on the app — including infinite scrolling and auto plays, features that are characteristic of TikTok’s For You page and widely imitated by other social media platforms.

These platforms would also need to offer parental tools that allow management of a minor user’s privacy, ability to purchase in-app items, and time spent on the platform. Platforms would also need to have a reporting system specifically for content that may cause harm to a minor.

The downsides of prioritizing online safety

If it passes — and that’s still a big if — KOSA would be the first major reform to rules governing online child safety since the Children’s Online Privacy Protection Act (COPPA), a 1998 law that regulates how a wide range of sites must handle information collected on users under 13. While COPPA does allow companies to collect information on these users with parental consent, the rules have, practically speaking, led many major platforms to simply ban users under 13 from having an account at all.

No matter the intent, however, many privacy advocates are skeptical of KOSA. While some recent changes won over the support of a few national organizations, the measure has struggled to gain support from LGBTQ+ organizations, who are concerned that the provisions could be used to restrict younger people’s access to resources about their identity. And while KOSA has undergone a few major revisions to address those fears, not all advocates are convinced.

The ACLU remains skeptical of KOSA, for instance. In a statement earlier this year, the civil liberties organization said that the bill would still harm the First Amendment rights of adults by incentivizing the removal of anonymous browsing on wide swaths of the internet, and by encouraging platforms to “censor protected speech” in order to ensure compliance with the bill’s provisions. Likewise, the Electronic Frontier Foundation has called the revised KOSA measure an “unconstitutional censorship bill” that would provide too much power to state attorneys general to determine how these provisions are actually enforced.

Pushes to regulate the internet have a deep connection to calls to protect children from its harms. That makes sense in some ways: The internet hosts a great deal of things that can be harmful to kids and adults alike, from privacy violations to networked harassment to incentivizing sensationalist and inaccurate content. But online access is always a both/and situation: the internet is both harmful to and a lifeline for young people. And it seems that organizations representing the interests of many marginalized communities aren’t convinced that KOSA will balance this.

You searched Google. The AI hallucinated an answer. Who’s legally responsible?

2024-05-23T13:45:12-04:00

Christoph Dernbach/picture alliance via Getty Images

" data-portal-copyright="

Christoph Dernbach/picture alliance via Getty Images

" data-has-syndication-rights="1" src="https://platform.vox.com/wp-content/uploads/sites/2/2024/05/gettyimages-2152425718.jpg?quality=90&strip=all&crop=0,0,100,100" />

Google CEO Sundar Pichai speaks at Google I/O in Mountain View, California, on May 14. At the developer conference, everything revolved around the topic of artificial intelligence (AI). |

Christoph Dernbach/picture alliance via Getty Images

Google’s shift toward using AI to generate a written answer to user searches instead of providing a list of links ranked algorithmically by relevance was inevitable. Before AI Overview — introduced last week for US users — Google had Knowledge Panels, those information boxes that appear toward the top of some searches, incentivizing users to get their answers directly from Google, rather than clicking through to a result.

AI Overview summarizes search results for a portion of queries, right at the top of the page. The results draw from multiple sources, which are cited in a drop-down gallery under the summary. As with any AI-generated response, these answers vary in quality and reliability.

Overview has told users to change their blinker fluid — which does not exist — seemingly because it picked up on joke responses from forums where users seek car advice from their peers. In a test I ran on Wednesday, Google was able to correctly generate instructions for doing a pushup, drawing heavily from the instructions in a New York Times article. Less than a week after launching this feature, Google announced that they are trying out ways to incorporate ads into their generative responses.

I’ve been writing about Bad Stuff online for years now, so it’s not a huge surprise that, upon gaining access to AI Overview, I started googling a bunch of things that might cause the generative search tool to pull from unreliable sources. The results were mixed, and they seemed to rely a lot on the exact phrasing of my question.

When I typed in queries asking for information on two different people who are widely associated with dubious natural “cures” for cancer, I received one generated answer that simply repeated the claims of this person uncritically. For the other name, the Google engine declined to create generative responses.

Results on basic first aid queries — such as how to clean a wound — pulled from reliable sources to generate an answer when I tried it. Queries about “detoxes” repeated unproven claims and were missing important context.

But rather than try to get a handle on how reliable these results are overall, there’s another question to ask here: If Google’s AI Overview gets something wrong, who is responsible if that answer ends up hurting someone?

Who’s responsible for AI?

The answer to that question may not be simple, according to Samir Jain, the vice president of policy at the Center for Democracy and Technology. Section 230 of the 1996 Communications Decency Act largely protects companies like Google from liability over the third-party content posted on its platforms because Google is not treated as a publisher of the information it hosts.

It’s “less clear” how the law would apply to AI-generated search answers, Jain said. AI Overview makes Section 230 protections a little messier because it’s harder to tell whether the content was created by Google or simply surfaced by it.

“If you have an AI overview that contains a hallucination, it’s a little difficult to see how that hallucination wouldn’t have at least in part been created or developed by Google,” Jain said. But a hallucination is different from surfacing bad information. If Google’s AI Overview quotes a third party that is itself providing inaccurate information, the protections would still likely apply.

A bunch of other scenarios are stuck in a gray area for now: Google’s generated answers are drawing from third parties but not necessarily directly quoting them. So is that original content, or is it more like the snippets that appear under search results?

While generative search tools like AI Overview represent new territory in terms of Section 230 protections, the risks are not hypothetical. Apps that say they can use AI to identify mushrooms for would-be foragers are already available in app stores, despite evidence that these tools aren’t super accurate. Even in Google’s demo of their new video search, a factual error was generated, as The Verge noticed.

Eating the source code of the internet

There’s another question here beyond when Section 230 may or may not apply to AI-generated answers: the incentives that AI Overview does or does not contain for the creation of reliable information in the first place. AI Overview relies on the web continuing to contain plenty of researched, factual information. But the tool also seems to make it harder for users to click through to those sources.

“Our main concern is about the potential impact on human motivation,” Jacob Rogers, associate general counsel at the Wikimedia Foundation, said in an email. “Generative AI tools must include recognition and reciprocity for the human contributions that they are built on, through clear and consistent attribution.”

The Wikimedia Foundation hasn’t seen a major drop in traffic to Wikipedia or other Wikimedia projects as a direct result of AI chatbots and tools to date, but Rogers said that the foundation was monitoring the situation. Google has, in the past, relied on Wikipedia to populate its Knowledge Panels, and draws from its work to provide fact-check pop-up boxes on, for instance, YouTube videos on controversial topics.

There’s a central tension here that’s worth watching as this technology becomes more prevalent. Google has an incentive to present its AI-generated answers as authoritative. Otherwise, why would you use them?

“On the other hand,” Jain said, “particularly in sensitive areas like health, it will probably want to have some kind of disclaimer or at least some cautionary language.”

Google’s AI Overview contains a small note at the bottom of each result clarifying that it is an experimental tool. And, based on my unscientific poking around, I’d guess that Google has opted for now to avoid generating answers on some controversial topics.

The Overview will, with some tweaking, generate a response to questions about its own potential liability. After a couple dead ends, I asked Google, “Is Google a publisher.”

“Google is not a publisher because it doesn’t create content,” begins the reply. I copied that sentence and pasted it into another search, surrounded by quotes. The search engine found 0 results for the exact phrase.

Teletherapy can really help, and really hurt

2024-05-16T14:23:53-04:00

The US has a therapist shortage. Even if you can find one, good luck snagging an appointment with a therapist who is affordable or covered by your insurance.

That’s where online therapy platforms like BetterHelp come in. Even if you’re not a user, you’ve almost certainly heard of them. For years, BetterHelp blanketed the online world with advertisements and enlisted an army of podcasters, influencers, and creators to produce sponsored content that promotes their service as a solution.

According to podcast analytics firm Magellan AI, BetterHelp spent an estimated $24.6 million on podcast ads in the first quarter of 2024, more than any other company in the audio space. These ads, sometimes personal and direct to camera, sometimes funny, and always earnest, have turned BetterHelp into probably the most recognizable sponsor of stuff people listen to on the internet. (Disclosure: BetterHelp advertises on several Vox Media podcasts.)

But the efficiency behind BetterHelp’s infinite expansion, it seems, can come at a cost to patients, as about 800,000 current or former clients are learning this week. In 2023, the FTC said that BetterHelp shared the sensitive data it collected on its users with advertisers, seemingly without their consent, or with provisions in place to limit how that data was then used. According to the AP, BetterHelp has said it was simply adhering to practices that were “standard for the industry.”

BetterHelp and the FTC eventually reached a settlement, and now anyone who signed up for BetterHelp between August 1, 2017, and December 31, 2020, is being notified of their eligibility for a refund. In a recent statement, BetterHelp said that the settlement was “not an admission of wrongdoing,” and clarified that the company did not share the “private information” of its members, such as their names or clinical data from sessions, with third parties.

There’s a lesson here about combining the optimization and efficiency tactics of any tech startup in order to provide something as vital and as sensitive as mental health care to people who may be in crisis. When these strategies intersect with a field that requires expertise and treats people in vulnerable moments, something’s almost bound to go awry. If you’ve been paying attention to the race to optimize therapy with technology, this is a lesson you’ve seen taught before.

Mental health care goes remote

The start of the Covid-19 pandemic in 2020 caused an abrupt shift in therapy access. Regulations and insurance policies restricting the feasibility of teletherapy were loosened. Some therapists and patients found that meeting online worked better for them. As Brian Resnick wrote for Vox at the time, the onset of the pandemic was a “much-needed kick forward into the 21st century” for mental health care.

For those seeking therapy, though, the new ease of finding virtual care meant sorting through waves of targeted ads on social media from companies that might not prioritize the quality of care over growing their businesses. Cerebral, founded in early 2020, enticed patients with easy, subscription-based access to virtual psychiatric treatment, including prescriptions for medication treating ADHD. But mental health professionals and patients raised worrying questions about the quality of that treatment. Underpaid providers were, according to Bloomberg, pressured to meet the expectations of patients who signed up for the service after seeing ads on social media promising quick and seamless access to medications.

These concerns led to investigations. The New York attorney general’s office fined Cerebral $740,000 over its “burdensome” cancellation policies and for manipulating online reviews. Cerebral, like BetterHelp, also recently settled with the FTC, which has accused the company of disclosing sensitive user data for advertising purposes, misleading customers over its cancellation policies, and of violating the Opioid Addiction Recovery Fraud Prevention Act. As a result of the settlement, Cerebral has agreed to pay more than $7 million.

The experience of being a patient through services like these will vary. Plenty of people who sign up for teletherapy through services like BetterHelp will have their needs met. I currently use a telehealth service provided by my insurance that matched me with a local practitioner in order to access medication, and I’m happy with it.

But during a more acute mental health crisis in 2021, I signed up for another teletherapy and medication platform that I’d seen advertised. While it was indeed easy for me to gain access to treatments for my anxiety, I ended up dropping my practitioner, and the service, months later after she missed an emergency appointment I’d made, and then spent the majority of the make-up appointment venting to me about a personal crisis in her life.

When things go wrong

Mistakes happen, but when services in something as vital as mental health go wrong, people get hurt. I still think a lot about a 2022 story in the Wall Street Journal, which detailed what this can look like.

Caleb Hill, a young adult who had been kicked out of his family home after coming out as gay, signed up for BetterHelp, requesting a therapist who specializes in LGBTQ+ issues. He was instead matched with a therapist whose private practice offers Christian counseling, and who told Hill that “either you sacrifice your family or you sacrifice being gay,” Hill told the paper. A former BetterHelp employee told the Journal that part of the issue was how the company’s focus on growth leads to a minimal training and oversight process for their therapists:

“I felt they were treated like Uber drivers,” Sonya Bruner, BetterHelp’s first clinical director and later a consultant to the company, told the Journal. “There are a lot of good counselors on there,” she said, “but you also find counselors who aren’t, who do the minimum. They don’t get paid a lot, so they’ll phone it in.”

I’ve thought a lot about the roles that technology can and can’t play in increasing access to mental health care like therapy. I’ve tried whenever possible to keep an open mind. I can see, for example, that there’s a pretty compelling argument to be made that, deployed ethically, chatbots and other implementations of AI can help improve patient access and results when seeking mental health care.

Even at their best, though, these tools and services are basically patches in an expensive system that creates its own barriers to care. And when technological solutions to mental health access do go wrong, the cost is steep: private, sensitive data is sacrificed to feed growth. And people, often in vulnerable situations, get hurt.

How TikTok Shop ads turned an obscure, inaccurate book into a bestseller

2024-05-08T18:13:13-04:00

If you’ve spent enough time scrolling through TikTok, you might have seen a video from an account like @tybuggyreviews, a handle with half a million followers that exclusively posts videos selling products through the TikTok Shop.

The creator, whose verified Instagram account identifies him as Tarik Garrett, used the @tybuggyreviews account to pitch viewers on supplements, water flossers, earbuds, workout machines, bible study guides, probiotics for women to help “that smell down there,” watch bands, inspirational hoodies, inspirational T-shirts, face massagers, foot massagers, rhinestone necklaces, oil pulling kits, and colon cleanses.

In the TikTok Shop, creators earn a commission for each sale linked to their account. Garrett’s product videos got tens of thousands of views. A few even topped a million views. But nothing from his account took off quite like his sales pitch for an obscure 2019 publication called The Lost Book of Herbal Remedies.

“Now I see why they’re trying to remove TikTok. This book right here? This book of herbal remedies? They do not want us to see this book,” Garrett said at the beginning of one Shop video, referring to a new US law that requires TikTok’s Chinese parent company to either sell the app or face a ban. TikTok is challenging the law in court, arguing that lawmakers citing national security concerns as a reason to pass the bill did not adequately argue why those concerns should supersede the First Amendment. The law, to be clear, does not cite the Lost Book of Herbal Remedies’s availability on the TikTok Shop as a reason for banning the platform.

Garrett posted his pitch for the book on April 15. As of May 7, the video had more than 16 million views. Garrett opened the book and showed pages of its recommendations, urging users to take screenshots (and purchase a copy of their own) before it’s too late.

The camera lingered on a list of plants that, the book claimed, were treatments for cancer, drug addiction, heart attacks, and herpes. As of Wednesday, the listing for The Lost Book of Herbal Remedies that Garrett linked to has more than 60,000 sales on the TikTok Shop. To put that number in perspective, appearing on a bestseller list generally requires 5,000–10,000 sales in a week.

And that interest isn’t staying exclusively on TikTok. Google search interest in the book’s title spiked on the same day Garrett posted his video. The Lost Book of Herbal Remedies was, as of Wednesday, May 8, ranked 10 on Amazon’s bestseller list for books, and has appeared toward the top of Amazon’s bestseller rankings for the past three weeks.

I sent a handful of Garrett’s videos advertising the book, along with about a half dozen additional widely viewed videos from other creators promoting The Lost Book of Herbal Remedies, to TikTok for comment. A spokesperson for TikTok said that videos linking to Shop products must abide by both the community guidelines, which ban medical misinformation, and Shop policies, which do not allow misleading content. If a video violates only the Shop policies, they said, they’ll simply remove the link to the Shop but keep the content up. If it violates community guidelines, the video comes down.

The violations were enough for TikTok to remove his product review account. Garrett did not respond to a series of emailed questions.

How e-commerce took over TikTok

TikTok has long been good at guessing what its users might want to see, but less good at monetizing that trick. When the platform launched its Shop feature in the United States last fall, the For You page shifted, pushing video after video like those made by @tybuggyreviews in the hope that its users will start buying the products that go viral on TikTok directly from their store.

The result became a For You page with constant interruptions from random product pitches. Right now, for instance, my For You page shows me a bunch of creators dancing to a German song about rhubarb, a bunch of pet birds behaving poorly, chaotic nonbinary people, and lots of ads from alternative wellness creators trying to sell me oils, mushrooms, and books.

The Shop ads I see, like much of the content pushed to me on TikTok, are personalized, though my TikTok Shop recommendations are heavily influenced by my reporting on stories like this one. Your results may differ. And yet, it is clear that TikTok has catapulted the Remedies book into relevance beyond a niche audience. The company earns money off of the explosion of sales on the shop, some of which come from creators who are explicitly promoting unproven cancer “cures” and conspiracy theories about the platform.

Like the Shadow Work Journal, a workbook that went super viral on TikTok Shop several months ago as a mental health tool — despite its dubious effectiveness — The Lost Book of Herbal Remedies is part of a swell of wellness creators, brands, and products that have found success reaching new audiences on TikTok Shop.

Shop videos have become a sort of “loophole” for health misinformation on TikTok, said Evan Thornburg, a bioethicist who posts on TikTok as @gaygtownbae and studies mis/disinformation and public health. Creators, and those with something to sell, know that Shop videos will get privileged on For You pages. Some creators may use those videos to promote dangerous health claims. In other cases, Thornburg noted, “the creator promoting the material isn’t necessarily spouting off disinformation, but the material that they’re convincing people to purchase is.”

A recipe for misinformation

The Lost Book of Herbal Remedies appears to be a case of both: The book contains misleading information, and creators are circulating misleading health claims in order to sell books. A video with nearly 1 million views promoting the book’s TikTok Shop listing is basically a series of ominous, AI-generated images with an AI voiceover. The video claims that the book contains secrets previously locked away in an ancient book located in the “Vatican library,” and that The Lost Book of Herbal Medicine was previously only available on the “dark web” before surfacing on TikTok. (Not true: The book is for sale on Amazon, the author’s website, and appears to be available through some academic and public library systems.) Another Shop video with more than 1 million views is captioned, “Cure for over 550 diseases, even cancer.”

I scanned through a copy of The Lost Book of Herbal Remedies this week. The 300-page book contains a disclaimer noting that it’s intended to “provide information about natural medicine, cures, and remedies that people have used in the past,” that it is not medical advice, and that some of the “remedies and cures found within do not comply with FDA guidelines.” It’s split into two parts: an alphabetical listing of ailments and conditions alongside the plants that the authors believe can cure or treat them, and an alphabetical list of plants, sorted by region, with instructions on how to prepare them.

The list of ailments the book includes proposed treatments for cancer, several STDs, mental health disorders, and digestive issues, among many other things. A few stand out: The book lists cures for smallpox, strep, and staph infections. There’s an emergency medicine section that includes plant remedies for serious medical conditions like internal bleeding and poisoning.

Flip to the entries for the plants and you’ll find lists of claims referring to research that is not cited. An entry promoting Ashwagandha’s “anti tumor effects” and ability to “kill … cancerous cells” refers to “research,” but does not note that, while there is some indication that Ashwagandha can slow the growth of cancer cells, these studies were conducted on rodents and have yet to be replicated on humans.

Nicole Apelian, one of the book’s authors, did not reply to an emailed request for comment. While active on TikTok, it’s not her main social media presence. Her TikTok bio encourages her 17,000 followers there to check her out on Instagram, where she has 100,000 followers. Apelain also runs Nicole’s Apothecary, an herbal shop mentioned in the book that sells some of the tinctures she recommends, sells memberships to an online “Academy” for fans of her book, and advertises her paid appearances and workshops.

The endless whack-a-mole

As a journalist, there’s a pattern that becomes evident when writing about health misinformation on social media: something gets views, you assess the real or potential harm and try to understand its context, you contact the company to ask about the harmful thing. Maybe the video or post or group is taken down, maybe it’s not. The company gives you a statement, refers you to their policies on misinformation, and then you publish the article. This happens over and over and over and over because writing about misleading health information is a game of whack-a-mole that feels harder and harder to win.

Thornburg, the bioethicist, noted a couple reasons why I can’t climb out of this purgatory. First, meaningful moderation of a platform like TikTok is somewhat implausible. Social media companies are “never going to prioritize the amount of labor that would need to consistently be put into misinformation management,” they said.

Most sites rely on a combination of human moderators and AI, and it’s difficult to create automated moderation tools that don’t also censor allowed content. For example: health misinformation targeting minority communities often taps into legitimate distrust of medical professionals and institutions that have roots in recent history. An AI tool designed to moderate keywords associated with this sort of targeted misinformation might also sweep up criticism of health care systems in general.

And second, the creators who profit off health misinformation are really good at figuring out what they can say where, and what Thornburg calls “life boating” their audiences from one platform to another as needed. “You will have people who will drive interest in something through TikTok because the virality and the algorithm are aggressive,” Thornburg said. Then, their profile will link out to their Instagram or Linktree or YouTube channel.

Health misinformation on social media is a million cross-pollinating moving targets. TikTok Shop is a hot spot right now. Later, it might be something else on another platform. Chasing this content from platform to platform, harm to harm, viral video to viral video, is exhausting. I am exhausted.

At the end of our interview, Thornburg shared the question that drives a lot of their work in this space, “Who do we consider accountable for these things that are harmful and regulate them or hold them to certain standards?” Often, it’s not really the person behind the individual piece of content driving the incentives for making it.

As a result of my reporting, Garrett’s account was taken down, along with a few other popular videos advertising a book that has already sold tens of thousands of copies. As long as the incentives remain, it won’t be long until the next product promising a miracle starts polluting my For You page.

The misleading information in one of America’s most popular podcasts

2024-05-02T13:12:22-04:00

Andrew Huberman, a neurobiology professor and host of the Huberman Lab podcast, attending INBOUND 2023 in Boston, Mass. | Photo by Chance Yeh/Getty Images for HubSpot

Sometimes, misleading information is easy to spot, traveling in the same conspiracy-theory-slicked grooves it has for decades. The same ideas that undermined belief in the safety of Covid-19 vaccines have been around for more than a century, adapting the same message to suit new media formats, new epidemics, and new influential endorsements. In a way, George Bernard Shaw’s outspoken opposition to the smallpox vaccine in the first half of the 20th century is not unlike that of, say, Aaron Rodgers’s misleading statements about the Covid-19 vaccines.

Such misleading information is relatively easy to see. But spotting other kinds of misleading information is more like identifying planets in other star systems. It’s difficult to find such a planet by just taking a direct image; the radiation from the star the planet orbits can obscure it. Instead, you might look for the shadow in front of the star or the “wobble” of a star caused by the gravitational pull of an orbiting planet. You find it by looking around it.

Over time, with this kind of misleading information, you learn to spot the wobble, the tells that something might not be right. This is what happened for me when I began to listen to Huberman Lab last fall.

Huberman Lab is one of the most popular podcasts in the country, led by Stanford neuroscientist Andrew Huberman. His most ardent fans — and there are millions — tend to be fitness enthusiasts, self-optimizers, and crossover listeners who heard about his podcast from other influencers in the Joe Rogan Extended Universe. Huberman looms large in the minds of his biggest fans. If you’re outside of that circle, perhaps you heard of his work after a New York magazine profile earlier this year detailed his personal conduct.

The podcast’s premise is simple: presenting science-based overviews and conversations on a broad range of topics, from longevity to mental health to nutrition. A fawning profile in Time magazine last summer credited Huberman with getting America to care about science again. More than anything, though, the episodes I listened to conveyed a promise: If you want to optimize your body and mind, science has the answers, and all we need to do is listen. It’s a riveting promise, one that Huberman is not alone in making.

Silicon Valley, in particular, is filled with wellness guides and well-funded laboratories seeking the secret to living the best and longest life. There are other well-credentialed promises of cures and solutions circulating, especially on podcasts, a format that seems to lend itself to this slippage between the reputable and the freewheeling.

Huberman’s rise to popularity during the Covid-19 pandemic should have been a win for information: Huberman, an associate professor of neurobiology at Stanford with an active lab, it seemed, was a respected researcher in his field of visual neuroscience, and he filled his multi-hour podcast episodes with citations and caution.

Popular science communication isn’t always the best science communication. The implicit pact that Huberman’s podcast makes with its audience — that it will, if you listen and follow, help you optimize your life — has turned the podcast into a powerful force that shapes how his audience of millions understands science. But listeners of Huberman Lab may be, at times, hearing what some call an illusion.

When good communication goes bad

In late March, New York magazine reported that Huberman’s Stanford laboratory “barely exists” and that, according to multiple women who dated him during his rise to fame, Huberman had manipulated and lied to his partners (Huberman’s spokesperson denied both of these allegations to the magazine, which shares a corporate owner with Vox).

The profile was one tell — obscuring aspects of his personal and professional lives. But even before it came out, the same subject experts on the topics Huberman covered had been questioning some of the science of the podcast itself.

This liminality, or in-betweenness, of Huberman Lab is key to its success. When speaking about vaccines, Huberman is no Alex Jones or Aaron Rodgers. He’s a real scientist who cites real studies. He approaches topics that might end up drawing scrutiny with a great deal of caution.

For example, Huberman never tells his audience to avoid the flu vaccine. All he’s saying is that he doesn’t take it himself. And yet, the subtext is there. “Now, personally, I don’t typically get the flu shot. And the reason for that is that I don’t tend to go into environments where I am particularly susceptible to getting the flu,” Huberman said in an episode earlier this year on avoiding and treating the cold and flu.

He went on: “When you take the flu shot, you’re really hedging a bet. You’re hedging a bet against the fact that you will be or not be exposed to that particular strain of flu virus that’s most abundant that season, or strains of flu virus that are most abundant that season, and that the flu shot that you’re taking is directed at those particular strains.” Make the choice that’s right for you, Huberman says. Talk to your doctor.

“He’s a good communicator, right? That’s why he’s a star,” Tim Caulfield, a professor of health law and science policy at the University of Alberta, told me in late 2023. Huberman often does a “very good job” talking about the science behind a topic he’s exploring in an episode, Caulfield added, but “in the end, the overall takeaway, I think, is less supported by the science than the impression you’re given listening to the episode.”

Instead of recommending a flu shot, Huberman introduces his listeners to a series of other ideas. Andrea Love, a microbiologist, immunologist, and science communicator herself, wrote a four-part newsletter series addressing Huberman’s claims in greater detail. She says he promoted possibly using a sauna to improve immune function, citing a study that had just 20 participants and did not directly measure immune function. She says he promoted the potential use of unproven supplements, including those sold by AG1, a company that partners with Huberman and sponsors his podcast. Huberman and his spokesperson did not respond to a request for comment on Love’s characterization of this episode.

For Love, it was easy to see Huberman Lab as sleight of hand even before the New York magazine story was published. The ingredients were there: Huberman is a magnetic personality capable of capturing attention with implied promises of the secrets to longevity, a perfect body, a perfect mind, even perfect sleep — much of which he says can be achieved with the help of the supplements that he himself advertises.

Love was part of a cohort of scientists and public health communicators who raised concerns about Huberman’s wildly popular podcast over several months. When Huberman had Robert Lustig on as a guest, those concerns grew louder. Lustig is a pediatric endocrinologist at the University of California San Francisco (UCSF), but he’s perhaps best known for arguing that sugar, particularly fructose, is a “toxin.” Love, who said that Lustig’s claims about the uniquely causal relationship between fructose and childhood obesity remain unproven, listened to the conversation between the two scientists. (Disclosure: I recently accepted a contract for non-editorial freelance work at UCSF Health.)

“I was floored with how many different types of misinformation he was able to shove into a single episode,” Love said earlier this year, after listening to the majority of Huberman’s 3-hour interview with Lustig. Like many of Huberman’s lengthy episodes, this one racked up millions of views on YouTube alone. In 2023, Huberman Lab was the eighth most listened to podcast on Apple Podcasts, and the third most popular on Spotify.

As she listened, she took notes, marking moments where she felt the podcast omitted important facts, misinterpreted the progression of disease, or provided confusing information to listeners.

At one point, Lustig cited a study that he said “showed” ultra-processed foods inhibit bone growth — one that, according to Huberman’s exchange with Lustig, used human subjects in Israel to test its claims. Love tracked down the 2021 paper easily. “This was in vivo – IN RODENTS,” she wrote in her notes.

In her view, the podcast was “outright LYING to listeners.”

A spokesperson for Andrew Huberman responded to a request for comment by noting that the podcast team “review studies mentioned on the podcast by guests, however the conclusions drawn by guests are their own and our guests are the foremost experts in their fields.” The show links to referenced studies in the show notes for each episode.

Misleading information can be hard to see

Nailing down Huberman’s beliefs is, likewise, tricky, straddling the line between endorsement and implication. In October, Huberman commented on an Instagram post by his friend Joe Rogan promoting an interview with Robert F. Kennedy Jr., the presidential candidate who was once a respected environmental lawyer but is now perhaps best known for promoting conspiracy theories about vaccines, including those for Covid-19.

“I’m eager to listen to this and to learn more about Robert’s stance on a number of issues. Whenever I run into him at the gym, he is extremely gracious and asks lots of questions about science and, by my observation, trains hard too!” Huberman’s verified Instagram account posted.

When I told Caulfield about this post, he described it as “infuriating.” Huberman and his spokesperson did not respond to a request for comment on his post about Kennedy.

“Any kind of legitimization and normalization of that rhetoric, especially by someone who professes to be informed by science and has the credentials of a renowned institution behind him should be ashamed of doing that,” he said.

Huberman’s relationship to the information in his podcast can be viewed through a series of glancing blows; through the subtext of deciding not to take the flu vaccine himself and telling that to his audience; through serious questions about how he handles himself in romantic relationships; and through the selection of his guests, the framing of his episodes, and his friends.

Although Huberman has not directly responded to the New York magazine piece after its publication, his friends in the podcasting world, along with several more right-leaning media personalities, have called it a hit piece, and dismissed criticism of Huberman as either sloppy or mean-spirited. “Andrew should be celebrated. Period,” wrote Lex Fridman, a computer scientist and podcaster who has long been one of Huberman’s friends. And it appears his podcast viewers are still tuning in.

Imagining an internet without TikTok

2024-04-24T17:48:37-04:00

The bill to require TikTok to separate from its Chinese parent company or face a nationwide ban made it to President Joe Biden’s desk on Wednesday as part of a huge foreign aid package that passed through Congress this week. And Biden, as he previously promised, signed the bill into law.

ByteDance now has nine months to sell TikTok, a deadline that Biden can opt to extend once by 90 days. And while TikTok could avoid a ban with a successful sale or court challenge, the new law means Americans might want to start imagining an online world without TikTok.

The push to either ban TikTok or excise the platform from its owner has been around for years. For instance, then-President Trump announced plans to ban the app in the summer of 2020, although Trump now says he thinks banning TikTok is a bad idea and that people should be mad at Biden about it.

The threat of a TikTok ban has always been a little weird and complicated, drawing from a mixture of valid concerns and questionable moral panics about the ills of social media. As I’ve written previously, TikTok’s moderation failures and data privacy concerns are hardly unique, even as some lawmakers seem to persist in holding TikTok uniquely responsible for perpetuating them.

With that in mind, let’s break down the implications of this new law, why it’s happening, and what the internet would look like if TikTok disappeared.

What you need to know about the ban

Now that Biden has signed the bill into law, ByteDance has at least nine months — and possibly one year — to sell TikTok. It’s not clear, though, whether the law will survive a court challenge, which TikTok has already vowed to do.

The government is likely prepared for this, as the new law was the result of years of planning by lawmakers, triggering waves of opposition from TikTok executives and the app’s huge user base, which includes 170 million Americans, according to TikTok.

Congress made one earlier attempt to pass such a ban in March. That bill, which passed the House but didn’t make it through the Senate, gave ByteDance just six months to sell TikTok. The new version’s extended deadline may have helped sway some people in the Senate to vote for the bill. It certainly didn’t hurt that the TikTok ban was attached to a $95 billion aid deal that would provide support to Ukraine and Israel.

TikTok CEO Shou Chew said Wednesday that the app wasn’t “going anywhere” and that the company believes the courts will ultimately find the ban unconstitutional, violating the First Amendment. The US government would need to meet a high standard to prove that a ban is necessary to protect the nation’s security and privacy in order to prevail. Montana’s statewide ban of TikTok was blocked by a federal judge late last year as a likely violation of the First Amendment. The state is appealing that decision.

ByteDance could also, you know, sell. However, the Chinese government has previously said that it would oppose a forced sale of TikTok.

Why is this happening?

Great question!

The lawmakers leading the charge on this ban have cited national security concerns stemming from the app’s Chinese ownership. Specifically, they’ve mentioned the possibility of the Chinese government accessing the data of American users and using the app to spread propaganda or influence foreign elections. Members of Congress have referred to information they learned in security briefings about the potential for TikTok to harm American interests, but the contents of those briefings are not public.

Arguments in favor of banning TikTok note that the app’s Chinese ownership puts user data at risk of access by an unfriendly foreign government; critics note that the Chinese government could access a lot of the same data by simply buying it from a data broker.

There’s another driving force here, though. As we’ve previously noted, the current push for a ban in Congress gained a lot of attention after a viral but unfounded accusation spread that TikTok was brainwashing the youth of America with anti-Israel content in the first days of the Israel-Hamas war. That narrative seemed to rekindle a lot of fears about the power of TikTok to become a propaganda tool.

What changes if TikTok goes away in the US?

This isn’t the first time a major force in internet culture has faced extinction (RIP Vine). In fact, this churn is increasingly part of being online now. But TikTok, arguably, is the most influential online platform in the US, and it won’t be easily replaced. If TikTok goes away, other platforms will try to jump into the void it will leave.

As the Washington Post’s Will Oremus wrote, a TikTok ban would provide an open space for Meta and Google to move in. Meta has already adapted a lot of TikTok’s features via Reels, and Google’s YouTube has its Shorts video format, but neither quite has the cultural force behind them that TikTok has right now.

A lot of bigger creators — those with resources, managers, and huge followings — will be able to switch to another platform, if they haven’t already. Everyone else might see things shift more dramatically.

Earlier this year, Zari A. Taylor, a doctoral candidate at the University of North Carolina at Chapel Hill who studies media and culture, explained to me that the biggest loss to online culture if TikTok goes away will be in the uniqueness of how the platform promotes videos into user feeds. TikTok is good at recommending videos by accounts with small followings, whose makers are often not professional content creators. These creators “don’t have the audience that could help them evolve into other areas of the entertainment industry,” she said, and will likely lose their audience should the ban stand.

In some ways, the constant threat of a ban has already taken a toll on TikTok’s appeal for creators. After the first wave of TikTok ban threats back in 2020, I spoke with Ryan Beard, a creator who at the time had nearly 2 million TikTok followers. The threat of a ban from President Trump sent his livelihood into a spiral, and he accelerated his efforts to get views and followers on other apps. These days, he’s all but stopped posting on TikTok and has instead become a commentary YouTuber.

When TikTok rose in influence, it was better than any other app at showing users what they wanted to see, for better or for worse. Short, vertical-format videos might be available on any old platform these days, but the format doesn’t replicate what keeps people scrolling.

Some, like Beard, will turn their modest TikTok success into views on another platform. For many others, even the threat of the ban is a harsh reminder of the realities of making content on the internet: Your livelihood is tied to the success and attention of platforms you don’t control.

A version of this story was published in the Vox Technology newsletter. Sign up here so you don’t miss the next one!

The slow death of Twitter is measured in disasters like the Baltimore bridge collapse

2024-03-27T16:03:35-04:00

Baltimore’s Francis Scott Key Bridge collapsed after being struck by a cargo ship on March 26. | Scott Olson/Getty Images

Line up a few years’ worth of tragedies and disasters, and the online conversations about them will reveal their patterns.

The same conspiracy-theory-peddling personalities who spammed X with posts claiming that Tuesday’s Baltimore bridge collapse was a deliberate attack have also called mass shootings “false flag” events and denied basic facts about the Covid-19 pandemic. A Florida Republican running for Congress blamed “DEI” for the bridge collapse as racist comments about immigration and Baltimore Mayor Brandon Scott circulated among the far right. These comments echo Trump in 2019, who called Baltimore a “disgusting, rat and rodent infested mess,” and, in 2015, blamed President Obama for the unrest in the city.

As conspiracy theorists compete for attention in the wake of a tragedy, others seek engagement through dubious expertise, juicy speculation, or stolen video clips. The boundary between conspiracy theory and engagement bait is permeable; unfounded and provoking posts often outpace the trickle of verified information that follows any sort of major breaking news event. Then, the conspiracy theories become content, and a lot of people marvel and express outrage that they exist. Then they kind of forget about the raging river of Bad Internet until the next national tragedy.

I’ve seen it so many times. I became a breaking news reporter in 2012, which means that in internet years, I have the experience of an almost ancient entity. The collapse of the Francis Scott Key bridge into the Patapsco River, though, felt a little different from most of these moments for me, for two reasons.

First, it was happening after a few big shifts in what the internet even is, as Twitter, once a go-to space for following breaking news events, became an Elon Musk-owned factory for verified accounts with bad ideas, while generative AI tools have superpowered grifters wanting to make plausible text and visual fabrications. And second, I live in Baltimore. People I know commute on that bridge, which forms part of the city’s Beltway. Some of the workers who fell, now presumed dead, lived in a neighborhood across the park from me.

The local cost of global misinformation

On Tuesday evening, I called Lisa Snowden, the editor-in-chief of the Baltimore Beat — the city’s Black-owned alt-weekly — and an influential presence in Baltimore’s still pretty active X community. I wanted to talk about how following breaking news online has changed over time.

Snowden was up during the early morning hours when the bridge collapsed. Baltimore’s X presence is small enough that journalists like her generally know who the other journalists are working in the city, especially those reporting on Baltimore itself. Almost as soon as news broke about the bridge, though, she saw accounts she’d never heard of before speaking with authority about what had happened, sharing unsourced video, and speculating about the cause.

Over the next several hours, the misinformation and racism about Baltimore snowballed on X. For Snowden, this felt a bit like an invasion into a community that had so far survived the slow death of what was once Twitter by simply staying out of the spotlight.

“Baltimore Twitter, it’s usually not as bad,” Snowden said. She sticks to the people she follows. “But today I noticed that was pretty much impossible. It got extremely racist. And I was seeing other folks in Baltimore also being like, ‘This might be what sends me finally off this app.’”

Here are some of the tweets that got attention in the hours after the collapse: Paul Szypula, a MAGA influencer with more than 100,000 followers on X, tweeted “Synergy Marine Group [the company that owned the ship in question] promotes DEI in their company. Did anti-white business practices cause this disaster?” alongside a screenshot of a page on the company’s website that discussed the existence of a diversity and inclusion policy.

That tweet got more than 600,000 views. Another far-right influencer speculated that there was some connection between the collapse and, I guess, Barack Obama? I don’t know. The tweet got 5 million views as of mid-day Wednesday.

Being online during a tragic event is full of consequential nonsense like this, ideas and conspiracy theories that are inane enough to fall into the fog of Poe’s Law and yet harmful to actual people and painful to see in particular when it’s your community being turned into views. Sure, there are best practices you can follow to try to contribute to a better information ecosystem in these moments. Those practices matter. But for Snowden, the main thing she can do as her newsroom gets to work reporting on the impact of this disaster on the community here is to let time march on.

“In a couple days, this terrible racist mob, or whatever it is, is going to be onto something else,” Snowden said. “ Baltimore … people are still going to need things. Everybody’s still going to be working. So I’m just kind of waiting it out,” she said “But it does hurt.”

A version of this story was published in the Vox Technology newsletter. Sign up here so you don’t miss the next one!