Kate Klonick | Vox

Here’s what it would take for Twitter to get serious about its harassment problem

2016-12-30T11:15:21-05:00

Ghostbusters star Leslie Jones has faulted Twitter for failing to stop harassment on its platform. | Alberto E. Rodriguez/Getty Images

In its early years, Twitter refused to address harassment concerns on free speech grounds. But the company changed its tune a few years ago, and now it insists it’s doing everything it can to fight online harassment.

But as many using the platform know, little has come of it and enforcement of new policies had been terribly inconsistent.

The extreme levels of harassment on the site and the inconsistencies in policy enforcement gained national attention this July, when Ghostbusters star Leslie Jones threatened to leave Twitter after facing a barrage of racist and sexist tweets.

Twitter responded by permanently banning professional alt-right troll Milo Yiannopoulos, one of Jones’s worst harassers, and Twitter founder and CEO Jack Dorsey acknowledged that the site “needs to do better.” He announced a new formal process for anyone to get a blue check mark — a symbol that verifies that a Twitter account is authentic. In mid-August the platform announced “New Ways to Control Your Experience on Twitter,” which include changes to “notification” settings as well as a new “quality” filter.

Yet these steps don’t seem to have fixed the problem. In August, BuzzFeed published an exposé on the company’s “inaction and organizational disarray” when it came to dealing with abuse. And my conversations with Twitter users who have been subject to abuse tell me the new tools are as clumsy as ever. When they do work, they stop abuse but at the cost of cutting users off from the positive features of the site.

The fundamental problem, as one subject of harassment on Twitter told me, is that Twitter still doesn’t seem to see fighting abuse as a core part of the product.

“If you don’t build content moderation in from day one and you tack it on after the fact, you’re facing a losing battle,” says Susan, an active Twitter user who preferred to use a pseudonym because of the volume of online harassment she has received due to her political work. It’s a lesson Twitter still needs to learn.

Twitter’s current anti-abuse features are too clumsy

Twitter’s early refusal to deal with abuse derived from early attitudes that viewed the internet — and Twitter’s platform in particular — as a forum for unfettered free speech. That was the company’s policy until July 2013, when the platform met with a wave of criticism after British feminists received rape threats and hate speech through the site and no recourse was available to report or stop it.

The incident finally prompted Twitter to not only change its policy but also add a “Report Abuse” button alongside the “Report Spam” button. Previously, users wishing to “flag” offensive content were required to fill out an extensive web ticket for customer service.

As high-profile, and often sexist, abuse rose on Twitter, the platform continued to make changes to its policy in an effort “to reduce abuse, including prohibiting indirect threats and nonconsensual nude images.” But while Twitter talked a lot about its anti-harassment efforts, the company made few significant changes until the summer of 2016.

In mid-July, perhaps prompted by the Leslie Jones fiasco, Twitter announced an expansion of its blue check mark program. Previously, only celebrities and media professionals could get a blue check mark next to their name to verify that they were the people they purported to be. Now any Twitter user can apply to have their identity verified. Verification fights harassment by tying people’s real identities to their accounts, diminishing the risk of anonymous harassment and supposedly enhancing the quality of discussion.

Before July, verification was even more significant because only verified users had access to anti-abuse features. But about a month after announcing the “transparent” verification process, Twitter rolled out two powerful new features: a “quality” filter, which aimed to algorithmically hide spammy tweets, and a “notifications” filter, which allowed a user to disable notifications of mentions and replies that came from people they didn’t follow.

So how have those new changes to settings been working out? A little too well, perhaps.

“I turned on the blue check mark and turned off all of the notification settings, and all of a sudden things got very quiet,” says Susan. “Between notification settings turned on and the ‘low quality’ filter on, it’s impossible to tell what you’re seeing and not seeing.”

What Susan is not seeing is not just the deluge of her frequent harassers riling up other Twitter users to attack her. She (happily) doesn’t see the expletives and the trolling, or the comments about her looks. But she also doesn’t see good things that Twitter brought her: connections with like-minded people and colleagues, positive reactions about her work.

Jessica Valenti, who left all social media a few months ago after threats to her daughter on Instagram, doesn’t quite know what to make of the new Twitter features. Valenti, who also founded the foundational feminist site Feministing, is sadly an old hand at online harassment.

“It’s just not the same platform,” she said in a phone call in late September, speaking of her diminished positive engagement on the site. “I’m glad not to see all the hate every day, but it’s sanitized.” Twitter used to be a fire hose. Today, if you engage all the filters, it’s more of a trickle.

The loss in the functionality of Twitter as a means of connecting socially and professionally was one of the largest complaints from those I talked to. If you only get notifications from people you already follow, you can’t easily or naturally expand your network or hear smart new voices — which is a big part of what made Twitter appealing in the first place.

Twitter’s defenders resist further anti-harassment measures with the reasoning that abuse is simply an inevitable consequence of the openness of the platform. In contrast to more closed platforms like Facebook and Instagram, Twitter is designed to facilitate open-ended conversations among total strangers. Free speech purists insist that if you want to use a platform like that, you just have to take the good with the bad.

But it’s also possible that this apparent trade-off between harassment and censorship is actually a symptom of Twitter’s underinvestment in resources to create an anti-abuse infrastructure. Users face a choice between getting harassed and missing valuable interactions because Twitter has built clumsy tools that don’t give users better options. And there’s plenty of reason to think Twitter could be doing better.

Twitter needs to view fighting abuse as an essential feature

Facebook is a fundamentally different platform from Twitter in some ways, but it still offers a valuable example. Facebook has prioritized content moderation and harassment from early on in its history.

That means it has an enormous and well-developed set of policies and procedures, and a team of trained humans to keep it running smoothly. It’s not perfect, but by the standards of the internet, Facebook’s system is a biplane, while Twitter’s is a dirigible.

“As long as abuse is not core to a product, as long as it’s seen as a cost center instead of table stakes, then it’s always going to be a subpar experience,” Susan told me.

Thankfully, the vast majority of users aren’t victims of the kind of harassment faced by the women I talked to for this story. But harassment is still a big threat to the long-term health of Twitter’s platform. The issue isn’t just that Twitter will lose harassment victims themselves as users. Users who enjoy their work — and some of them have significant followings — will also miss out. More importantly, when trolls succeed in driving prominent women off Twitter, it has a toxic effect on the culture of the platform as a whole, making it a little less welcoming for women and minorities in general.

Anita Sarkeesian, founder of Feminist Frequency, a nonprofit organization that looks at pop culture from a feminist perspective, thinks about this a lot. In earlier days of harassment online or problems with her site, often her only recourse was to reach out to people she knew at these platforms, many of whom she got to know because she reported harassment so frequently.

But if you’re not best friends with someone at Twitter, a famous Hollywood actor, or president of the United States, how good is the site at dealing with abuse complaints?

Building resources that can better respond to bystander reports, or reports from non-elite Twitter users, is essential to keeping the democratic function of the platform.

One easy — though expensive — step is to simply hire more people to respond to harassment reports, verify users, and focus on breaking up troll communities. But for more boots on the ground to be a practical solution, it requires that those humans be assisted by algorithms. This is nothing new: There are patented systems that have existed for years to make this easier. Right now, however, it’s unclear precisely what human or software resources Twitter is using to deal with this problem.

Creating a better system to verify users for blue check marks is also essential — and it can help everyone, not just celebrities and the harassment “elite.” As of late this summer, only 0.061 percent of total daily active users have the elite mark. And while that’s better than before, if the site is verifying users at a rate of fewer than 300 a day, it’s going to be a long time for this feature to help the masses.

Twitter could provide users with more powerful anti-harassment tools

Even if Twitter doesn’t beef up its behind-the-scenes resources, at the very least it could offer better anti-harassment features directly to users, according to the harassment victims I talked to.

“Right now there’s only one tier available for filtering,” says Susan, who dismissed the new “quality” filter as inadequate. “You can either have the deluge or you can have a sanitized little community.”

Sarkeesian, who also consults with Twitter as part of the company’s new Trust and Safety Council, thinks both user- and content-based tools are the way forward to combat online harassment. She suggests the possibility of more content-based tools that allow you to mute conversations or threads, working in conjunction with filters for the users themselves.

“The question of users versus content — I think you have to do both,” she says. “There are people who skirt the terms of service, who are harassing and creating discord, but because they’re inside the terms of service, no action can be taken on them.” Sarkeesian gives the example of one of her harassers, who might tweet simply in reply to a new video that she “looks stupid with makeup on.”

“That specific tweet is not against the terms of service,” she explains, “but he has an army of people who follow him that all hate me, so my Twitter feed gets flooded in reply, and some of them are mean, some of them are threats, and some of them are against terms of service. And there are hundreds in a row — so that [original] user can send mobs my way or sow disinformation but never directly threaten me or use slurs against me — but their followers can.”

She also points out that even if you block people, they can still tag you in a tweet, so their legion of followers can easily attack. Simply removing the ability to hyperlink tag someone who has blocked you can create a natural barrier to mob harassment, without any censorship.

Renee Bracey Sherman, an abortion activist and co-author of the online safety guide Speak Up & Stay Safe(r), with Sarkeesian and activist Jacyln Friedman, points out that blocking users isn’t all a platform can do. “On Facebook, I have in my settings a list of words that are banned from my feed,” says Bracey Sherman. “So I can block ‘baby killer,’ or ‘you’re a murderer,’ and the post automatically won’t show up.”

Many victims of harassment have turned to outside applications to do the heavy lifting between content and user blocking. Bracey Sherman suggests Block Together, an app to help you use others’ block lists to avoid users and customize your settings to ban new accounts or accounts with the anonymous egg avatar. “These are all stopgaps that we put in place,” she says. “They do help, but they’re not the end.”

The fact that users are turning to third parties for this kind of capability is a clear sign that Twitter is falling down on the job. Twitter has access to vast amounts of data about its users, which should allow the company to build a much more powerful version of the Block Together feature — and more power to encourage its users to engage with it.

The early form of Twitter made it a remarkable and world-changing tool for communication and change. But failing to get its harassment problems under control could put its future in doubt.

Kate Klonick is a legal academic and resident fellow at Yale Law School’s Information Society Project. She writes about technology, psychology, and the law and is currently working on a project about content moderation.

The science of blame

2018-09-14T15:27:09-04:00

After Germanwings Flight 9525 crashed into the French Alps last month, the questions started out simple. Who or what caused the crash? Was it terrorists? Engine problems? A strange weather incident? Flight control issues?

When the answer was revealed to be a pilot — acting alone, with no known motive beyond his own self-destruction — the questions intensified: did the airline know he was depressed? Should it have known he was depressed? Could it have known he was depressed? Did the agency err in having lax guidelines for the number of pilots required to be in the cockpit? Was the German government complicit in the lack of knowledge of the pilot’s mental state because of the country’s strong health-care privacy laws?

This is a familiar pattern of questioning: when Adam Lanza shot and killed dozens at Sandy Hook Elementary School, a frenzied search for answers followed. What took the school so long to lock down? Why did Lanza’s mother let him have guns? Why did the government allow the purchase of those types of guns? Were we as a society paying enough attention to those with autism spectrum disorders?

Punishing someone for a past event doesn’t always make it easier to prevent bad events in the future

And again with Elliot Rodger, the “Santa Barbara killer” who left lengthy YouTube manifestos online and in print about killing women he felt had spurned him before he took to the Southern California streets with a handgun. How did people miss these public messages? How were the threats not flagged? How did his parents not know about his dangerous behavior? Did the sexual objectification of women in video games and media contribute to Rodger’s sense of entitlement to female affection?

At least in the case of Andreas Lubitz, the Germanwings pilot, the answers to all of those questions might very well be “no”: while Lubitz did tell the airline in 2009 that he was depressed and needed a break from training, almost nothing in his more recent medical history or treatment gave any indication that an incident like this could have happened. So it seems like the blame ends at Lubitz. Then why do we keep looking for answers and trying to hold others responsible?

Our impulse to blame is strong, but it’s also complicated and imperfect. Cognitive psychology tells us that following a negative event our impulse to blame has both an emotional “why” component driven by anger and sadness, and a more rational “who or what is going to make sure this doesn’t happen again” set in motion by anxiety and fear. These processes are closely related and often at cross-purposes. Punishing someone for a past event doesn’t always make it more likely we can prevent bad events in the future. Conversely, making things safer in the future doesn’t always give us the vindication of punishment. Having a better understanding of why we blame and what we’re seeking when we do can perhaps get to a more satisfactory — and productive — end.

“People see something happen and they want to know why”

The first step in satisfying the questions we ask after a tragedy requires finding a person to blame. The answer in the Germanwings tragedy, though obscured in the immediate aftermath of the crash, is now clear: Lubitz.

That brings us to the next question, one of intentionality: did Lubitz do it on purpose? If he hadn’t — if he had a heart attack, for instance, and fell forward onto the controls — we would hold him much less accountable. But here, Lubitz’s actions were unequivocally purposeful. Which leaves us grappling to find an explanation, to try to understand Lubitz’s reasons for committing such a horrible act.

“People see something happen and they want to know why,” said Bertram Malle, professor of psychology at Brown University and author of A Theory of Blame. “That’s true for even small events in the world — your pants have a rip or your car has a scratch in the paint, and you want to know why. And when someone acts intentionally, you want to know what the reasons are.”

There are many “reasons” for a person to crash an airplane. Oft cited are: religious extremism, inciting general terror, political extremism, and personal gain. But Lubitz had no “reason” to crash flight 9525 and kill everyone on board — he was mentally ill, severely depressed, and he wanted to die.

“In this case, there are no understandable reasons for his actions,” said Malle, “and that’s dissatisfying because it fundamentally leaves us in flux, with no answer to our ‘why.'”

There are a few more exacerbating factors that make it so unsatisfactory to let the lack of answers — or the blame — end with Lubitz.

“There are no understandable reasons for [Lubitz’s] actions, and that’s dissatisfying”

“This situation is especially difficult because the person who is obviously responsible killed 150 people and himself,” said Lawrence Solan, professor of law at Brooklyn Law School and author of Cognitive Foundations of the Impulse to Blame. “We now know he was mentally ill, but if you’re that sick it’s hard to ascribe moral responsibility.”

The reality Solan describes is reflected in our legal system’s concepts of culpability: we do not think blame or accountability should be placed on people with diminished mental capacity; we place little or no blame on children, those with developmental disabilities, or, in some circumstances, those with mental and emotional problems.

Moreover, as was the case with not only Lubitz but also Lanza and Rodger, the event culminated with their suicides. Not only does that mean we can’t ask the perpetrators for answers — or punish them for their actions — it also means we can’t satisfactorily answer the other side of the blame equation: how do we keep this from happening again?

“If you see someone who is perfectly fine in principle but then kills himself, in theory you suddenly look around, see a person you care about that seems fine and you have to think, ‘They could kill themselves, too,'” said Malle.

Combine this with the magnitude of an event like the Germanwings crash or Sandy Hook, and you have blame that can be extended much further than if Lubitz had just been in a plane by himself or with one other co-pilot.

The cascading chain of blame

This is what leads us to look down the chain of causality, to those with varying degrees of obligation to act as Lubitz began his sabotage. The closest actor — the co-pilot, who we now know died as he tried desperately to reenter the cockpit and stop Lubitz — was physically blocked from preventing the attack, and therefore unlikely to be blamed.

From there, we look to people and agencies who might have known about Lubitz’s mental state but let him fly anyway: to his employer, Germanwings, which could have or should have known about his mental health issues; to the employer’s parent company, Lufthansa, for instituting policies that might have contributed to Germanwings’ failure to ask questions; to Lubitz’s doctors, for not alerting anyone of his potentially worsening state; and to his family and friends, for not noticing the signs of suicide or deep depression.

Finally, when and if it is hard to ascribe blame to the above, we look even wider: to the flight agency policies that approved his licensing and made rules that allowed just one pilot to be in the cockpit, or to the German state, which created strict privacy laws that made it difficult to suss out or disclose this kind of mental illness.

For some of these questions, the answer can come from the legal system. Germanwings, via its parent company Lufthansa, is subject to strict liability laws — which means Lufthansa will have to prove that it could have done almost nothing to prevent the crash. That’s a very difficult task, and a reason almost every modern airline crash case settles out of court with the victims’ families and estates. But if the law recognizes that the airlines are virtually de facto to blame, why are we still looking for answers?

“If you have an impulse to blame, the criteria you look at will also trigger decisions about legal responsibility — but they’re not really always the same thing,” says Solan. “Assigning them moral responsibility is more satisfying, and it also helps answer the question of how to prevent this from happening again.”

This is the second part of blame. If the search for the answer to “why” is a primary emotional response to anger and sadness, then this is the more “rational” part that tries to gain a semblance of control, to soothe the anxiety and panic that emerge in the wake of such a massive tragedy.

The problem of trying to “fix” a tragedy

In the Germanwings case, lots of possible solutions have been proposed to prevent similar future situations, and some have already been put in place. In Europe, at least four airlines have announced they will adopt new cockpit rules to require two pilots in the cockpit at all times while in flight. Discussion about redesigning cockpit doors has begun. And calls to change legislation around mental health privacy has been proposed in Germany.

The impulse to “fix” the problem so it doesn’t happen again is a natural one. But it is unclear whether it is a wise one.

“The result is you wind up with laws that are hyper-reactive to situations,” explains Solan. “The traditional notion of Madisonian republicanism is precisely because if you do things too quickly, in reaction to immediate situations, you’ll respond to blame impulse and not more contemplative pragmatic ideas.”

While we can’t quash our impulse to blame, we can do something about how we react to it — by slowing down

The potential folly of hastily passed laws in response to tragedy is ironic in the Germanwings case. As even Lubitz discovered in his searches prior to crashing the plane, the reinforced cockpit doors that his co-pilot was unable to open were mandatorily reinforced in all aircraft only a month after the 9/11 attacks on the United States. Today the doors are too strong; 10 years ago, they weren’t strong enough.

Pilots and aircraft experts have pointed out that even if Lubitz’s co-pilot had been present in the cockpit or able to open the doors, there is little he could have done. “It turns out that things can go bad very quickly in a jet. Pushing one button will cut off the fuel to an engine . . . [s]tomping on the rudder can break off the tail of an Airbus . . . [p] ushing forward on the stick or yoke can put the airplane into a 30-degree nose-down attitude within a few seconds. . . Automated systems can be disabled by pulling circuit breakers,” writes Philip Greenspun, a pilot and engineer who publishes a blog on flying and aviation. History teaches this, too: in 1999, EgyptAir Flight 990 crashed into the Atlantic Ocean when one co-pilot seized the controls, with the other pilot present, and flew the plane into the sea, killing all on board. But when you have an enormously tragic event where hundreds are killed, impulse to blame can trump this kind of expert and historical knowledge.

How we should actually respond to disasters

So what is a better way to go forward in the wake of such tragedies?

The answer, told to us by both history and cognitive psychology, would seem to be: very slowly and deliberately. A contrast to the story of Lubitz or Lanza is that of Dzhokhar Tsarnaev, the Boston Marathon bomber. In the months following the attack, surveys of Americans found that 70 percent favored the death penalty in his case — but two years later and in the wake of a guilty verdict, only 47 percent thought the death penalty should apply. The time and deliberation associated with a trial seems to have eased some of the initial impulse to blame or punish.

And while we can’t quash our impulse to blame — and wouldn’t want to — we can do something about how we react to it, simply by slowing down.

“[Mechanisms like] stretched-out investigations help people to calm down and think relatively rationally,” says Malle. “It helps them to acknowledge they can never be completely safe, and ultimately to learn to live with that.”

Lead image: Kateleen Foy/Getty Images; Robyn Beck/Getty Images; FBI image handout; Getty Images