Troll hunters: the Twitterbots that fight against online abuse

Social media has a big problem with racist and misogynistic harassment and human intervention isn’t working. Time to bring in the bots

By Sally Adee

WHEN Ghostbusters actor Leslie Jones was hounded off Twitter last month, having braved several days of racist and misogynistic abuse, many people decried the social network’s failure to enforce its policies. If a star of a Hollywood blockbuster can be treated like that, what hope is there for the rest of us? For some, such cases show it’s time for a new approach to dealing with online abuse.

“Social media is a shitshow,” says Libby Hemphill, an online communication researcher at Illinois Institute of Technology.

The statistics may feel familiar: a 2014 study by Pew Research in Washington DC showed that 40 per cent of internet users have been harassed and 66 per cent of those said the most recent instance was on social media.

Since then, despite many promises made by internet companies, efforts to curb online harassment using human moderation have fallen flat. Last year, Twitter’s then CEO Dick Costolo took “personal responsibility” for the continuing abuse problems on his site. “We suck at dealing with abuse and trolls on the platform and we’ve sucked at it for years,” he told employees in an internal memo.

“If a star of a Hollywood blockbuster can be treated like that, what hope is there for the rest of us?“

But the problems haven’t stopped. That’s because social networks are too large to police by hand and their approach tends to be reactive rather than proactive. So an increasing number of people are turning to another solution: bots.

The simplest way bots can help is with block lists, which specify the accounts you don’t want to see in your feed. You can block accounts yourself. But reporting them to prevent harassment of others is a hassle. For example, a form must be filled out for each abusive tweet. Apart from being slow, it’s also unpleasant – someone may have to trawl through hundreds of personal slurs, reporting each individually.

It would be better if you didn’t receive abusive messages in the first place – something bots could help with by managing block lists automatically. Subscribe to a blockbot, which continually updates a list of accounts blocked by other users, and you should receive less invective. But that approach only works if someone adds an abusive account to the block list. Is it possible to automate the detection of harassment?

Hemphill and her colleagues tried to do this by first asking people on crowdsourcing platform Mechanical Turk to identify instances of abuse. But they hit a snag: there was less agreement between crowdworkers than they would have liked. “Humans don’t agree on what constitutes harassment,” she says. “So it’s really hard to train computers to detect it.”

Feed the trolls

Enter the argue-bots. These distract trolls from their human victims by drawing their attention and engaging with them, often with entertaining results.

One, called @Assbot, recombined tweets from its human creator’s archive into random statements and then used these to respond to tweets coming from Donald Trump. The result was a torrent of angry Trump supporters engaging with a bot spouting nonsense.

@Assbot simply deployed a mishmash of existing tweets. But what if it had been smarter? Kevin Munger at New York University is interested in group identity on the internet. Offline, we signal which social groups we belong to with things like in-jokes, insider knowledge, clothes, mannerisms and so on. When we communicate online, all of that collapses into what we type. “Basically, the only way to affiliate yourself is with the words you use,” says Munger.

This squares with research on online abuse. Sometimes it is malicious, intentional and directed against specific minorities. But other times it functions more as a way to signal group affiliation and affinity. “Abuse is less of a problem for niche sites than a catch-all platform like Twitter,” says Hemphill. “Group dynamics and norms of behaviour have already been established there.”

So Munger wondered if he could create a bot to manipulate a troll’s sense of group dynamics online. The idea was to create bots that would admonish people who tweeted racist comments – by impersonating a higher-status individual from their in-group.

First he found his racists. He identified Twitter accounts that had recently issued a racist tweet, then combed through their previous 1000 tweets to check that the user met his standards for abuse and racism. “I hand-coded all of them to make sure I didn’t have false positives,” he says.

He then created four bot accounts, each with a different identity: white male with many followers, white male with few followers, black male with many followers and black male with few followers. To make his automated account look legit, he bought dummy followers from a website. “They were $1 for 500,” he says.

At first, people turned on the bots. It was unnerving, he says. “The most common response was ‘kill yourself’.” But something seems to have sunk in. After a short-term increase in racist language, he found that abusers who were admonished by a bot that appeared to be a high-status white male reduced their use of racist slurs. In the month after the intervention, these people tweeted the n-word 186 fewer times on average than those sanctioned by a bot that appeared to be a low-status white male or a black male.

“Abusers sanctioned by a bot impersonating a high-status white male reduced their use of racist slurs“

“It’s useful to know that such an impersonator-bot can reduce the use of slurs,” says Hemphill. “I’m surprised the effects didn’t decay faster.”

It doesn’t work on everybody, however. “The committed racists didn’t stop being racist,” says Munger. Another problem, as Hemphill found, is that identifying abuse is hard. Munger wanted to target misogyny as well as racism but gave up when he found words like “bitch” and “whore” were so widespread that it was impossible to distinguish genuine abuse from casual chat.

There is also an inherent weakness in the system. “The more people become aware that these are out there, the less effective they’ll be,” says Munger.

For now, Hemphill thinks it’s the best we can do. “Nothing else is working,” she says. “We may as well start using bots.” But Munger doesn’t wants bots to be the endgame. “I don’t envision an army of bots telling people to behave themselves,” he says. “I’m looking for what works, so we can figure out what kind of language and moral reasoning works best to stop a racist troll.”

Munger is now looking at politics-based abuse. He has his work cut out.

Who controls the bots?

Bots may be set to tackle harassment online (see main story), but methods to deflect abuse and manipulate behaviour could themselves be abused. Who should control them?