The coming crackdown on click fraud is going to hurt your favorite indie website

Bloomberg has a fantastic feature story on the internet’s epidemic of click fraud. Over the past decade, the internet’s infrastructure for selling and displaying ads has become increasingly complex and increasingly automated. That, in turn, has created growing opportunities for scammy behavior.

For example, companies create websites that humans hardly ever visit, at least on purpose, and then pay other companies to generate cheap, low-quality traffic to them. Sometimes traffic brokers use shady techniques like pop-under windows to trick users into opening a page. In other cases, the traffic is purely automated. The upshot is that advertisers wind up paying for ads that will never be seen by any human being.

This kind of scammy behavior is made possible by ad networks that automate the process of buying and selling ad space. Advertisers use ad networks because they want to reach large numbers of people and don’t have time to negotiate deals with individual websites. But that also means they don’t have time to carefully vet the websites that are showing their ads, leaving room for fraud.

Of course, advertisers aren’t stupid. They want their ads shown to human beings, not botnets, and over time they’re going to be looking for better ways to combat ad fraud.

One consequence of this is that larger websites — and media companies whose sites collectively reach a lot of people — are going to have a growing advantage over smaller, independent sites. Advertisers can be pretty sure that a well-known brand like the New York Times or BuzzFeed won’t sell them fake traffic. So buying directly from publishers makes them less vulnerable to click fraud than buying through ad networks.

But the people buying the ads only have so many hours in the day. So given a choice between buying from a few big publishers or a lot of smaller ones, they’d much rather deal with large publishers. Indeed, the convenience of one-stop shopping will likely allow larger publishers to charge a bit more per reader than smaller publishers can. This is a big reason why media organizations — including my employer, Vox Media — are rushing to get bigger.

And as Vox’s Matt Yglesias wrote last week, the growing popularity of ad blockers is likely to push in the same direction. Ad blockers do the most damage to sites that rely on automated ad networks to sell ads for them. The largest media companies will be able to adapt by selling ads themselves and making them look more like legitimate content. Indeed, the largest publishers might even benefit from the reduced competition for ad dollars.

This is likely to be a mixed blessing for readers. Obviously, it wouldn’t be great for readers if a bunch of small, independent websites were forced out of business. On the other hand, consolidation in the media industry could ultimately raise the average quality of online journalism.

Many readers complain about the internet’s “clickbait” problem, where publications seem more focused on generating clicks than on delivering quality content to readers who do click. Larger media companies that sell directly to advertisers have more reason to worry about the quality of the audience they’re building, because they want repeat business and know advertisers hate clickbait as much as readers do. So if the advertising market drives consolidation in the media world, it could ultimately raise the average quality of online journalism.