A new study finds there are too many scientific studies

There might be too many scientific studies out there. That’s the upshot of one new study — though more research will no doubt be necessary.

In a new working paper published on ArXiv, researchers from Finland and California argue that the sheer amount of published research is growing so rapidly that scientists appear to be losing their ability to keep track of all the work that’s been done.

Here’s the basic idea: important scientific papers typically get cited a fair bit by other researchers shortly after publication, but then get cited less frequently over time.

Why is this? One possibility is they’re simply getting surpassed by fresher, superior research. But previous studies have found this isn’t necessarily the case — the shape of the “decay” in citations suggests scientists are simply forgetting about older papers, much like you might forget about, say, a terrific novel you read six years ago.

In this new ArXiv paper, the authors explore whether this decay issue has been getting worse over time. First, they note the amount of published research in medicine, biology, chemistry, and physics keeps growing. (There’s evidence that a lot of this research is low-quality or redundant, but let’s set that aside for now.)

What’s more, they find that the rate of decay for citations actually appears to be speeding up in these fields over time — and this decay seems to be largely dependent on how many papers are published:

This all suggests that too much new research can actually be tough to follow. Scientists are having trouble keeping up with the onslaught of studies and tend to forget older studies more quickly.

In this way, it’s sort of like ... the Internet. “Over the past years, thanks to the Internet, a huge amount of data has allowed a thorough investigation of the dynamics of collective attention to online content, ranging from news stories to videos and memes,” they write. “Here attention is measured by the number of users’ views, visits, posts, downloads, tweets. It is also noted that the attention decays over time, not only because novelty fades, but also because the human capacity to pay attention to new content is limited.”

Now, granted, this ArXiv paper is only one single study. And one potential caveat here is that the researchers mainly looked at very broad fields like “chemistry” and “medicine.” To really see if scientists are getting inundated with publications and forgetting stuff, you’d want to narrow in on specific subfields that individual scientists actually follow closely.

So no doubt we’ll need further studies on the question.

(For the link, thanks to Chris Matyszczyk at CNET)

—Q&A with John Ioannidis, who has dedicated his life to quantifying how science is broken.