Skip to main content

The context you need, when you need it

When news breaks, you need to understand what actually matters — and what to do about it. At Vox, our mission to help you make sense of the world has never been more vital. But we can’t do it on our own.

We rely on readers like you to fund our journalism. Will you support our work and become a Vox Member today?

Join now

Fleming’s discovery of penicillin couldn’t get published today. That’s a huge problem.

Alex_Traksel/Shutterstock

After toiling away for months on revisions for a single academic paper, Columbia University economist Chris Blattman started wondering about the direction of his work.

He had submitted the paper in question to one of the top economics journals earlier this year. In return, he had gotten back nearly 30 pages of single-space comments from peer reviewers (experts in the field who provide feedback on a scientific manuscript). It had taken two or three days a week over three months to address them all.

So Blattman asked himself some simple but profound questions: Was all this work on a single study really worth it? Was it best to spend months revising one study — or could that time have been better spent on publishing multiple smaller studies? He wrote about the conundrum on his blog:

Some days my field feels like an arms race to make each experiment more thorough and technically impressive, with more and more attention to formal theories, structural models, pre-analysis plans, and (most recently) multiple hypothesis testing. The list goes on. In part we push because want to do better work. Plus, how else to get published in the best places and earn the respect of your peers?

It seems to me that all of this is pushing social scientists to produce better quality experiments and more accurate answers. But it’s also raising the size and cost and time of any one experiment.

Over the phone, Blattman explained to me that in the age of “big data,” high-quality scientific journals are increasingly pushing for large-scale, comprehensive studies, usually involving hundreds or thousands of participants. And he’s now questioning whether a course correction is needed.

Though he can’t prove it yet, he suspects social science has made a trade-off: Big, time-consuming studies are coming at the cost of smaller and cheaper studies that, taken together, may be just as valuable and perhaps more applicable (or what researchers call “generalizable”) to more people and places.

Do we need more “small” science?

Over in Switzerland, Alzheimer’s researcher Lawrence Rajendran has been asking himself a similar question: Should science be smaller again? Rajendran, who heads a laboratory at the University of Zurich, recently founded a journal called Matters. Set to launch in early 2016, the journal aims to publish “the true unit of science” — the observation.

Rajendran notes that Alexander Fleming’s simple observation that penicillin mold seemed to kill off bacteria in his petri dish could never be published today, even though it led to the discovery of lifesaving antibiotics. That’s because today’s journals want lots of data and positive results that fit into an overarching narrative (what Rajendran calls “storytelling”) before they’ll publish a given study.

“You would have to solve the structure of penicillin or find the mechanism of action,” he added.

But research is complex, and scientific findings may not fit into a neat story — at least not right away. So Rajendran and the staff at Matters hope scientists will be able to share insights in this journal that they may not been able to publish otherwise. He also thinks that if researchers have a place to explore preliminary observations, they may not feel as much pressure to exaggerate their findings in order to add all-important publications to their CVs.

Smaller isn’t always better

Science has many structural problems to grapple with right now: The peer review system doesn’t function all that well, many studies are poorly designed so their answers are unreliable, and replications of experiments are difficult to execute and very often fail. Researchers have estimated that about $200 billion — or about 85 percent of global spending on research — is routinely wasted on poorly designed and redundant studies.

A big part of the reason science funders started emphasizing large-scale studies is because they were trying to avoid common problems with smaller studies: The results aren’t statistically significant, and the sample sizes may be too tiny and therefore unrepresentative.

It’s not clear that emphasizing smaller-scale studies and observations will solve these problems. In fact, publishing more observations may just add to the noise. But as Rajendran says, it’s very possible that important insights are being lost in the push toward large-scale science. “Science can be small, big, cure diseases,” he said. “It can just be curiosity-driven. Academic journals shouldn’t block the communication of small scientific observations.”

See More:

More in Science

Future Perfect
Human bodies aren’t ready to travel to Mars. Space medicine can help.Human bodies aren’t ready to travel to Mars. Space medicine can help.
Future Perfect

Protecting astronauts in space — and maybe even Mars — will help transform health on Earth.

By Shayna Korol
Podcasts
The importance of space toilets, explainedThe importance of space toilets, explained
Podcast
Podcasts

Houston, we have a plumbing problem.

By Peter Balonon-Rosen and Sean Rameswaram
Climate
How climate science is sneakily getting funded under TrumpHow climate science is sneakily getting funded under Trump
Climate

Scientists are keeping their climate work alive by any other name.

By Kate Yoder, Ayurella Horn-Muller and 1 more
Good Medicine
You can’t really “train” your brain. Here’s what you can do instead.You can’t really “train” your brain. Here’s what you can do instead.
Good Medicine

The best ways to protect your cognitive health might surprise you.

By Dylan Scott
Future Perfect
Humanity’s return to the moon is a deeply religious missionHumanity’s return to the moon is a deeply religious mission
Future Perfect

Space barons like Jeff Bezos and Elon Musk don’t seem religious. But their quest to colonize outer space is.

By Sigal Samuel
Health
Why the new GLP-1 pill is such a big dealWhy the new GLP-1 pill is such a big deal
Health

The FDA just approved Foundayo. Here’s what it can and can’t do.

By Dylan Scott