The Replication Crisis

As scientists, we are faced with the ever-present demand to publish our findings – to publish them soon, and in the best journals. Without this, we learn, our labs will not be considered vibrant research communities. As graduate students, we are less and less marketable with fewer publications in high impact journals. This “publish-or-perish” culture lends itself to the current replication crisis that science finds itself in. Fewer and fewer scientific results are able to be reproduced in different hands, making it difficult to interpret which findings have true impact. In the endless race towards publication, we find ourselves taking shortcuts and sensationalizing small findings in order to make ourselves stand out. As a science community, we need to stop prioritizing publication over validity.

But it’s a tradition that is pretty well entrenched, and a status quo we can’t easily avoid. Scientists are hesitant to even attempt performing replication experiments, let alone submit them for publication. As Jeremy Berg points out, replicated results are not “sexy” results. As such, publishers are less likely to accept these papers, making scientists less likely to perform the experiments in the first place. If we want to see a change in quality standards of data output, we need to likewise find ways to value a replicated result ( or a failed to be replicated result) as highly as a novel one. Or at least make it available in a more public forum.

As Jared Horvath says in his recent Scientific American article, “ In reality, science progresses in subtle degrees, half-truths, and chance”. Without accepting these less-certain truths into the canon of scientific research, can we really expect to make scientific progress at the rate we need to? While we may doubt some of what is published, it at least allows us a certain mobility in the generation of new ideas. Without the stepping stones that the science published before us provides, it would be impossible to move forward with our own scientific thinking.  Yet, with our drive to avoid stagnancy, we are letting things slip through the cracks. Our job now is to find a way to produce replicable results without hindering the progression of modern science.

The best and simplest steps that we can take as scientists is to be transparent about the science that we are doing. The clearer we are about the methods we use to perform and interpret our experiments, not only will it be easier for others scientists to replicate our work, but we will be able to have more honest conversations about the relative merits and pitfalls of any given experiment. This in turn needs to be communicated to the public clearly and without sensationalizing.