Publish... and Perish?

We like to think that scientists are innately good people, spending countless hours at the bench to help catapult us into the next generation of life-saving medication or medical procedures. On the surface it seems wholesome and altruistic, but diving deeper into the scientific community it becomes apparent that there is a very large elephant in the room: the issue of bias and irreproducibility. In an article published by The Economist the author describes the cut-throat culture that academia has established and how it leads to bias; for example, the motto “publish or perish” may influence researchers to embellish their work in order to publish in high-impact journals or to even publish at all. These high-impact journals then fight back by having egregiously high rates of rejection for manuscripts, leading researchers to cherry-pick their data further to make the cut. The author then goes on to state that companies like Bayer and Amgen failed to replicate more than half of studies they found on breakthrough cancer research, a section of research that is highly esteemed by scientists and the general population alike. 

 The scientific community has created a vicious cycle that seems to keep growing. This immense amount of pressure is leading scientists to falsify or alter data to fit a specific agenda, and soon it will cost them more than their reputation in the field. Flawed research costs us time, money, resources, and the trust of the general population. This puts scientists at a bit of a crossroads, but I think it is up to us to begin making the changes necessary to fight bias. Ethics should be taken more seriously and started even before entering graduate school even though sometimes it can seem like a “no brainer.” Additionally, having a grasp on statistical analysis is imperative for all scientists and not just the PI; if more people understand statistics then it may yield more sound data or make it easier to spot falsified data instead of relying on someone’s best judgement. 

The Replication Crisis

As scientists, we are faced with the ever-present demand to publish our findings – to publish them soon, and in the best journals. Without this, we learn, our labs will not be considered vibrant research communities. As graduate students, we are less and less marketable with fewer publications in high impact journals. This “publish-or-perish” culture lends itself to the current replication crisis that science finds itself in. Fewer and fewer scientific results are able to be reproduced in different hands, making it difficult to interpret which findings have true impact. In the endless race towards publication, we find ourselves taking shortcuts and sensationalizing small findings in order to make ourselves stand out. As a science community, we need to stop prioritizing publication over validity.

But it’s a tradition that is pretty well entrenched, and a status quo we can’t easily avoid. Scientists are hesitant to even attempt performing replication experiments, let alone submit them for publication. As Jeremy Berg points out, replicated results are not “sexy” results. As such, publishers are less likely to accept these papers, making scientists less likely to perform the experiments in the first place. If we want to see a change in quality standards of data output, we need to likewise find ways to value a replicated result ( or a failed to be replicated result) as highly as a novel one. Or at least make it available in a more public forum.

As Jared Horvath says in his recent Scientific American article, “ In reality, science progresses in subtle degrees, half-truths, and chance”. Without accepting these less-certain truths into the canon of scientific research, can we really expect to make scientific progress at the rate we need to? While we may doubt some of what is published, it at least allows us a certain mobility in the generation of new ideas. Without the stepping stones that the science published before us provides, it would be impossible to move forward with our own scientific thinking.  Yet, with our drive to avoid stagnancy, we are letting things slip through the cracks. Our job now is to find a way to produce replicable results without hindering the progression of modern science.

The best and simplest steps that we can take as scientists is to be transparent about the science that we are doing. The clearer we are about the methods we use to perform and interpret our experiments, not only will it be easier for others scientists to replicate our work, but we will be able to have more honest conversations about the relative merits and pitfalls of any given experiment. This in turn needs to be communicated to the public clearly and without sensationalizing.

Reproducibility… or rather accountability “crisis”

This week’s readings on bias and the reproducibility “crisis” were perhaps more eye opening than anything else. I realized how completely unaware I was on the issue of experimental reproducibility, which includes myself as part of the larger scientific community. I knew the issue of scientific reproducibility was a problem. In general, though, I hear about it in the context of the relatively small percentage that fabricate data and have their papers retracted, rather than the unknowing majority that are unsure of how to properly design experiments and to process resultant data. The scientific market to publish data that is “reproducing” or reaffirming an existing finding rather than a novel discovery is simply non-existent, which furthers the problem. To think that I, personally, could unknowingly be contributing to this frankly made me sick.

We as scientists are pushed to publish… or perish. We need to publish papers to receive grants, we need funding to pay researchers, and the vicious cycle repeats. However, all of this is built on the foundation that we publish almost exclusively positive results. A slew of negative results are not going to pay the bills. The Economist article, “Unreliable Research: Trouble at the Lab,” broke down this argument of only publishing the positive results into numbers. Only then did I get a real look into the glaring problem. After number crunching, I realized that even a small number of false positives in a data set can artificially inflate positive results; therefore, negative results can prove to be far more trustworthy but, unfortunately, often remain unpublished.

Jared Horvath penned a more optimistic perspective, in which he argued that many studies done by the likes of Galileo, Dalton, and others were not reproducible by scientists that followed. In these cases, the fact that their original works were not reproducible did not make the original scientist’s discoveries incorrect, but instead their findings and theories sparked further questions and other theories for future scientists to pursue. Horvath’s outlook was refreshing: “if replication were the gold standard of scientific progress, we would still be banging our heads against our benches trying to arrive at the precise values Galileo reported.” We should focus less on how reproducible a specific result is and more on holding ourselves accountable for the experiments we design, the results we publish, and the literature we review by taking the proper precautions to combat our inherent biases.