The Elephant in the Room – Irreproducibility

As a scientist-in-training, it’s hard to ignore the issues of bias and reproducibility in the field. Whether I am at an ethics workshop or keeping up to date with my social media, I always seem to come across an article reminding me that scientists are living in dark times. But how much of an issue is irreproducibility? According to an article in Nature, 90% of 1,576 researchers surveyed think there is a reproducibility crisis, with 52% believing it’s a significant crisis. As for my field, a research article published in PLoS Biology by Freedman et. al. estimates that at least 50% of preclinical research is irreproducible. As someone who works with mouse models, this figure hits close to home. It turns out that differences in mouse feed and mouse microbes can actually make it difficult to replicate in vivo mouse experiments. If this itself wasn't a problem, the authors of the PLoS Biology article also estimated that this results in about $28 billion dollars spent a year on irreproducible research. In a time where funding is extremely tight, this amount is ridiculous. All of these issues are also added on top of the fact that lack of reproducibility is costing us time. Even though it is important that biomedical research be replicated, having to replicate most of the research already out there is taking time away from more pressing issues and could hinder progress. Although there are factors that can't be controlled which might lead to lack of reproducibility, the Nature article also pointed out that most of the factors that do contribute to this problem are those that we can control. It’s hard to believe that scientists have created more problems than they have solved.  

After doing more research on the topic, I believe that the problem of reproducibility and bias in science is significant and can’t be underestimated. It is difficult to accept that a lot of research in my field can’t be reproduced, but, this does not mean that all hope should be lost. Work is already underway to help tackle some of the causes of irreproducibility. I applaud Emory for implementing ethics seminars and requiring an ethics program to be completed before graduation. I also applaud scientific journals such as Nature and Science, and funding bodies such as the NIH for taking action to ameliorate the situation. Ultimately, however, I believe that these changes won’t be enough in the long run. To truly work towards ending this problem, we need to start at the most basic level – with the scientist. This is the reason why I particularly liked Jeff Leek’s article, his suggestions seemed very reasonable and not too tedious to implement. One suggestion that was particularly interesting to me is to stop publicizing our scientific results as miracle solutions. This directly feeds into Julia Belluz’s Vox article about “revolutionary” cancer drugs that are not really what they claim to be. This could be a result of our bias, which is another important issue. Scientists, much like everyone else, are not immune to having biases. While we can’t always avoid having them, recognizing them and overcoming them is already a step in the right direction. Finally, I believe we need to work together on changing the culture around publishing, which places a lot of stress on scientists and definitely contributes to irreproducibility. I am confident that with this new generation of scientists can make headway on these problems!