Risky Reproducibility

Reproducibility is difficult. While reproducibility is one of the cornerstones of modern science it is much more difficult in practice than it is in theory. It has been well described how there is little incentive in the current structure of academic science to reproduce results, as touched upon by the article “Trouble in Science” by the Economist. The current system of ‘publish or perish’ leaves no room for the busy and aspiring scientist to spend any time replicating a previous study, as there is no reward and occasionally even backlash to questioning a result. However, one of the interesting points of the article was that even if there was a good system for rewarding reproducibility, some experiments are just difficult to reproduce, even if the original science is sound. This problem is particularly rampant in the biological sciences, where long and complicated procedures can be influenced by the slightest  factors that the authors may not even have realized were important. Things such as the microbiome of their mice (which is virtually impossible to control for), the particular humidity of their lab, or the ultra precise pipetting skills of a certain technician. These are the sort of methods that are overlooked and can be impossible to reproduce, even if the original author is highly cooperative in sharing their methods and results. This raises the uncomfortable notion that some scientific studies are basically irreproducible. The question then becomes can you place any trust in such studies? This becomes a pressing issue when you realize that many of science’s greatest discoveries fall into this category. The article by the Economist cited an example where drug company Amgen attempted to repeat the 50 greatest discoveries in cancer biology over the last few decades and failed to achieve the same results from all but 6 of those studies. This is a difficult issue, but one that must be addressed, or else we may find that there have been fundamental discoveries in biology that were wrong, and 50 years later all research based on that one discovery is also incorrect.

Link to Economist article:

https://www.economist.com/news/briefing/21588057-scientists-think-science-self-correcting-alarming-degree-it-not-trouble