Reality check on Reproducibility

In 2016, the article, “Reality check on Reproducibility” was published as an editorial in Nature as a response to the 2016 article, “1500 scientists lift the lid on reproducibility.”  This survey firsts ask if there a reproducibility crisis in science, to which roughly half of the respondents answer, “Yes, a significant crisis.”  Surprisingly there was a large percentage of respondents who believe the crisis is only slight or does not exist.  As a graduate student, it is quite interesting to read scientists comments stating that “failing to reproduce results is a rite of passage.”  While this may seem like a valuable learning process for students and post-docs, if they are fully unable to reproduce results, this lends its hand to the larger reproducibility problem.  

To fix reproducibility, we must first collectively define reproducibility with the same language.  When referring to reproducibility, it must consider empirical, conditional and statistical aspects of experiments.  However, even then the criteria for reproducibility are subjective between researchers.  To help fix the crisis with reproducibility, we must all define the term and its criteria the same.  In addition, students should be taught the basics of reproducibility in an experimental design class to provide the necessary foundation for a basic understanding. 

The editorial points out that “Senior scientists will not expect each tumour sample they examine under a microscope to look exactly like the images presented in a scientific publication; less experienced scientists might worry that such a result shows lack of reproducibility.”  As a graduate student who looks to publications to compare my work to, it is very hard to reproduce certain findings because of the expectation for my results to look exactly like the images published.  A well written publication that correctly reports its design and conclusions should be reproducible in empirical, conditional, and statistical aspects.  If it is not, communication with the authors of the published document is essential.  After communication with authors, if the results are still not reproducible, the publication opens itself up for retraction.  To avoid this humiliating experience, it is important for labs to take a step back from novel questions and focus on reproducibility within their respective labs.  It is important for members of one’s own lab to be the harshest critic on other members’ work.  This is just one aspect among many others that will help limit the reproducibility crisis in research.

https://www.nature.com/news/1-500-scientists-lift-the-lid-on-reproducibility-1.19970

https://www.nature.com/news/reality-check-on-reproducibility-1.19961