Bias and Reproduciblity

An essential piece of the scientific method is forming a hypothesis, an educated guess regarding the outcome of an experiment that you have not yet performed. In doing so, we create bias before we even pick up a pipette – we want our results to be statistically significant so that we can reject the null. So that our experiment means something - so that we can publish (preferably in a high impact journal). To graduate, to secure funding, to make an impact... As scientists, we don’t just want to publish - we need to publish for the sake of our careers. But as scientists, we also have a responsibility to ensure that what we are publishing is true, reproducible, and free from bias. So how do we merge those two ideas? How do we ensure that we are performing high-quality research while also meeting the pressure to publish (preferably in a high impact journal)?

Science is not easy, and sometimes science just doesn’t work. Not every experiment goes as we expect it to. Sometimes our results don’t make sense – or worse contradict the story that we are trying to tell. Then when our experiments do go as we “want” them to and we are able to publish, we face the problem of reproducibility.

In his piece about the reliability of scientific research, Jeremy Berg writes of a study performed by the pharmaceutical industry in which only 10-25% of the key findings of published preclinical cancer research could be reproduced by independent scientists. While those results may be shocking to someone outside of science, as someone that has tried and failed to replicate a published method (more than once) I’m not surprised.

I think recognizing that these problems with both bias and reproducibility exist is important, and we as scientists need to find a solution to overcoming it. I’m not sure what that solution is, but in 2016, Cell implimented their STAR★METHODS requiring all publications to provide a detailed account of their methods including the reagents that were used and I think that this is a good step in the right direction.