Combating Bias and Irreproducibility in Research

Recent reports by the news media have begun to shine light on a serious issue facing the scientific research community: bias and irreproducibility in research are very real problems that often result in wasted resources. During my time as a research technician in a highly competitive academic environment, I frequently witnessed the ‘forces’ that I believe are responsible for these problems: 1) a ‘publish-or-perish’ culture that encourages academic faculty (particularly junior faculty) to rapidly and continuously produce exciting results, sometimes at the expense of being painstaking in their research efforts; 2) scientists’ lack of substantive knowledge regarding how to use statistics to appropriately design experiments and analyze and interpret data; 3) opaque communication of experimental methodology; and 4) lack of communication of negative experimental results.

Moving forward, concrete steps can and should be taken to avoid bias and minimize irreproducibility wherever possible. While the first ‘force’ mentioned above would be impossible to change quickly because it would require a dramatic shift in the deeply ingrained culture of science, I believe the second, third and fourth ‘forces’ could be readily addressed by 1) enhancing scientists’ training in statistics, 2) raising standards for data transparency, rigor and reproducibility, and 3) encouraging more communication between scientists. Fortunately (and also likely in response to increasing public attention to bias and irreproducibility in research), leading scientific journals and grant-funding institutions have already started to push scientists toward more rigorous statistical treatments of data and greater transparency in methodology. I’m optimistic that these efforts will help to positively shape the scientific research landscape such that current problems with bias and irreproducibility will be increasingly mitigated.

For the specific purpose of increasing transparency in scientific methodology and enhancing communication between scientists, another effort that I think could benefit the scientific community would be to establish an online forum dedicated to each scientific paper. In each forum, the different scientists attempting to reproduce the results of an original scientific paper could directly communicate both with one another and with the authors of the original paper. This forum could be used as a ‘safe space’ to ask for help in trouble-shooting problems that arise, to ask for clarifications in methodology, and, most importantly, to report conflicting or negative results. This approach has already been implemented to great effect by some research groups, e.g. the Zhang group for the development of CRISPR/Cas9 gene editing systems.

In sum, bias and irreproducibility in research are finally receiving their due attention from the public as real obstacles that must be addressed and overcome by the scientific community. I’m hopeful that current and future efforts undertaken by the scientific community to address issues in bias and irreproducibility will continue to enhance the quality of published scientific research.