Re-thinking the peer-review process

Ideally, science is defined as systematic, neutral and reproducible. Yet, it is becoming increasingly more apparent that these features are lacking even in studies published by high-profile scientific journals. It is not difficult to suggest reasons that could account for the irreproducibility and bias across scientific fields: poor peer-reviews, funding pressure and lack of publications of replication studies. In The Economist, a journalist reports on why science is not as self-correcting as we think, pointing our attention to various aspects of the publication process and execution of science. The article touches on some interesting statistics, which help us understand why poorly performed, and even falsified studies, make it through the peer-review process ultimately getting published. Specifically, the journalist points to the fact that over time, peer reviewers become worse at identifying errors and holes in the articles they are reviewing. While The Economist calls it “carelessness”, one has to acknowledge that these peer-reviewers are usually selected because of their respectable position at a research-intensive institution. Peer-reviewers lead laboratories that need funding and student mentoring and they may also have teaching responsibilities. The fact that peer reviewers are “busy with their own life” does obviously not justify their insufficient analysis of research studies; however, it does suggest that selecting “disinterested” scientists as peer-reviewers is not the optimal way to ensure sound science is published. On Vox.com, the journalist Julia Belluz interviews the founders of PubPeer, a blog that facilitates an anonymous discussion of published journals’ scientific methods and findings. What is inspiring about this blog, is that it enables a neutral platform for ongoing discussion based on insightful comments and well-constructed arguments. Moreover, these discussions are not dominated by high-profile researchers that tend to dominate (and bias) specific scientific fields. An obvious problem with PubPeer is that it is entirely voluntary and therefore unreliable in its consistency and standards. However, PubPeer inspires us to re-consider the traditional peer review process. Perhaps, instead of “careless and flawed” high-profile peer-reviewers, independent organizations should be in place to fulfill the practices of peer-reviewing. Specifically, such an independent and neutral institution can ensure that replication studies are given the necessary attention, for example, even if journals are not interested in publishing the findings, they can enhance their public attention through the media. Although history can suggest that science was conceived through bias and irreproducility (see  Scientific American), it does not mean the present scientific community should not improve and develop their standards for the research that is being performed and, as importantly, communicated to the public.