Prospective Solutions to the Lack of Peer Review Rigor and Reproducibility in Science

The Vox article about PubPeer discussed how the platform provides a simple, yet interesting, solution to the lack of rigor in peer review. We as media consumers observe every single day how the court of public opinion makes for an important swing vote when judging the actions of the most elite within society. PubPeer capitalizes on this idea, though its court is filled with a much more niche audience of scientists. I wholeheartedly support this platform as bias and unreliable data is so easily missed during the peer review process. PubPeer provides additional support to ensure that articles that slipped through the cracks of a broken peer review process are retracted. I also see PubPeer as a mechanism to provide a forum for collaboration. Its electronic platform could be a nidus for the formation of collaborations between research groups. In my opinion, these types of collaborative research projects would foment independent replication of experiments and ultimately promote scientific rigor.

However, I believe the systematic problem with peer review that cannot be addressed by PubPeer is the issue surrounding publication bias and the fact that unpublished articles are enriched with null results. Changing this requires a paradigm shift in which scientists and editors embrace the knowledge that a null result provides whilst simultaneously railing against the mentality that “impact factors” reign supreme. As was mentioned in the Economist article about unreliable research, “researchers ought to be judged on the basis of the quality, not the quantity, of their work.” However, the scientific community needs to begin to realize that quality science does not always lead to hypothesis confirmation.

The Economist article about unreliable science also sparked thoughts on additional ways in which the peer review process could be improved on the front end. For example, better training should be provided to editors. In addition, editor performance should be incentivized. Lastly, journals should foster a closer working relationship with statisticians so that raw data of prospective journal articles can be cross-checked by a journal-affiliated statistician.

Overall, I believe that PubPeer and replication initiatives by PLoS ONE are good starting points when attempting to fix the peer review and replication problems that run rampant within the scientific community. However, as mentioned above, the most difficult change to make relates to the pervasive mentality about science that propagates these issues upstream of where our current initiatives are working.