Tuesday, January 19, 2016

15 Seconds of Significance



As pointed out by many, a reigning mentality in science is “publish-or-perish.” The pressure to publish results, positive ones at that, is high. And as a graduate student, I find myself being taught that this how science is conducted.
To earn my PhD, I have to publish.
The publications generated during my years in graduate school are supposed to be a way for me to demonstrate what I have learned. Sadly, publications are frequently plagued by significance bias.
The bias of significance can mean many things. The straightforward interpretation is that of statistical significance. Any positive data published must reach significance, and often times it is reached unethically. I agree with Victoria Stodden, scientists often do not use appropriate statistical methods to interpret their data. But how a scientist comes to use the wrong method is varied and is normally born of ignorance and choice. Inappropriate methods and fishing for significance is a frequent occurrence in science, and it is often caused and perpetuated by a scientist’s pressure to publish.
Another aspect of the significance bias is the drive for something to be novel. What is often valued in science is something that shakes the foundation of its field. Not so surprisingly, many publications that do this are often untruthful, irreproducible or flawed. Replication of data is important to determine the truth in a situation and no scientist should be afraid of it. Horvath stated that, “If replication were the gold standard of scientific progress, we would still be banging our heads against our benches trying to arrive at the precise values that Galileo reported. Clearly this isn’t the case.” I disagree, replication is a gold standard for the progress of science. Replication by many is simply more evidence that what was observed is real, and we can come to accept it as fact and move on from it. And yet, science as a community rewards those who publish in high impact journals, even if there data is wrong. The rewards of more funding and notoriety just reinforce the “publish-or-perish” mentality.
At present, science is stuck in the mentality of “publish-or-perish.” It is unlikely, that this and our obsession with significance will ever change, until science is no longer approached as a for profit business.

2 comments:

  1. I definitely agree that the "publish or perish" mentality sets up for a lot of pressure to deliver data that doesn't always find significance, truth, or is reproducible. As a student having to achieve publications is the metric to success, so I wonder what other items may be better serve as a goal to establish success in the field. Is there another method we can use to evaluate students and young scientists that may help to establish an environment that does not hinge so heavily on publications? Should retractions and incorrect data negatively affect grant funding, future publications acceptance?

    ReplyDelete
  2. The mentality that our worth as a scientist comes purely from the number of publications can be flawed. It often neglects to take into account the difficulty of the experiments and techniques used by the scientist and can lead to bias. Even though some rules and regulations exist like peer review that attempt to filter these "flawed" findings I find it important to note the movement towards ethical science. It is often times that mistakes come from ignorance; therefore, I applaud institutional attempts to decrease the bias towards significance. For example, the Emory Integrity project as well as multiple programs by the Ethics office of Emory attempt to educate scientist on ethical and unbiased research methods.

    ReplyDelete