Showing posts with label misleading headlines. Show all posts
Showing posts with label misleading headlines. Show all posts

Tuesday, January 19, 2016

Intrinsically Intertwined

After reading the posted articles on irreproducibility and bias in science, I am surprised that there are not further measures in place to combat these issues. The use of anonymous post-publication peer review and Bayesian statistics to justify redoing an experiment seem like common sense measures. Why have these not become standard practice among the scientific community?

As the article “Trouble in the Lab” states, “more than half of positive results could be wrong.” This was revealed by John Ionnidis’ 2005 paper, which proved the cost of a seemingly small number of false positives. When I connect this thought to my own research, I am horrified. What if the claims that helped me to develop my experimental theory are unreliable? Though they were published in peer-reviewed journals, perhaps their results do not reflect actuality. These “discoveries” might have not been discoveries at all, but simply instances in which the data told an incorrect story. Because research builds on previously published results, a false published result could lead to a chain reaction on incorrect assumptions. How can this chain of events be halted?


Statistics proved that irreproducibility is a rampant issue among many life science research investigations. I believe that statistics can similarly be used to combat this problem. Even though “most scientists are not statisticians,” acceptance of the explanation laid out by Ionnidis should be a prerequisite for performing research. It should guide scientists to perform more experiments and to not be fooled by false positives. Perhaps, then, greater care will be made to distinguish discoveries made by statistical anomaly from discoveries that represent the laws of science. Because of the nature of their work, scientists should take it upon themselves to become as versed as possible in statistics. The two are so intrinsically intertwined—this fact can no longer be ignored in the scientific community.

The Paradox of Trust in Science

As an undergraduate student, I find that most, if not all, of my lectures incorporate information which students naturally presume is credible, reliable evidence. Very rarely does a professor inquire about the shortcomings of data or urge us to approach publications from a critical perspective. Upon reading a PubPeer article and learning that 25% of 120 randomly selected published papers revealed image data errors made me ponder why we constantly accepted literature as truth. I was frequently under the impression that if a study was published, it underwent a rigorous review process and was subsequently immortalized in an impressive journal....thus what was there to question? It was only after I joined a lab when I began to pause and truly assess studies within the growing scope of my knowledge. I also became increasingly aware of one of the greatest challenges facing research scientists: striking a balance between the demanding pressure to publish and being conscientious about data production throughout. 

The case of the overhyped medical press does not surprise me, however. I personally have fallen prey to such articles, namely those that occupy headlines with "breakthrough findings" that daily use items are carcinogenic. It's no secret that journalists are cognizant of how to grasp the layman's attention, albeit with content that isn't scientifically sound. Often, such articles are followed by disclaimers that, for example, the hot new weight loss drug working "miracles" is contingent upon x, y, z and/or has yet to be tested on humans. 

On another note, it is reassuring to know that individuals are making strides towards developing peer feedback initiatives such that widespread expertise can be offered (PubPeer) or in the case of PubMed Commons, establish an ongoing review system. But, there is much more that needs to be done to achieve the level of scientific credibility humanity deserves. As we discussed in lecture, statistics' primary objective is to identify and avoid bias, therefore it is imperative that as a scientific community, we understand and implement the appropriate tests/tools to prevent unacknowledged flaws from persisting throughout our comprehension. Further, we must better integrate sound statistics knowledge with our perception of the research process - specifically as elucidated by Bayesian analysis, what a 5% false positive rate of scientific hypotheses means for our ability to reproduce given results. I hope that all contributors to this dynamic field can eventually become more well-versed in statistics to better uphold ethical standards and ultimately, allow the public to regain its trust in the currently misleading realm of scientific research.