Restore Public Faith in Science - Fix Bad Statistics

Stepping out of the ivory tower of academia, we come face-to-face with a public that has a growing distrust of science. When bad science makes its way into the media, it feeds the distrust by giving leverage to their arguments – if scientists say it, it must be true, right? For example, we can point to study after study where it has been shown that vaccines do not cause autism, but one heavily flawed paper from the 1980s has now convinced thousands of new parents that vaccinating your children is unnecessary and even dangerous, despite the eventual retraction of that paper (thanks, Andrew Wakefield). Biomedical science is probably the most publicly discussed field, because the bench results will theoretically make their way to humans as treatments and cures.

The “publish or perish” mentality has driven scientists to value results over process. And who could blame us, when our ability to do research is funded based on our ability to produce novel results, and our skill in gaining this funding is what keeps us employed? With the pressure on to achieve the desired results in the shortest possible time, we arbitrarily decide a sample size of three is enough to detect an effect, if one exists. Results in hand, we open the door for our intrinsic biases to sneak in and permeate themselves throughout our data analysis, hoping to achieve a value of p < 0.05. Further, we encourage this behavior throughout the tiers of the lab – it starts with the PI, who is thankful this data will fit nicely into the grant renewal and trickles down to the relieved graduate students, who can write the data up into a manuscript to check a box off their graduation requirements.

This results in inadequate training in statistical design and analysis of experiments, generates science that produces results that cannot be replicated, and perpetuates a cycle of scientists untrained in recognizing an inherently flawed study. If the people who are considered well-informed are unable to identify these issues, it is almost certain that a layperson would not be able to distinguish between statistically sound and cutting-corners science.

We owe it to the public, and to ourselves, to do better.

How to change a field that seems doomed with bias

With the many recent articles siting bias as an extreme impediment to the purpose of science, it is difficult to not stand on top of the cafeteria table and call B.S. to the entire field. Although this seems drastic, faith in science and the scientists that perform it become more and more futile as knowledge of how bias has infiltrated research becomes known. An article published in The Economist lists several ways that biased and unrepeatable research can become published. How can we change an entire field that consists of researchers partial to find their hypotheses correct, reviewers that don’t have enough time to complete an accurate assessment of the science, or journals that rarely publish needed and essential repetition of previous results?  

The psychology of science is complex, and it seems that bias is unavoidable, especially in the “publish or perish” mentality that exists in the field of academia. Jared Horvath, in an article in Scientific American, states that bias and the lack of reproducible research is not just a recent phenomenon, but can be seen for many centuries previous and among even the most lauded scientists, including Galileo, Dalton, and Einstein. So again, how are we to avoid something that has been innate in our field for centuries?

We as scientists must acknowledge that science not properly designed, analyzed, reviewed and repeated exists and is pervasive in our field, even among our own institutions, departments, and laboratories. We also must acknowledge that this kind of science is not trustworthy and can mislead not only the scientific field but the general public to hope for cures that might not exist—wasting hope, time and funding on biased hypotheses and results. Although changing a field seems impossible, I believe it starts in one laboratory that is willing to fight for good scientific practices (click here for a list of biases common in design and analysis of research written by Drs. Pannucci and Wilkins).

If we truly want to change the field to one that is trustworthy and contains good, unbiased scientific practices, we must become our own “bias police” in which we are adamant about self-critiquing and repeating experiments from our laboratories. As we begin to transform our own laboratories, we can begin to hope for the transformation of our entire field.