m,

Hidden Biases: Looking for Bias in Unexpected Places

           Assuming that we, and all of our colleagues, are striving to conduct unbiased science, I often forget to look for bias in obvious places. Sure, there’s the scientist trying to make his or her research appear more important, the prototypical bias we have come to expect, but what about the biases we can’t control?

            This article fromNPR points out that bias can be virtually impossible to avoid. To make a long story short, a researcher found that the results of his experiment had been skewed by natural hormone differences between men and women. Depending on the gender of the person administering the experiment, the results were different.

            Of course this doesn’t mean that one gender gets inherently better results than the other, but it does illustrate an important idea. Not only do we have to be proactive in controlling our own personal biases, but also we have to look for external factors that may affect or otherwise disrupt results.

            I wouldn’t have expected gender of the experimenter to affect results of an experiment. This could be the tip of the iceberg. Does the color of the experimenter’s eyes matter? What about the perfume the experimenter was wearing? Of course some external factors are inevitable, and the point of the article wasn’t to condemn scientists of either gender. The article was a simple plea that scientists report these kinds of external factors.

            To me, this requires two fold effort from scientists: First, we need guidelines on the myriad external factors that can affect our science. Second, we need to be vigilant, reporting every factor that could possibly affect results, no matter how seemingly inconsequential. On the surface, it sounds like a pain. I understand that. It isn’t easy to keep copious and detailed notes. Reporting like this will, almost certainly, be tedious. Still, if there is any chance that a factor could affect reproducibility, it should be recorded.

            More than anything, I think the real call to action is to think about hidden ways that bias could exist in our work. I think our experiments and our credibility will be all that much better for it.

The peer review process is in need of in-depth review

At the core of scientific publishing is peer review. During this process, scientific literature is scrutinized, and errors or horrors are addressed. As mentioned in a Vox.com article: Why you can't always believe what you read in scientific journals, this process serves as a cleaning machine that gets rid of all “junk” data and polishes scientific literature. However, this process is far from perfect, as exemplified by the number of retractions of articles published.

Steen RG, Casadevall A, Fang FC (2013). Why Has the Number of Scientific Retractions Increased?. PLOS ONE 8(7)

To address this issue, a group of scientist founded PubPeer, described as an “online journal club” that allows the scientific community to openly and anonymously review literature. The founders of this website argue that an important reason for the mistakes overlooked is the centralized nature of the peer review process. In the traditional peer review process, a selected few experts are allowed to review a manuscript before publishing. This process, rely on the capacity of the few reviewers to identify and notify mistakes on the manuscripts. This website opens the peer review responsibility to thousands of experts.

Even when this alternative seems far from perfect, it is definitely a step in the right direction. Opening the review process to the scientific community, and making this process more decentralized and transparent, will continue foster the ultimate goal of improving the quality of research articles published. In the end, all literature should be objectively reviewed. As Carl Sagan said: “the hard but just rule is that if the ideas don’t work, you must throw them away.”

Who is to blame for irreproducibility?

“Blame It On The Antibodies” <— click here to link to article

This article highlights the controversy behind the irreproducibility of antibody research and how the use of one antibody over another can affect the outcome of an experiment. I really don’t understand how this is something that still exists in this day and age where science has progressed rapidly, becoming more accurate, faster, and precise. Yet, we have not been able to resolve this gray area of consistency and accuracy of antibodies.

One thing I battled with when thinking about this topic is who carries the blame when the antibody research is irreproducible. Should the blame be placed on the scientist who assumed that the antibody would work on its intended target or should the blame be placed on the company who sold a product that simply did not work?

Companies should be held accountable when it comes to the distribution of products intended for scientific research point blank, especially in the scope of research where peoples’ livelihood is at stake and the price is high.  At the same time, the reality of the situation is that knockoffs are everywhere and we should be protecting ourselves from them.

Thus, it should be the duty of the scientist to ensure that experiments are being conducted accurately and test the question at hand. How can you assure the validity and accuracy of an experiment if you don't first test the validity and accuracy of the antibody? At the same time, that scientist paid for a convenient and accurate product and that is what they should receive-a product that works upon arrival.

I guess my biggest turmoil with the controversy behind irreproducibility in science is how do we deliver blame? How do we identify where or at what point the scientific integrity of the experiment gave way? Before we can even address this problem to fix it, we would have to choose where the blame lies: the company, the scientist, or even the more complicated both.