Seeking Integrity on the Shoulders of Giants

One of the books that has most shaped my view of the world is Mother Night by Kurt  Vonnegut. In it, the narrator describes his philosophy of how people’s minds are like clockwork, with every fact we know serving as a tooth in a cog. In many ways, though, we are missing teeth. Either we are unintentionally blinded to truth because of the environment in which we were raised, or we purposely deny truths because they do not fit nicely with our mental constructs. He says, “I… will say that I have never tampered with a single tooth in my thought machine, such as it is. There are teeth missing, God knows -- some I was born without, teeth that will never grow. And other teeth have been stripped by the clutchless shifts of history -- But never have I willfully destroyed a tooth on a gear of my thinking machine. Never have I said to myself, 'This fact I can do without.’” As a scientist, my greatest quest is to find truth and to try to fill the gaps in our understanding of the world. Yet, one of my greatest fears is that, knowingly or unknowingly, I am missing teeth in my gears.

Our first lecture with TJ Murphy struck home with me not only because it brought to light the “reproducibility crisis,” but it also reminded me of my part in it. I see that we today are standing upon the shoulders of giants. Everything that we know and believe to be truth is based upon the years of work of those before us. Though I am astounded by the brilliance, creativity, and passion of our predecessors, a part of me is also frightened. How much of what we take for granted is actually truth? In what ways do I contribute to the reproducibility crisis, and where does my own blindness come into light? How can I help promote an environment of scientific integrity? Christopher Pannucci and Edwin Wilkins cite the many different forms of biases that can distort an investigator’s ability to assess their findings, so I must confront and address my biases head-on before starting a project. As stated in The Economist, science is not necessarily self-correcting, so my fellow researchers and I must begin the movement to be more open about science as a powerful, yet fallible, tool for approaching our world’s many questions.

Publication bias in my own research

Unbiased research is the “gold standard” of science and aims to prevent scientists from achieving “self-fulfilling prophecies” and deceiving themselves into believing an idea or hypothesis is true when it is actually not true or completely due to random chance. Properly applied statistics, experimental design, replication, among other methods are ways to prevent bias from happening, but preventing bias is actually against our normal human nature as discussed in Dan Ariely’s TED Talk.  Additionally, the many forms that bias takes whether it be known or unknown to the person performing the experiments can confound studies and/or hinder scientific progress. Panucci et al. (2010) reviews numerous types and examples of bias in tangible research situations that most biomedical and/or clinicians will find themselves involved in during their careers, whether it be conducting research with such biases or reviewing it. Even though there are many types of research bias, I think that one of the major forms that I currently face and/or am frustrated with as a fourth year graduate student is publication bias, which in my view is the preference of publishing positive results and/or the inability to publish data that goes against the current paradigm of a particular field.

Recently, I designed an experiment to analyze the cytokine response in nonhuman primates with malaria using two different species of malaria parasites. I went through the standard process of generating my samples, developing and designing the protocol, etc. Then, I proceeded to perform the experiment. In one set of samples, I found that Interferon gamma, a hallmark cytokine in malaria involved in immune responses, was upregulated as we had predicted. In my other set of samples, the cytokine was undetectable. I immediately thought, “How could this be? The assay must be wrong because this has been seen before and is found in all malaria infections from mice to humans to monkeys.” Like a good student, I proceeded to validate the data on a different platform and confirmed my results, which took about three more weeks considering the need to standardize yet another assay, place orders, etc. Whenever I presented the results to my advisor, he or she responded, “I am not surprised; I’ve seen this before.” I was shocked because in the literature, this cytokine is always reported, and my advisor had even advised me to use this cytokine as a “calibrator”. However, it turns out that this cytokine has a temporal dynamic during the infection so if one misses a certain time frame, the cytokine may not be detectable. To my knowledge, not a single paper addresses this observation, and this is largely due to the need to have a positive result to get the data published. Obviously, it is much more interesting to talk about an increase in a particular cytokine than no change at all. Furthermore, the dogma and published literature are skewed to presenting this cytokine as central in malaria so if it is not present in detectable levels, researchers and reviewers begin to question the integrity and quality of the experiment and samples.  

            Publication bias thus caused me to lose time in finishing this component of my project. If the literature was not biased based to current opinions and also by the need for a positive result, I and others likely in my situation would not be surprised by such results and spend time triple checking all aspects of an assay, repeating results too many times, etc. Publishing negative results and results that may go against current understanding or dogma should be welcomed in science and are ultimately what will help current scientists address new and interesting questions in novel way. Of course, this opinion is qualified by the fact that the experiments are designed appropriately and replicated.