Seeking Integrity on the Shoulders of Giants

One of the books that has most shaped my view of the world is Mother Night by Kurt  Vonnegut. In it, the narrator describes his philosophy of how people’s minds are like clockwork, with every fact we know serving as a tooth in a cog. In many ways, though, we are missing teeth. Either we are unintentionally blinded to truth because of the environment in which we were raised, or we purposely deny truths because they do not fit nicely with our mental constructs. He says, “I… will say that I have never tampered with a single tooth in my thought machine, such as it is. There are teeth missing, God knows -- some I was born without, teeth that will never grow. And other teeth have been stripped by the clutchless shifts of history -- But never have I willfully destroyed a tooth on a gear of my thinking machine. Never have I said to myself, 'This fact I can do without.’” As a scientist, my greatest quest is to find truth and to try to fill the gaps in our understanding of the world. Yet, one of my greatest fears is that, knowingly or unknowingly, I am missing teeth in my gears.

Our first lecture with TJ Murphy struck home with me not only because it brought to light the “reproducibility crisis,” but it also reminded me of my part in it. I see that we today are standing upon the shoulders of giants. Everything that we know and believe to be truth is based upon the years of work of those before us. Though I am astounded by the brilliance, creativity, and passion of our predecessors, a part of me is also frightened. How much of what we take for granted is actually truth? In what ways do I contribute to the reproducibility crisis, and where does my own blindness come into light? How can I help promote an environment of scientific integrity? Christopher Pannucci and Edwin Wilkins cite the many different forms of biases that can distort an investigator’s ability to assess their findings, so I must confront and address my biases head-on before starting a project. As stated in The Economist, science is not necessarily self-correcting, so my fellow researchers and I must begin the movement to be more open about science as a powerful, yet fallible, tool for approaching our world’s many questions.

Intrinsically Intertwined

After reading the posted articles on irreproducibility and bias in science, I am surprised that there are not further measures in place to combat these issues. The use of anonymous post-publication peer review and Bayesian statistics to justify redoing an experiment seem like common sense measures. Why have these not become standard practice among the scientific community?

As the article “Trouble in the Lab” states, “more than half of positive results could be wrong.” This was revealed by John Ionnidis’ 2005 paper, which proved the cost of a seemingly small number of false positives. When I connect this thought to my own research, I am horrified. What if the claims that helped me to develop my experimental theory are unreliable? Though they were published in peer-reviewed journals, perhaps their results do not reflect actuality. These “discoveries” might have not been discoveries at all, but simply instances in which the data told an incorrect story. Because research builds on previously published results, a false published result could lead to a chain reaction on incorrect assumptions. How can this chain of events be halted?

Statistics proved that irreproducibility is a rampant issue among many life science research investigations. I believe that statistics can similarly be used to combat this problem. Even though “most scientists are not statisticians,” acceptance of the explanation laid out by Ionnidis should be a prerequisite for performing research. It should guide scientists to perform more experiments and to not be fooled by false positives. Perhaps, then, greater care will be made to distinguish discoveries made by statistical anomaly from discoveries that represent the laws of science. Because of the nature of their work, scientists should take it upon themselves to become as versed as possible in statistics. The two are so intrinsically intertwined—this fact can no longer be ignored in the scientific community.

Research Misconduct and the Importance of Scientific Accountability

Working in science is privileged in that not everyone has the opportunity to do it; thus being a scientist comes with a great deal of responsibility.    Biomedical research in particular holds the promise of understanding a disease better, providing hope to the sick and healthy alike that we may one day better combat (just to name a few) cancer, heart disease, or neurodegenerative disorders within our lifetime.

Excellent science involves investigating and answering the important questions and the ability to follow through on this by forming solid conclusions that propel the field forward. I would argue that an emerging quality of a strong researcher is the ability to be resilient and open-minded, empirically following the data produced without letting opinions (or the allure of a high impact publication!) influence the direction of the work. In a recent TED talk on human moral code, Dr. Dan Ariely put it well when he said that scientists have a responsibility to “do more systematic experimentation of our intuitions.”  

Unfortunately, the present scientific environment makes this process quite difficult. Funding opportunities are competitive and scarce, leaving little time to follow up on negative data and perform trials in all preferred configurations. A recent article from the Economist points out the unfortunate reality of our situation; “irreproducibility [in science] is much more widespread,” and data backs this up.  It has been suggested that approximately three quarters of all biomedical studies are not reproducible.  So, if we can’t reproduce it, can we even trust in science? 

Furthermore, instances of research misconduct have been given a high profile in the media, giving the reliability of research a poor reputation. The retraction of the discredited Wakefield study, which infamously produced a great deal confusion in the public sphere over a putative link between vaccines and autism had severe consequences.  Many parents still choose not to vaccinate their children, leading to events such as the recent measles outbreak in Disneyland.  The profound impact of science on society makes it imperative that each increment of data added to the field of biology be examined with a high level of scrutiny.  

A recent interview with neuroscientist Dr. Brandon Stell, echoed these observations and opined that modern science is geared “toward mostly illusory ‘breakthroughs' and ‘high-impact research’ at the expense of careful work.”  To help reduce research misconduct, he and his colleagues created a tool called PubPeer. Pubpeer is an anonymous forum for other scientists to comment on the quality of the data published in the literature. It shows great promise as a system where scientists can begin to hold one another accountable in a public realm.  It is certainly a step in the direction towards filtering the sea of literature for the data that is high fidelity.