Irreproducibility from Dishonesty or Technical Incompetence?

While Dan Ariely is an exceptionally good speaker, I didn’t find the entire Dan Ariely video overly relevant to our discussions on bias and irreproducibility in science. The majority of the video dealt with overt and pre-meditated dishonesty in an attempt to intentionally deceive others. In reality, the instances of pre-meditated, outright fraud and data manipulation are very low in science. Rather, I think that most instances resulting in irreproducible science are a result of both poor technical execution and improper experimental “optimization” (i.e. “optimizing” the experiment to give you the results that you expect to receive). I have seen many examples of technicians and young scientists disregarding experimental results as technical failures simply because the results were not easy to interpret at supporting their expected outcome.

Secondly, the individuals doing the conscious and deceptive lying were motivated by their own short term personal gain. The experiments he and his team conducted were very specific and the outcomes represented very immediate and short term gain, and I believe these experiments represent a biased design if one is trying to apply the conclusions to irreproducible science. The expected reward of scientific fraud is not immediately realized, nor does the risk of exposure vanish immediately.

However, I did find the section regarding bankers and mortgage backed securities (MBS) particularly relevant to some of the cognitive biases that are introduced at all stages of one’s career in science. The example given regarding being rewarded for supporting MBS leading to individuals developing a bona fide belief that MBS are inherently good, specifically coupled with other cognitive biases regarding a belief in markets as self correcting, etc. The belief in a hypothesis handed down from a PI, mentor, post doc for whom a young scientist has a lot of respect can result in the development of similar biases. And subsequently, introduce many biases into experiments resulting from both a belief in the hypothesis and a belief in science in general.