Oh what a mess!

Oh what a mess!

I’ve struggled writing this blog post: (1) I’ve never written a blog post before but always imagined my first blog post to be edgy, witty, or ending with a picture of me raising a champagne flute into the air with the sun setting in the background as I dig my toes into the luxurious sand of some tropical island that’s the perfect mix of exotic, difficult to pronounce, yet still elicits an envious response from all of my followers, (2) bias and irreproducibility are both distinct and related, difficult to separate and examine independently, (3) concerning the “reproducibility crisis” well even that’s being debated.  

In a Nature News Feature article, 1,576 researchers were asked a simple question: “is there a reproducibility crisis?” 52% of those surveyed said “Yes, a significant crisis,” while 38% said “Yes, a slight crisis,” 3% answered that there was no crisis and 7% said they didn’t know.

When broken down into factors that contribute to irreproducible research, the Nature news article concludes that top related factors relate to intense competition and time pressure… umm let me check the honor code really quickly… yeah, as I expected there’s no clause excusing cheating because of impending deadlines or pressure to get a better grade. So why do fool ourselves into thinking that dishonesty motivated by intense competition and time pressure might be acceptable? Naming contributing factors like selective reporting, pressure to publish, low statistical power or poor analysis, and not replicating enough in original lab.

Tying this into Dan Ariely’s video, “The Honest Truth About Dishonesty”, perhaps the real crisis lies in the lies and biases we hide behind, things like “everybody is doing it” or there’s no identifiable victim. Dr. Ariely suggests that exercises like signing an honor code primes people with their own morality and decreases incidences of cheating, maybe priming people with their own competence, asking them to verify they’ve received appropriate mentoring, developed the proper methods or code, and designed the best experiment could work to decrease incidences that lead to irreproducibility.