A Student's Reaction to the Reproducibility Crisis

A common fear among academic scientists today is that many of the latest and greatest findings, which cost years of work and millions of mostly tax-payer dollars, are not actually true. Even as a new graduate student, I have already come across the challenge of reproducing other's work in my own research. The author of a recent Scientific American article claims that the lack of reproducibility in science is why science actually works, why progress can be made, and how science evolves. The author takes the position that if scientists worked until their research was valid (measured by the gold standard of reproducibility), then nothing would ever be published. His conclusion is that scientists should be honest with themselves, their peers, and the public about the impossibility of perfectly reproducible science, and the likelihood that untrue findings will be published with great frequency. I feel that the author of this article confuses validity of data with data that doesn't support the original hypothesis. It's true that a scientist must never allow himself to become so committed to his model that he forces his data to match it (either through outright fraud or innocent tinkering), however, a scientist should always seek to obtain and publish data, that to the best of his ability represents the truth, whether that matches the original hypothesis or not. Perhaps I am still early enough in my scientific career that my naive hopefullness has not yet been replaced by this author's cynicism, but I feel that this author's position sets the bar for scientists too low. As scientists, we must do the best we can with our current knowledge and technological capabilities, acknowledge the limitations of what we are capable of, and accept that the findings we discover may later be disproven by other scientists who are just trying to do the same thing. 

Science: With a grain of salt

At almost every decision-making point, humans are innately biased beings. Our unique capacity for pattern recognition has allowed us to make great scientific strides, but undoubtedly has resulted in bias and false positives when interpreting scientific data; and the “publish or perish” culture in academia promotes novel findings and positive results over reliability of findings. These are things that most scientists are aware of, but very few take adequate measures to minimize the effects of bias; including myself. Many of the articles reveal the daunting commonality of the false positive, but Horvath presents it as a reality of science; a statement which resonated with me. It is our responsibility as scientists to conduct investigation and report findings as ethically as possible. But with that said, we are humans and therefore fallible. It would be impossible to expect that every reported finding would be true. To obtain such a goal, there would need to be a shift in the current expectations of the publication “race”. A slower pace of review and publication would promote greater validation of studies and an in-depth peer review process. But, a slower pace of publication could prevent a rapid rate of discovery. I am in the “self-correcting “science camp. I think that false theories can be filtered out if validation studies fail to replicate the results. I know if I’ve tried a technique and failed to replicate the result after a few attempts, I move on to a different methodology. The emerging trend toward post-publication review will be a great tool for this weed-out process and will ultimately lead to better science. I believe that we as scientists know to take scientific findings with a grain of salt. However, the size of that grain can vary based on whether it corroborates or contradicts our own theories, something that should give all of us pause for reflection.