Thoughts on Bias in Science

As a science teacher for five years prior to entering graduate school, it became apparent pretty quickly that my students had no concept of “Science” beyond its place on the cover of a colorful textbook, a word that might be followed by a homework assignment in their little notepads, or possibly a vague notion of half-baked “experiments” and often pointless projects.  Hoping to engender a more informed idea of Science, I introduced a new session at the start of the school year that covered many aspects of the process of science that are often poorly understood by the general public—hypotheses versus theories, the importance of experimental controls, peer review, and, of course, bias and reproducibility.  Students immediately appreciated the way bias might enter into a scientist’s experimental results, as kids themselves operate on a system of rewards (or avoidance of punishment) based on their ability to show those in charge of them what they want to see.

At that time, I had not yet decided upon a career in research, but it seemed so simple to me that a real scientist in control of a real experiment should just report what they see and keep their scruples about them like a holy scientific bulwark.  In other words, I was sure a person of uncompromising integrity would not have to worry about bias and could rest assured that their work, when published would be reproducible.  Oh, bless my past self’s little heart.

Okay, I still do believe that strong ethics in science make a difference, but what truly frightens me are systematic biases buried in the experimental design.  Protection against this insidious form of bias seems incredibly difficult, and to avoid it personally will require more diligence on my part when conducting my experiments. Beyond unintentional bias, I would also like to touch on one of of the thoughts that came to me while reading our class’ various articles. 

Julia Belluz of Vox writes of the media’s irresponsible overuse of laudatory prose in describing new medical discoveries, and how it’s not always the media—sometimes it’s the doctors themselves.  This idea really struck a nerve on a few different, somewhat conflicting levels, prompting even a bit of self-examination.  My first thought is that YES, I completely agree that the over-hyping of new drugs has become egregious.  The media, and apparently researchers as well, constantly sell new scientific advancement like snake oil, which ultimately erodes the public trust in the scientific community.  Public support, especially in the US, feels under assault with well-intentioned but hurtful crusades against established and researched fields like GMOs and vaccines.

That being said, even scientists end up overselling their findings to other scientists on grant committees.  In our grant writing class here at Emory, we’re not taught to be dishonest or forge data, but we are taught to sell our ideas with the implication that you had better grab those reviewers’ attention quick or there goes your chance for funding. In no way do I mean to disparage what I found to be an excellent and helpful course.  I wish to make the point that from the beginning of our careers as real scientists, we are taught that the competition is fierce, and we need to extrapolate our work’s impact to connect with some sexy scientific or medical problem.  In the lab, I’m very cautious to believe anything I’m seeing, but you’d never know it from my confidence-exuding grant!  Is this sort of mindset enough to infect our work with “I need to get a grant/to publish/to graduate/to get a good job/etc.” bias?

I may have gone a teeny bit off topic here, and I honestly have no answer for my own question.  However, I do believe that a combination of education in bias and a strong sense of morality are an essential start to a career in science.  There are innumerable other factors that still come into play, especially the need for the general public to be well-informed consumers of science.  Ultimately, the problem of bias in science is much more complex than I originally thought, and I look forward to learning more about improving experimental design and use of statistical analysis in our class.

We Aren't Paid to Replicate

When applying for government funding for scientific research in the United States, most agencies require a section describing the novelty of the project. If it isn't very innovative or ground breaking, chances of being funded are very slim; our government prioritizes the promise of new discoveries and breakthroughs over repetition. Scientific journals also focus on novel research and rarely publish papers that describe the replication of an older study. In general, the scientific community does not care to read about negative findings. This is reinforced by popular culture, which likes to emphasize the exciting and dramatic bioscience findings rather than remain objective and guarded about new discoveries until they are verified by other research groups. 

These expectations in academic institutions also contribute to the emphasis on novel findings. In order to be awarded tenure, a junior faculty member must receive multiple large grants and be senior author on publications in respected journals. These pressures and the competition for funding mean that most scientists don't have the time or the money to further verify their results by independent verification. Scientists have little incentive, and do not face much scrutiny even if later studies call into question the original results. However, this is becoming a crisis as more and more research indicates that only a small fraction of scientific papers can be easily replicated by other laboratories.

Recently, organizations have formed that are trying to address these issues. The most notable is a program run by the Science Exchange Network called the Reproducibility Initiative.  Scientists can pay to have their studies replicated by an independent and skilled group of researchers, who are then able to publish their results (positive or negative) in a PLOS journal. While this is expensive, it may ultimately save money by shifting research away from false leads.