Rewards for Data Encourage Bias

As a second year graduate student, this is the first time I have been asked to contemplate the issue of unreliable research with any depth. This is a problem. It seems important for people carrying out research to at least be aware that they have bias, and yet this isn’t a topic talked about frequently. Why wasn’t the topic of bias introduced in the first year of grad school? Why isn’t this topic covered as early as grade school? I don’t think it is possible to remove all bias from a system that relies on people to collect and analyze data. After all, bias is a natural part of human nature. Perhaps part of the solution is to start teaching students to acknowledge the problem at an early age.

                If I think back to the way that science labs were taught when I was in high school I believe that the course structure itself encourages dishonesty in reporting and analyzing data. With the majority of these labs there was a predetermined answer. For example, students were required to figure out the thickness of tin foil. The lab was then graded on how closely the students could come to the truth. Because of this, it was easy to fudge the answers to get closer to the number printed on the box of tinfoil. Instead of grading students on the accuracy of their final answer, perhaps they should be graded based off of their skills in collecting data or some other facet of lab work.

Receiving a reward for correct data in high school seems eerily similar to the publish-or-perish culture in academia.  Academics are often judged based off of the number of published papers they have. This not only comes from peers, but it comes from the people who sit on review boards for grants. I have heard several people talk about the pressure to get a paper out prior to their grant being submitted because it would look more favorable if the paper has already been accepted. This creates a push for speedy publishing rather than accurate publishing. Graduate students have the goal of collecting as much data as possible in the shortest period of time possible. This is simply so that they can receive their reward, which in this case is a degree. This is another instance in which data can be prematurely published or reported incorrectly because of the motivation of the scientist doing the experiments.

I’m not sure there is a way to change the culture of academia and eliminate the publish-or-perish ideals. It seems that journals themselves are taking measures to eliminate address the problems with bias and irreproducible research. PubPeer in itself seems like a good way to handle any published data that could be contended, however, this relies on another person in your field openly debating your findings in a public forum. I don’t think many people would find motivation to openly dispute published data without strong evidence. This is even more difficult considering some fields can be small and you may have a personal relationship with the people publishing data you can’t reproduce. Overall, I think the best way to prevent the problem with reward based bias is to teach people at a younger age to recognize it as a problem. This discussion should start at a much earlier age, certainly before the second year at grad school.