Well, for the most part they aren't.
A good scientist can distinguish between reliable and unreliable results. Of course that's because she's trained to do so. But it's also because as scientists, we know to be skeptical when exposed to new information. We want to know how it was obtained, why it's relevant in the field, and what the statistics say about the results.
That's not always the case though. Not every practicing scientist is a firm follower of the path trekked before by Plato, Eratosthenes, Galen, Descartes, Vesalius, or Brahe. Bias happens, and a few statistical results can be misinterpreted, and all that goes under the radar, and finds itself in a scientific journal.
Let's switch to talking about the journalist – one who has decided to write about science for whatever reason. Journalism is of course about integrity. A good journalist is trained to be skeptical. He will talk at great lengths to the authors of the paper he is writing about, and will usually talk to a scientist or two that was not the author of the paper, and get their opinions as well. If he's pretty cool he might even have a look at the stats.
But that's not always the case. A general assignment reporter at a local newspaper or an online news site may be under pressure from his editor to write something with appeal. They want it to be an accurate and reliable article, but they also want more papers to sell and/or more clicks on their article. He'll find something that looks interesting. If it's in a peer-reviewed scientific journal, who is he to question the integrity of the work. And the next thing you know, vaccines cause global warming.
What I'm trying to say here is not that scientists can do some pretty bad science, and bad stats. We've talked about that in great detail. But that we should be aware that, just like scientists, journalists aren't doing their job perfectly either. And what they do reaches a lot more people than what we're doing. In fact, it shapes the public opinion on scientific matters.
So it's always good to teach others about being skeptical, and not believing everything they read.
A good scientist can distinguish between reliable and unreliable results. Of course that's because she's trained to do so. But it's also because as scientists, we know to be skeptical when exposed to new information. We want to know how it was obtained, why it's relevant in the field, and what the statistics say about the results.
That's not always the case though. Not every practicing scientist is a firm follower of the path trekked before by Plato, Eratosthenes, Galen, Descartes, Vesalius, or Brahe. Bias happens, and a few statistical results can be misinterpreted, and all that goes under the radar, and finds itself in a scientific journal.
Let's switch to talking about the journalist – one who has decided to write about science for whatever reason. Journalism is of course about integrity. A good journalist is trained to be skeptical. He will talk at great lengths to the authors of the paper he is writing about, and will usually talk to a scientist or two that was not the author of the paper, and get their opinions as well. If he's pretty cool he might even have a look at the stats.
But that's not always the case. A general assignment reporter at a local newspaper or an online news site may be under pressure from his editor to write something with appeal. They want it to be an accurate and reliable article, but they also want more papers to sell and/or more clicks on their article. He'll find something that looks interesting. If it's in a peer-reviewed scientific journal, who is he to question the integrity of the work. And the next thing you know, vaccines cause global warming.
What I'm trying to say here is not that scientists can do some pretty bad science, and bad stats. We've talked about that in great detail. But that we should be aware that, just like scientists, journalists aren't doing their job perfectly either. And what they do reaches a lot more people than what we're doing. In fact, it shapes the public opinion on scientific matters.
So it's always good to teach others about being skeptical, and not believing everything they read.
In an ideal world a journalist would act as a watchdog and police and publicize people acting in bad faith, or people who just plain get things wrong. In this world however they are not incentivized to do so, the market wants sensation, and in science that is either new discoveries or thorough disgrace. The first is easy to write about and scientists facilitate it, the second takes a huge amount of work and the intuition to spot wrongdoing in the first place. I am not hopeful about the role of journalists in helping to improve the use of statistics in science, quite the opposite in fact.
ReplyDeleteThis actually reminded me of a story in This American Life that aired in NPR not too long ago. In April of 2015 they had a piece called The Incredible Rarity of Changing Your Mind, where Ira Glass talked about canvassers that had developed a way to go door to door and, in a 22-minute conversation, change people’s minds on issues like same sex marriage and abortion rights. They had run the piece because the research had been published in Science and had been gathering a lot of attention. However, it turned out that a graduate student had doctored the data from the canvassers and the paper had to be retracted, along with the NPR piece. Which kinda brings me to my point, if the peer review process was not able to spot the inaccuracies, I think journalists would be even less likely to spot them.
ReplyDelete