Scientists,
especially graduate students, have become too focused and driven on results
being statistically significant. We play statistical significance up to be all-important
in science; most of our experiments and projects focus on finding some difference.
If we don’t get the results that are “statistically significant”, we feel like
failures and that something went wrong. Maybe I am generalizing too much of my
own experience in graduate school, but bear with me. Graduate school is
notoriously viewed (well, at least by me) as “soul-sucking.” I believe that
much of these feelings of hopelessness and failure originate from the moment
you press “Analyze” on Prism and see “ns.” Imagine how different graduate
school would be if that feeling of failure were eliminated…how would things be
if we took every negative result and no longer viewed it as a dead end or a
reflection of our abilities as scientists? What if when we saw “ns” we could feel
joy and not distress? I feel like our success in graduate school is defined by
statistical significance; without a p<0.05, our hard work means nothing.
When was the last time that any of us went to a thesis defense that focused on
non-significant results? Why has it become that a statistically significant
result is necessary to earn our doctorate? Would our education be at a
disadvantage if were not required to present statistically significant data?
In a
way, statistical significance helps to remove bias by allowing for
quantification and comparison of results in order to look for a difference. Statistics
and calculating a P value are what allow western blots to be informative and
unbiased. Without P value, there would likely be variation in what some would
say “looks” like a difference between two groups. Science needs statistical
significance. However, statistical significance has also created bias in the
way that we approach problems. The need for statistical significance prevents
us from exploring concepts and hypotheses that may turn up to be of no
significance. The need for statistical significance may also lead a researcher
(without proper statistical training) to increase the n of their experiment to
the point where a p value of <0.05 is inevitable. It has become unacceptable
to just say no significance; we force our P value to mean something, even if
it’s just “trending” towards significance. Statistical significance and p
values both eliminate bias as well as create it.
I
feel that people don’t actually think about what “statistically significant”
means; all a P value can tell us is the probability that we could see that a result
of the same magnitude if the null hypothesis were true. It cannot actually tell
us how likely the alternative hypothesis is true. Thus, we need to stop
defining the importance of our work by the P value. Motulsky brings up that
colloquialisms may contribute this problem of focusing on statically
significance a P values. We associate the term significance with importance,
which is incorrect when interpreting statistics. In order to interpret
statistics, one must understand the theory and definitions of the terms used.
Then, and only then, can we understand that statistics does not interpret the
importance of our experimental results; it only allows us to accept or reject
the null hypothesis. We can no longer
define our work and goals by “statistical significance”; instead we should be
seeking scientific importance.
