I’d like to raise a question that’s occurred to me several
times during this class. Many of the problem sets we’ve been given have
required the use of a one-tailed t-test, and TJ has been pretty dismissive of
two-tailed t-tests, saying that they’re used by people who aren’t willing or
haven’t collected enough preliminary data to make a real hypothesis. Before
this class, however, I was taught that two-tailed tests were preferable and
that one-tailed tests were usually used inappropriately for, as TJ would call
it, “p-hacking.”
I can see merit in both arguments. Typically, we have a
pretty good idea of what we expect to happen in our experiments, and we’ve
often done enough preliminary tests to formulate a hypothesis that does predict
a change in only one direction. In that case, why not get as much rigor as we
can from our statistical test and devote all the alpha allowance to testing our
exact prediction? However, in my years of research, I’ve rarely been in a
situation where I only cared about a change in one direction. If you’re testing
a drug, you certainly want to know if it improves your disease condition, but
everyone who might ever be affected by that drug hopes that you’re also making
sure it doesn’t increase the severity of the disease. In my dissertation
research, I’ve often predicted that knocking out an anti-inflammatory regulator
in the gut will increase expression of pro-inflammatory mediators and decrease
levels of barrier-promoting proteins, but I’ve frequently seen that what really
happens is that the protein level of a pro-inflammatory cytokine increases, but
its mRNA levels decrease because of negative feedback, or that the expression
levels of a barrier protein increase because it’s being degraded more rapidly.
If I tried to do statistical tests that only looked for change in one
direction, I’d miss these realities even though my fundamental hypothesis about
increased inflammation was correct.
If you decide to do a one-tailed test and see a change in
the opposite direction, not only do you get no statistical info about it, but
technically, if you want to evaluate whether that change is significant, you
have to go back and repeat the study again with a different statistical plan.
Now in reality, no one’s going to do that; they’re just going to go back and
run a different statistical test on the same data in hopes of getting a
significant result. At that point, you’re changing your stats for a p-value, no
way around it. It’s even worse if you then repeat the test with a one-tailed in
the opposite direction, because then you’re essentially performing a two-tailed
test with an alpha of .1. If you did decide to go by the book and repeat the
experiment planning for a two-tailed or a different one-tailed test, then at
least you’re wasting time, lab resources, and taxpayer/donor money, and
frequently also animal lives or human samples, which raises an ethical dilemma.
It seems to me that there are very few times when it would
be appropriate to do a one-tailed test (for instance, you’re transfecting with
an overexpression plasmid; you only care if there’s an increase in expression
of your target, because if not, you’re going to repeat the transfection), and
the rest of the time, you should just power your studies appropriately for a
two-tailed t-test rather than risking missing a potentially important
biological result.
