One-tailed or two-tailed?

I’d like to raise a question that’s occurred to me several times during this class. Many of the problem sets we’ve been given have required the use of a one-tailed t-test, and TJ has been pretty dismissive of two-tailed t-tests, saying that they’re used by people who aren’t willing or haven’t collected enough preliminary data to make a real hypothesis. Before this class, however, I was taught that two-tailed tests were preferable and that one-tailed tests were usually used inappropriately for, as TJ would call it, “p-hacking.”

I can see merit in both arguments. Typically, we have a pretty good idea of what we expect to happen in our experiments, and we’ve often done enough preliminary tests to formulate a hypothesis that does predict a change in only one direction. In that case, why not get as much rigor as we can from our statistical test and devote all the alpha allowance to testing our exact prediction? However, in my years of research, I’ve rarely been in a situation where I only cared about a change in one direction. If you’re testing a drug, you certainly want to know if it improves your disease condition, but everyone who might ever be affected by that drug hopes that you’re also making sure it doesn’t increase the severity of the disease. In my dissertation research, I’ve often predicted that knocking out an anti-inflammatory regulator in the gut will increase expression of pro-inflammatory mediators and decrease levels of barrier-promoting proteins, but I’ve frequently seen that what really happens is that the protein level of a pro-inflammatory cytokine increases, but its mRNA levels decrease because of negative feedback, or that the expression levels of a barrier protein increase because it’s being degraded more rapidly. If I tried to do statistical tests that only looked for change in one direction, I’d miss these realities even though my fundamental hypothesis about increased inflammation was correct.

If you decide to do a one-tailed test and see a change in the opposite direction, not only do you get no statistical info about it, but technically, if you want to evaluate whether that change is significant, you have to go back and repeat the study again with a different statistical plan. Now in reality, no one’s going to do that; they’re just going to go back and run a different statistical test on the same data in hopes of getting a significant result. At that point, you’re changing your stats for a p-value, no way around it. It’s even worse if you then repeat the test with a one-tailed in the opposite direction, because then you’re essentially performing a two-tailed test with an alpha of .1. If you did decide to go by the book and repeat the experiment planning for a two-tailed or a different one-tailed test, then at least you’re wasting time, lab resources, and taxpayer/donor money, and frequently also animal lives or human samples, which raises an ethical dilemma.

It seems to me that there are very few times when it would be appropriate to do a one-tailed test (for instance, you’re transfecting with an overexpression plasmid; you only care if there’s an increase in expression of your target, because if not, you’re going to repeat the transfection), and the rest of the time, you should just power your studies appropriately for a two-tailed t-test rather than risking missing a potentially important biological result.