Multiple Comparisons Testing: Tukey Method

Greetings!
So, if you’re like me, you’ve probably played around on prism for a little too long, and you’ve managed to find your way to some confusing drop-down menus.

In class, we’ve covered some of these in varying detail, and many have been elucidated, but there are still some that remain mysterious to me.

With that inspiration, I wanted to go ahead and write a post or two about what exactly some of these different tests we can run are, and when to use them.
If knowledge is power, then hopefully describing these tests, what they do exactly, and when to use them will empower us to make smarter statistical decisions regarding our data and presentation of it.

Tukey’s Test

Available in both one-way and two-way ANOVA tests, the Tukey test comes recommended under the “Options” tab when you’re defining the guidelines of comparisons for you ANOVA. This is an example of post-hoc analysis that is performed on your collected data. Tukey test will only become available to you when you when you select to guideline options that compare every possible pairwise difference of all means, at the same time.

Tukey’s Test relies on three key assumptions:

1.     All observations are independent of each other

2.     All means in this test associated with a particular group is normally distribution

3.     There is homogeneity of variance (or the variances within groups are equal for each mean associated with that group).

Essentially, the Tukey method is a t-test (or relies on a studentized range (q) distribution) that corrects for the extra type-1 error probability that increases with multiple comparisons.
The primary use of Tukey’s test would be when you want confidence intervals or when your sample sizes between groups are unequal. If these two are not a factor in you decision for statistical tests, then consider another test because Tukey’s tend to be less powerful than some of the alternative options. If your experimental design is interested in all potential comparisons available from the data and you’re interested in confidence intervals, it is recommended to use Scheffe’s method rather than Tukey’s. This is because Scheffe’s method provides narrower confidence limits than Tukey’s and would thus be more beneficial in this particular instance.

There is a variation of the Tukey procedure, which is often called the Tukey-Kramer method and is used when you have unequal sample sizes. In this instance, standard deviations from for each paired comparison. Unfortunately, this is more interesting factoid for us, since Prism doesn’t offer this test (at least to my knowledge).