The last installment of my brief guide to multiple
comparisons testing are the Bonferroni and Sidak correction methods. The
Bonferroni method, or correction as it is sometimes referred to, is probably
the simplest and easiest of the multiple comparisons tests to comprehend, (at
least for me). In short, the correction adjusts the individual thresholds for
p-values when multiple independent or dependent statistical tests are performed
all at the same time. Based around the central problem with multiple
statistical tests, which is that you burn your type-one error with every
comparison you make, the Bonferroni method corrects for this problem in the
most direct route possible. If you have X number of comparisons to be made, you
simply take your threshold for type one error, (α), and divide it by X
comparisons (α/X), which provides a much smaller threshold for the type one error
for each individual comparison.

To be put even more
simply, when you use the Bonferroni method, the chance of knowing a result
reported as significant is actually significant increases with the more
comparisons you make.

The Sidak (the first
S is pronounced as a SH) method or correction for multiple comparisons was
published in 1967 by the statistician and probabilist Zbyněk Šidák. Where the
Bonferroni correction focuses on the threshold of type one error for each
comparison made, the Sidak correction focuses on the opposite question. If the
Bonferroni correction asks the question of “ What is the probability that my
statistically significant result is actually statistically significant?” then
the Sidak correction asks the question, “What is the probability that the
differences are not statistically significant?”

The example the Graph Padwebsite provides suggests to think about it in this way: If we want to make three comparisons, and each one is under the same threshold for type one errors, which is 5%, (α = 0.95, then 1-α = 0.05) and that means that the chance of all three comparisons being not statistically significant is equal to (0.95*0.95*0.95 = 0.8574). And thus the chance that one or more of the comparisons will actually be statistically significant is 1-0.8574 = 0.1426. The only catch with Sidak’s method for correction is that you have to assume that all of your individual tests are independent of each other.

The example the Graph Padwebsite provides suggests to think about it in this way: If we want to make three comparisons, and each one is under the same threshold for type one errors, which is 5%, (α = 0.95, then 1-α = 0.05) and that means that the chance of all three comparisons being not statistically significant is equal to (0.95*0.95*0.95 = 0.8574). And thus the chance that one or more of the comparisons will actually be statistically significant is 1-0.8574 = 0.1426. The only catch with Sidak’s method for correction is that you have to assume that all of your individual tests are independent of each other.

There is plenty more math out there
detailing how exactly the Bonferroni and Sidak methods differ, but the easiest
juxtaposition I can discern is how they go about calculating α. Bonferroni sets
α for each comparison based on the number of comparisons being done, and the
Sidak method calculates an exact α all comparisons in a reverse-thinking
method. Based on these, it is often considered that Sidak’s method produces α
values that are less stringent than those of the Bonferroni corrections.

## No comments:

## Post a Comment