Statistics to many is a non-intuitive, mathematical jungle
fought through by hours of rigorous study and contemplation. Even to those that
are well versed, there are still aspects of statistics that seem to defy logic.
One phenomenon that illustrates this paradigm well is Simpson’s paradox, which
is defined as a set of data where a trend observed in each of the individual
groups disappears or reverses when the groups are combined. This paradox has
been shown in data collected from baseball, clinical research, and sociology studies.
For
example, let us consider a group of five individuals: Josh, Michael, Jessica,
Faith and Marcus. Each of these individuals decided after watching the Nathan’s
Famous hotdog eating competition that their new passion in life was to become a
competitive eater. Over a ten-year period, each of the individuals competed in
five competitions and enlisted the help of a statistician to analyze their
progress. As you can see in the graphs below, through rigorous training each of
the individuals managed to steadily increase the amount of hotdogs they could
consume in a ten-minute period over the ten-year time period they were
followed.
From this data, one may conclude that overall
individuals will tend to increase the amount of hotdogs they can consume in a
ten-minute period. This seems logical considering there is a strong upwards
trend in each of the graphs above. When all of the data is grouped together,
however, the opposite trend appears. As shown in the graph below, the grouped data
from the five individuals clearly shows that the overall trend between hotdogs
eaten in ten-minutes and age is negative.
This is Simpson’s paradox in action. The trend observed in
each of the individual data sets reversed when these data sets were combined.
Personally,
I believe this example, and many data sets in which Simpson’s paradox is
observed, should raise some questions. Do these data belong in a grouped
analysis? Are there confounding variables that can explain the trend in the
grouped data relative to the individual data? Were the proper experimental and
statistical protocols followed?
In this
example, one could argue that the data do not belong in a grouped analysis
since each individual began competitive eating at a different age. If the data
were plotted with the x-axis as years after beginning competitive eating, the
overall trend matches the individual data as seen below.
Overall, if data you are analyzing displays Simpson’s
paradox, it may be wise to step back and analyze your data collection and
analysis. Although the data may just display a paradoxical trend, it is quite
possible that the trend arose from flawed collection or analysis.
This post does a wonderful job of explaining a difficult topic and the graphs give a great illustration on what Simpson's Paradox is. I have seen this in papers before, when the author will put up a bunch of different graphs, all with different scales and make conclusions about the group in general having a specific trend. Sometimes, the conclusions are probably correct, but other times maybe not.
ReplyDeleteCam, this is such an insightful post. I've certainly seen this phenomenon occur in other studies where data is grouped that likely should not be due to the vast disparities in uncontrolled variables. Many of said studies should attempt to have the data normalized through a shared measure amongst the experimental groups. For example in your situation - eating ability -> normalization could be garnered from gross stomach size by means of volume. Though this would be hard to measure and would need to be generalized by total body mass. Thanks for such a thought provoking post!
ReplyDelete