Tuesday, April 5, 2016

Simpson's Paradox

Statistics to many is a non-intuitive, mathematical jungle fought through by hours of rigorous study and contemplation. Even to those that are well versed, there are still aspects of statistics that seem to defy logic. One phenomenon that illustrates this paradigm well is Simpson’s paradox, which is defined as a set of data where a trend observed in each of the individual groups disappears or reverses when the groups are combined. This paradox has been shown in data collected from baseball, clinical research, and sociology studies.

For example, let us consider a group of five individuals: Josh, Michael, Jessica, Faith and Marcus. Each of these individuals decided after watching the Nathan’s Famous hotdog eating competition that their new passion in life was to become a competitive eater. Over a ten-year period, each of the individuals competed in five competitions and enlisted the help of a statistician to analyze their progress. As you can see in the graphs below, through rigorous training each of the individuals managed to steadily increase the amount of hotdogs they could consume in a ten-minute period over the ten-year time period they were followed.
From this data, one may conclude that overall individuals will tend to increase the amount of hotdogs they can consume in a ten-minute period. This seems logical considering there is a strong upwards trend in each of the graphs above. When all of the data is grouped together, however, the opposite trend appears. As shown in the graph below, the grouped data from the five individuals clearly shows that the overall trend between hotdogs eaten in ten-minutes and age is negative.
This is Simpson’s paradox in action. The trend observed in each of the individual data sets reversed when these data sets were combined.
Personally, I believe this example, and many data sets in which Simpson’s paradox is observed, should raise some questions. Do these data belong in a grouped analysis? Are there confounding variables that can explain the trend in the grouped data relative to the individual data? Were the proper experimental and statistical protocols followed?
In this example, one could argue that the data do not belong in a grouped analysis since each individual began competitive eating at a different age. If the data were plotted with the x-axis as years after beginning competitive eating, the overall trend matches the individual data as seen below.
Overall, if data you are analyzing displays Simpson’s paradox, it may be wise to step back and analyze your data collection and analysis. Although the data may just display a paradoxical trend, it is quite possible that the trend arose from flawed collection or analysis. 


  1. This post does a wonderful job of explaining a difficult topic and the graphs give a great illustration on what Simpson's Paradox is. I have seen this in papers before, when the author will put up a bunch of different graphs, all with different scales and make conclusions about the group in general having a specific trend. Sometimes, the conclusions are probably correct, but other times maybe not.

  2. Cam, this is such an insightful post. I've certainly seen this phenomenon occur in other studies where data is grouped that likely should not be due to the vast disparities in uncontrolled variables. Many of said studies should attempt to have the data normalized through a shared measure amongst the experimental groups. For example in your situation - eating ability -> normalization could be garnered from gross stomach size by means of volume. Though this would be hard to measure and would need to be generalized by total body mass. Thanks for such a thought provoking post!