John W.Tukey - Playing in Everyone's Backyard

John W. Tukey, a chemist, mathematician, educator, consultant, researcher, data analyst, and statistician, was born in New Bedford, MA in 1915, the only child of Adah M. Tasker and Ralph H. Tukey. It was evident from an early age that Tukey had great potential. His parents discovered that he had learned to read at age three after he informed them about a bridge closure that he had read about in the newspaper. Tukey’s parents were both trained as high school teachers, and his mother educated him at home until college, when he attended Brown University and received both his bachelors and masters degrees in chemistry in 1936 and 1937, respectively. Tukey then entered Princeton University with the intention of studying chemistry, but instead earned his Ph.D. in mathematics in just two years. His thesis entitled “On Denumerability in Topology” was submitted in 1939 under the supervision of the distinguished mathematician Solomon Lefschetz. After receiving his doctorate, Tukey became an instructor of mathematics at Princeton. Shortly thereafter, however, the second World War began and changed the trajectory of Tukey’s career.

Tukey served as a consultant then as the assistant director of the Fire Control Research Office (FCRO), one of the departments Princeton organized to help support the goals of the National Defense Research Committee during U.S. involvement in WWII. The FCRO studied the mechanics of artillery weapons in order to improve their precision and efficacy. Here, Tukey worked closely with physicists and engineers as well as mathematicians, including an engineer with a Ph.D. in physiology who focused on statistics named Charles Winsor. Tukey later explained that, “It was Charlie and the experience of working on the analysis of real data that converted me to statistics.” By the end of the war, Tukey described himself as a statistician, a field that he viewed not as an offshoot of mathematics but as a scientific discipline equivalent to chemistry, biology, or physics. Tukey maintained his faculty position at Princeton, helping to establish and direct its first statistics department. Following the war, he also began work at Bell Telephone Laboratories as a member of the technical staff, where he contributed to research in numerous diverse areas over the course of his career, eventually serving as the Associate Executive Director of Research Information Sciences.

Harvard professor Frederick Mosteller is credited with saying that Tukey “probably made more original contributions to statistics than anyone else since World War II,” an assertion that seems credible when looking at a list of Tukey’s accomplishments. To the field of statistics, Tukey contributed a method of determining confidence intervals used in ANOVAs, the fast fourier transform, and a series of methods of data depiction that includes the stem-and-leaf diagram and box-and-whisker plots, among many other works. He also contributed substantially to paradigms of statistical analysis, promoting the utilization of and distinction between “exploratory data analysis” and statistical hypothesis-driven “confirmatory data analysis,” and he was a vocal advocate for statistical rigor in experimental design.

John Tukey’s accomplishments were recognized with numerous accolades including the National Medal of Science awarded by President Nixon in 1973 and the IEEE Medal of Honor in 1982 for his research in the development of the fast Fourier transform (FFT) algorithm. Aside from his work in academia and at Bell Laboratories, he made an impact in many other areas, for, as Tukey put it, “The best thing about being a statistician is that you get to play in everyone’s backyard.” The NY Times’ obituary of Tukey credits him with coining the term “bit” in reference to computer data and the word “software” in 1958. He also served in a number of public capacities, including chairing a committee investigating ozone depletion in the 70s, serving as a U.S. delegate to nuclear discontinuance conferences, and serving on the President’s Science Advisory Committee.

John Tukey retired in 1985 at the age of 70, and he passed away in 2000. He aspired to be a scientific generalist, and he achieved this, leaving a broad legacy of advancement in science and technology.

Another good reference:

Brillinger, D.R. (2002). John W. Tukey: His life and professional contributions. The Annals of Statistics. 30(6): 1535-1575.

Karl Pearson: Mathematician, Statistician and a healthy dash of Sketch

Karl Pearson’s impressive career as a statistician was driven by his desire to marry statistics with biological observation. He was a devout social Darwinian who truly wanted to use science and statistics in order to contribute to the idea of “survival of the fittest.” Unfortunately, his philosophy as a social Darwinian can be summed up into a very contentious idea: Eugenics.

But before we embark on the controversy of eugenics, let’s take a step back and discuss Pearson’s early life and educational history. He was the second child of Fanny Smith and William Pearson, a barrister of the Inner Temple. Interestingly, he was born Carl, but at the age of 23 changed his name to Karl, which is how he is known today. It goes without saying that Pearson was highly educated. He, as well as his brother and sister, were educated in the home until the age of nine when he was sent to the University College School in London. He studied there until age 16, when he had to leave due to illness. He was then educated by a private tutor. He received a scholarship to King’s College, Cambridge. During his formal education, he dabbled in everything from literature, writing, law, and must importantly, mathematics.  His primary interest in statistics can be traced to his collaboration with Walter Weldon, with whom he founded biometry, which used mathematical modeling to explain biological sciences. Pearson’s interest in statistics continued through his introduction to Francis Galton, who was Darwin’s cousin and also known as the “father of eugenics.”

After graduating from Cambridge, Karl trained at the University of Heidelberg in Germany. There he studied physics under G.H. Quincke and metaphysics under Kuno Fischer. He traveled to Berlin to attend lectures and was strongly influenced by the courses he took there in physics as well as Roman literature, medieval and 16th century German literature, and socialism. He then moved back to London working first at Cambridge, then took rooms at the Inner Temple but never practiced law. He finally moved to a faculty position at University College, London. There he was appointed to the Chair of Applied Mathematics as a Goldsmid Professor.

At this point, most of Pearsons’ statistical contributions have a distinct air of eugenics about it; he used correlations and regressions to make a point that biological fitness can only be brought about by selective breeding (ie. Eugenics) and cannot be done with social reform.  So… Yeah. His motivations are a little dubious.

Pearson's Chi Square Test Statistic

But his statistical contributions were still impressive. Pearson is considered one of the founding fathers of modern statistics. One of his most recognized contributions is his work establishing the Pearson Chi-Square test which measures a “goodness of fit” between two independent, unpaired categorical variables and their respective categorical frequencies. Additionally, he built on the ideas of Galton, and laid the groundwork for future work by RA Fisher and Egon Pearson. He further developed Galton’s idea of correlation generating the correlation coefficient, known as Pearson’s r. Pearson's r is a measure the linear relationship between two variables. Pearson also developed tests to assess the distribution of data, such as evaluating skewness and kurtosis. Many people before him would often force data into a normal distribution. Last but not least, Pearson is responsible for first introducing the p-value into modern statistics. P-value statistics were further developed by RA Fisher. While Pearson’s motivations are dubious, he did lay the foundation for modern statistics, and without Pearson’s contributions we may not have seen the advancements in statistics that we know today.

William Sealy Gosset (a.k.a. Student)

William Sealy Gosset (a.k.a. Student)

William Sealy Gosset was born on June 13, 1876 as the oldest of five children. After studying chemistry and mathematics, Gosset went on to work at Guinness brewery in Dublin in 1899 where he worked unofficially as a statistician. When the brewery began research on the best yielding but cheapest varieties of barley, Gosset was able to really work on his statistical craft. By 1903 Gosset was able to calculate standard error, and in 1904 he published a report titled The Application of the Law of Error to the work of the Brewery that got the attention of Karl Pearson. Gosset spent two terms in 1906 and 1907 in Pearson’s biometrics lab studying distributional theory and the correlation coefficient. 

After learning under Pearson, Gosset took up the study of law of error and published the Probable Error of a Mean in 1908 in Pearson’s journal Biometrika. Due to disclosure issues at the brewery, Gosset was forced to publish under the pseudonym Student. This explains why his most noteworthy achievement is now called the Student’s t-distribution and not the Gosset t-distribution. Gosset’s work attracted the attention of Ronald Fisher who believed that Gosset had caused a “logical revolution”. 

Gosset would eventually go on to publish 22 manuscripts ranging from experimental design and robustness to the theory of Natural Selection. When not studying statistics, Gosset was noted to be interested in gardening, boat-building, biking, golfing, sailing, and fishing. William Gosset left Dublin in 1935 to head a new Guinness brewery in London where he later died of a heart attack in 1937 at the age of 61. He was survived by his wife, three children, and grandson.

In pursuing his goal of estimating population parameters from a small sample, Gosset ran into some heavyweight opposition, not the least of which being his former mentor Karl Pearson who said, “only naughty brewers take n so small that the difference is not of the order of the probable error”. Increasing n was too expensive for Guinness, so Gosset set out to make his own probability tables for small samples.  He designed an analog simulation, and the tediousness really makes one appreciate Excel.  Man, that thing is nice. Sans Excel, he wrote down the heights and lengths of the left middle finger of 3000 convicts (he chose convicts just because that was the data available to him) on separate pieces of cardboard.  He then separated the data into 750 groups of 4, recording the mean and standard deviation of each group. He calculated his z statistic, being (X - µ)/s, and plotted the probability scores against the population (of 3000 convicts) parameters.

Awesomely, a team at McGill ran Gosset’s methods through R, to see if his observations would hold up to the high output of computerized simulations.  See for yourself:

As they report: “Scotland Yard precision [of measuring by 1/8” instead of 1” increments] and today’s computing power would have left Gosset in no doubt that the distribution of s which he ‘assumed’ was correct was in fact correct.”

The data handed to Gosset when he began working for Guinness was difficult for him to interpret at first. The sample sizes were very small, and these small sample sizes resulted in large standard deviations that made analyzing differences between means impossible. At the time there were no distribution curves or probability tables that were appropriate to apply to data from small sample sizes. One of Gosset’s statistical focuses became how to properly analyze these types of data. It is evident today that this biased the focus of his more famous accomplishments towards studies that primarily dealt with resolving issues of small sample sizes.

Gosset needed a version of the normal distribution curve and probability table that would allow for differences between two means to be observed in experiments with small sample sizes. After studying under Karl Pearson, Gosset came up with an alternative distribution for smaller sample sizes. He published this work in Biometrika, and in his article titled “The Probable Error of a Mean,” he offered an alternative distribution (t-distribution table) that was modeled after normal distributions for larger sample sizes. His provided evidence that his new t-distribution could be applied to data from small sample sizes. He also came up with the Student’s t-test, which was later optimized by a collaborator, RA Fischer.

Since Gosset was responsible for verifying his t-distribution by comparing his calculations, curves, and tabled data to those that were previously well-established using normally distributed data, this biased the application of  t-distribution tables and the t-test to data from small samples sizes that are normally distributed. Today, when deciding on whether to use a t-test or not, we still have to be aware of one of the multiple assumptions that a t-test makes, which is that the data that you apply a t-test to are normally distributed.

It is hard to imagine a world in which inferences about a whole cannot be made from a small number of samples. Without this capability much of the research in the biological sciences would be impossible. William Gosset and his “Student’s t test” were instrumental in laying the statistical foundation upon which much of research now rests.

The Student’s t test is one of the most widely utilized statistical tests in biological research to this day. His work informed the work of Fisher and the development of the concept of “significant difference.” While Gosset found the idea of statistical significance “nearly valueless,” it has nonetheless become a pillar of modern scientific interpretation.

Gosset also made substantial contributions to the field of industrial quality control. His ideas heavily influenced W. Edward Demings, a pioneer of the quality evolution and the idea of “six sigma.”

Despite these contributions, Gosset remains little known in the lore of statitistics. This may be due to his use of the pseudonym “Student” in his publications, the fact that he was not a professor like Fisher or Pearson, or his apparent lack of an ideology. Despite his significant role in the development of modern statistics, Gosset seems to have been less interested in revolutionizing a scientific field than in solving practical problems.

Reverend Thomas Bayes

Thomas Bayes was born in 1702 in London, England. His father was one of the first nonconformist ministers in England, a path that Bayes himself would later follow. Normally, sons born into wealthy families would pursue ministry through The University of Oxford or Cambridge, but Bayes instead attended the University of Edinburgh, which was more accepting of the beliefs of nonconformists. Here he studied divinity, logic, philosophy, mathematics, and metaphysics. Bayes’ family wanted him to enter the ministry and his father was the one who recommended that he attend divinity school, which he did while he was at Edinburgh. Bayes’ was a Presbyterian minister, but he had many views that were not common among the group. During his ministry, Bayes’ wrote “Divine Benevolence” where he attempted to address the motivation of God’s actions in the world. This too, for the time but not so much anymore, was considered controversial since he attributed God’s actions to goodness and benevolence instead of God’s morals.  One can imagine that having grown up in a nonconformist family he had strong opinions of his beliefs, and I believe this reflected in his actions, the book he wrote, and the books on math he would later write.

Image Credit: https://bayesian.org/bayes

Although mathematics was not the primary focus of Bayes’ studies, he did learn mathematics at the University of Edinburgh from James Gregory, and Newtonian “natural philosophy” from David Gregory. At this time, Divinity, mathematics, physics, and divinity were all fields that contained aspects of philosophy—physics and mathematics trying to understand the world through facts and numbers, and the study of divinity trying to understand the world through God. Bayes’ contributions to mathematics were not rooted in any clear drive, but it seems that he was a philosopher true and true, and enjoyed studying philosophies. His first written was “An Introduction to the Doctrine of Fluxions, and a Defense of the Mathematicians Against the Objections of the Author of The Analyst” (which he published anonymously  in 1736). It was a rebuttal to George Berkeley, a philosopher who wrote the satirical novel The Analyst, which  essentially stated that mathematicians assume too much in their calculations and logic is not involved in the process.

As to how he became interested in probability, not much is known. Some say he became interested after reviewing an article by Thomas Simpson, which proved that a mean from a set of observations is a better estimate than one single observation of that parameter. This could have sparked his interest in the subject of probability, and led to his greatest thoughts that produced the Bayes Theorem. Although not much is known about why there was a need to develop ideas about probability, it seems that Bayes was an intellectual man that enjoyed thinking through theories and working with numbers, which enabled him to join in with the major ideas of the time and contribute to the beginning of the field of statistical thinking.

Despite his legacy, Bayes actually published very little during his lifetime. One of his most famous papers is “Essay Towards Solving a Problem in the Doctrine of Chances”, published through the Philosophical Transactions of the Royal Society of London in. In this paper, he discussed Bayes Theorem (not named at the time), in which the probability of an event was stated to happen based on a conditions that could be related to an event. This was found and submitted posthumously by Bayes’ friend Richard Price. In this paper he provided the basis for what is to be later known as the  Bayesian estimation, a method for evaluating beliefs based on the concept “the probability of an event that has to happen in a given circumstance on a prior estimate of its probability under these circumstances.”

Image Credit: https://en.wikipedia.org/wiki/Bayes_theorem

The Bayes Theorem is a way of directly applying conditional probabilities to account for new evidence. For example, our belief P(A|B) is determined by multiplying our belief P(A) by the likelihood P(B|A) that B will occur if A is true.

To illustrate his theorem, Bayes used the following thought experiment: Drop a ball on a table. The table is such that the ball has just as much chance of landing at any one place on the table as anywhere else. Now figure out where the ball is, without looking.Throw another ball on the table and report whether it is to the left or the right of the first ball. If the new ball landed to the left of the first ball, then the first ball is more likely to be on the right side of the table than the left side. Throw the second ball again. If it again lands to the left of the first ball, then the first ball is even more likely than before to be on the right side of the table. With each throw, you can narrow down the area in which the first ball probably sits. Each new piece of information constrains the area where the first ball probably is. Another good example and explanation of Bayes Theorem based on cancer screening can be found here.

For many years after the work was published, it did not receive much criticism until 1854 when George Boole published his work “Laws of the Thought”, which questioned many of the ideas put forth by Bayes. The criticisms put forth by Boole are still alive today. He argued that the use of inference for model parameters is “misguided and uses theoretical and unobservable, theoretical quantities”.

An article by Andrew Gelman at Columbia University published in 2008 states that there are two main reasons why the Bayesian method could be biased. First, they infer that the Bayesian method is presented as automated hierarchical models used to handle multiple parameters, while in the modern statistical research the focus is on the minimization of the assumptions to find a reasonable answer. The second point they make in the article is that Bayes methods raise concern in terms of the objectivity of the knowledge, in the sense that prior and posterior distributions are dictated from a subjective experience, which is harder to assess. The authors also claim that Bayesian statisticians are more prone to a computer-based approach to solve problems rather than analyze the experimental design and analysis of variance.

Bayes remained a minister until 1752, at which point he retired. He was admitted into the Royal Society in 1742, a private group of prominent scientists from all disciplines. The funny thing is, he had no published work on mathematics at that time (just the rebuttal to Berkeley). According to The Official Guide to Bunhill Fields, he was admitted to the Royal society also based on a tract of 1736 in which Bayes defended the views and philosophy of Sir Isaac Newton.

Thomas Bayes died on April 17th, 1761, three years before the publication of “Essay Towards Solving a Problem in the Doctrine of Chances”. He was buried in the Bunhill Fields in London, a burial ground for many nonconformists.

Sir Ronald Aylmer Fisher

Sir Ronald Aylmer Fisher (1880-1962) is a sublime candidate for a mathematician who developed statistical means to test the validity of scientific research happening in the scientific community around him. Luckily, Fisher being active in London in the early 20th century, he was surrounded by the relatively recent work of prominent names in evolutionary biology and statistics such as Charles Darwin, Gregor Mendel.

Though Fisher studied Astronomy at the University of Cambridge, he had a deep interest in evolutionary theory. In 1918, he published a paper title The Correlation of Relatives on the Supposition of Mendelian Inheritance that bridged the inconsistencies between the works of Mendel and Darwin. It is often thought that if Darwin and Mendel had not been so isolated from each other's’ work during their lives, the field of evolutionary biology would have developed very differently. Fisher was aware of both of their work, and his paper caught the attention of renowned statistician Karl Pearson, who offered him a spot in his laboratory at University College London.

Instead, Fisher chose to take up a position at the Rothamsted Experimental Station, which today is the world’s oldest agricultural research station. This turned out to be a favorable step in Fisher’s career as a statistician, because at Rothamsted, he was given access to incredibly large databases of plant breeding experiments dating back decades. Not only did he have access to past experiments, Fisher was able to design his own studies, and be part of the statistical process in planning and setting up an experiment from the very beginning. In 1925, Fisher published Statistical Methods for Research Workers, which remained in print for over 50 years, serving as a guidebook for statistics for scientists around the world.

In fact, when Fisher did go to University College London, he returned not as just a statistician, but as a professor. He would then go back to Cambridge in 1943 as the Balfour Professor of Genetics, where he continued working until 1957. During his time at Cambridge, Fisher published Statistical Methods and Scientific Inference, a publication that still serves as a landmark in the history of statistics as a collection of all the important achievements of Fisher’s career as a statistician, and was knighted by Queen Elizabeth II for his contributions to science and society.

The legacy left behind by Fisher is a tall order to list briefly. He was the first to introduce the concept of randomization during experimental setup. Aware of the biases that occurred during scientific research, Fisher emphasized the role of randomizations and controls in ensuring the validity of an experimental finding. Fisher’s earliest work developed the idea of maximum likelihood, the method for estimating the parameters of a statistical model given a dataset, the popularization of which Fisher was responsible for. He coined the term ‘Null hypothesis’ and even set the convention of the arbitrary p value to 0.05 to determine statistical significance. Perhaps one of his most prominent achievements was the development of analysis of variance, or the ANOVA test.

Fisher is known for several other contributions, including Fisher’s principle, the model explaining the uniformity of the male/female ratio in sexually reproducing populations. He explained the Fisherian Runaway, the model of evolution that outlines female preferences for highly ornamented males and the Sexy Son Hypothesis, the model of evolution that females select mates based on the mate’s ability to produce sexually attractive offspring, not the male’s ability to provide or nurture. Fisher is even considered by some to be one of the founders of population genetics, as he spent the majority of his career studying the distribution and frequency of alleles within a population along with factors of adaptation, speciation, and population structure.

Even with such a resume, Fisher was not completely free of biases. He never agreed to the association between tobacco and lung cancer, as was shown in the mid-20th century. Though it is somewhat comical that some think he accepted funds from the tobacco firms for his opinion on that work.

After his time at Cambridge, Fisher moved to Australia, where he continued to conduct research until his death in 1962. He was a statistician with impeccable contributions to his field and immaculate skill in mathematics to execute his ideas, which will continue to always play a central role in how statistics is done by scientists.