Reverend Thomas Bayes

Thomas Bayes was born in 1702 in London, England. His father was one of the first nonconformist ministers in England, a path that Bayes himself would later follow. Normally, sons born into wealthy families would pursue ministry through The University of Oxford or Cambridge, but Bayes instead attended the University of Edinburgh, which was more accepting of the beliefs of nonconformists. Here he studied divinity, logic, philosophy, mathematics, and metaphysics. Bayes’ family wanted him to enter the ministry and his father was the one who recommended that he attend divinity school, which he did while he was at Edinburgh. Bayes’ was a Presbyterian minister, but he had many views that were not common among the group. During his ministry, Bayes’ wrote “Divine Benevolence” where he attempted to address the motivation of God’s actions in the world. This too, for the time but not so much anymore, was considered controversial since he attributed God’s actions to goodness and benevolence instead of God’s morals.  One can imagine that having grown up in a nonconformist family he had strong opinions of his beliefs, and I believe this reflected in his actions, the book he wrote, and the books on math he would later write.

Image Credit: https://bayesian.org/bayes

Although mathematics was not the primary focus of Bayes’ studies, he did learn mathematics at the University of Edinburgh from James Gregory, and Newtonian “natural philosophy” from David Gregory. At this time, Divinity, mathematics, physics, and divinity were all fields that contained aspects of philosophy—physics and mathematics trying to understand the world through facts and numbers, and the study of divinity trying to understand the world through God. Bayes’ contributions to mathematics were not rooted in any clear drive, but it seems that he was a philosopher true and true, and enjoyed studying philosophies. His first written was “An Introduction to the Doctrine of Fluxions, and a Defense of the Mathematicians Against the Objections of the Author of The Analyst” (which he published anonymously  in 1736). It was a rebuttal to George Berkeley, a philosopher who wrote the satirical novel The Analyst, which  essentially stated that mathematicians assume too much in their calculations and logic is not involved in the process.

As to how he became interested in probability, not much is known. Some say he became interested after reviewing an article by Thomas Simpson, which proved that a mean from a set of observations is a better estimate than one single observation of that parameter. This could have sparked his interest in the subject of probability, and led to his greatest thoughts that produced the Bayes Theorem. Although not much is known about why there was a need to develop ideas about probability, it seems that Bayes was an intellectual man that enjoyed thinking through theories and working with numbers, which enabled him to join in with the major ideas of the time and contribute to the beginning of the field of statistical thinking.

Despite his legacy, Bayes actually published very little during his lifetime. One of his most famous papers is “Essay Towards Solving a Problem in the Doctrine of Chances”, published through the Philosophical Transactions of the Royal Society of London in. In this paper, he discussed Bayes Theorem (not named at the time), in which the probability of an event was stated to happen based on a conditions that could be related to an event. This was found and submitted posthumously by Bayes’ friend Richard Price. In this paper he provided the basis for what is to be later known as the  Bayesian estimation, a method for evaluating beliefs based on the concept “the probability of an event that has to happen in a given circumstance on a prior estimate of its probability under these circumstances.”

Image Credit: https://en.wikipedia.org/wiki/Bayes_theorem

The Bayes Theorem is a way of directly applying conditional probabilities to account for new evidence. For example, our belief P(A|B) is determined by multiplying our belief P(A) by the likelihood P(B|A) that B will occur if A is true.

To illustrate his theorem, Bayes used the following thought experiment: Drop a ball on a table. The table is such that the ball has just as much chance of landing at any one place on the table as anywhere else. Now figure out where the ball is, without looking.Throw another ball on the table and report whether it is to the left or the right of the first ball. If the new ball landed to the left of the first ball, then the first ball is more likely to be on the right side of the table than the left side. Throw the second ball again. If it again lands to the left of the first ball, then the first ball is even more likely than before to be on the right side of the table. With each throw, you can narrow down the area in which the first ball probably sits. Each new piece of information constrains the area where the first ball probably is. Another good example and explanation of Bayes Theorem based on cancer screening can be found here.

For many years after the work was published, it did not receive much criticism until 1854 when George Boole published his work “Laws of the Thought”, which questioned many of the ideas put forth by Bayes. The criticisms put forth by Boole are still alive today. He argued that the use of inference for model parameters is “misguided and uses theoretical and unobservable, theoretical quantities”.

An article by Andrew Gelman at Columbia University published in 2008 states that there are two main reasons why the Bayesian method could be biased. First, they infer that the Bayesian method is presented as automated hierarchical models used to handle multiple parameters, while in the modern statistical research the focus is on the minimization of the assumptions to find a reasonable answer. The second point they make in the article is that Bayes methods raise concern in terms of the objectivity of the knowledge, in the sense that prior and posterior distributions are dictated from a subjective experience, which is harder to assess. The authors also claim that Bayesian statisticians are more prone to a computer-based approach to solve problems rather than analyze the experimental design and analysis of variance.

Bayes remained a minister until 1752, at which point he retired. He was admitted into the Royal Society in 1742, a private group of prominent scientists from all disciplines. The funny thing is, he had no published work on mathematics at that time (just the rebuttal to Berkeley). According to The Official Guide to Bunhill Fields, he was admitted to the Royal society also based on a tract of 1736 in which Bayes defended the views and philosophy of Sir Isaac Newton.

Thomas Bayes died on April 17th, 1761, three years before the publication of “Essay Towards Solving a Problem in the Doctrine of Chances”. He was buried in the Bunhill Fields in London, a burial ground for many nonconformists.