How Bayes Rule Helped Solve a Historic Mystery
Article link: https://priceonomics.com/how-statistics-solved-a-175-year-old-mystery-about/
The article is about the mystery behind the Federalist Papers, specifically, who exactly wrote the 12 unclaimed papers. It had been known that John Hay only wrote 5 papers and that 73 others were claimed by either James Madison or Alexander Hamilton, which left 12 without a definite author. As all three men wrote under a united front, Publius, they wrote similar in style and content. However, to solve the mystery behind these 12 papers, mathematicians used an early form of Bayes Rule to distinguish differences between the papers in order to find the true author. While in style and sentence length Madison and Hamilton were practically indistinguishable, the mathematicians decided to look at individual words the men used in their writing as well as how frequently each word came up. They examined the use of common words, typically conjunctions like “while” vs “whilst” and articles and the probability that a word would be said in a certain author’s paper, such as the probability that “upon” would be used by Hamilton vs by Madison. As this study was done in the 1960s, the mathematicians actually had to cut up each word in the papers and group them to see the probability a word was used every so many words. They found roughly 30 discriminator words that set apart the writings of Hamilton and Madison, and found that Hamilton was much more likely to use than Madison; more specifically, in the 49 Hamilton papers, he used on average 3.24 “upons” per 1000 words whereas Madison’s average was 0.23. The 12 papers were compared with the discriminator words, and it was found that the 12 papers very rarely used “upon”, which highly suggested that Madison was the true author as he also had low probability of using that word. The odds were found to be 12.6 million-to-1 that Madison wrote the 12 papers. The following graph from the article depicts the relationship between the frequency of the use “upon” in a paper and the number of papers it is used in for each author.
( https://priceonomics.com/how-statistics-solved-a-175-year-old-mystery-about/)
As Bayes Rule was a main topic of chapter 16 and is the basis of this discovery, it is obvious the article connects greatly to the topic discussed in class. Bayes Rule gives us a way to determine probabilities based on information that is observed. For example, the probability that an event A will occur given that some other even B has occurred can be found using Bayes Rule:
.The mathematicians solving the case used a form of Bayes Rule to help determine the author of the papers. While the article doesn’t go into the algorithm, it can be seen by how they describe how they determined the author of the 12 papers that the probability that Hamilton wrote the 12 authors is dependent on observed information about the already claimed papers (the probability that Hamilton uses certain discriminators in his works, the probability that Madison uses certain discriminators in his papers, and the probability of discriminators in the 12 papers). These probabilities can be analyzed to determine a good probability of an event occurring, such as the probability that Hamilton wrote the 12 papers. Bayes Rule explains well what the mathematicians did as they determined the probability that Hamilton wrote the papers given the probability Hamilton uses certain discriminator words as well as the probability Madison does and the probability the papers use those words. I found it really interesting how the mathematicians found such a high odds that Madison wrote the 12 papers and how they had to spend years deciphering the texts to determine a results whereas now technology has developed so much that there are computer programs that can solve these issues in no time at all.