Skip to main content



Statistical Fix for the Replication Crisis and Bayes Theorem

The article suggests that the statistical standard for “statistically significant” is too high to take the fact that many statistically significant results aren’t replicating into consideration. Instead of setting the p-value as 0.05 to mark as “statistically significant”, the author believes that 0.005 seems like a more accurate bar. Although their idea gets rejected by another group at the ASA Symposium on Statistical Inference, the author still insistes on a higher standard.

The author starts the claim by first providing an example of a trail of a new drug to cure cancer. He is trying to understand if the new treatment is better than standard. Then he points out that the problem is the results can’t be replicated and when they are doing statistical test, it should also be considered in p-value. Then he mentions Bayes theorem, what we discussed in networks lecture, to support his idea and figure out what the true p-value should be in that case.

In general, Bayes’ theorem describes the probability of an event, based on prior knowledge of conditions that might be related to the event (Wikipedia). The most straightforward understanding of the concept is the formulation P(A|B) = P(B|A)P(A)/P(B). Here in the article, the author suggests that since the most results are not replicating, they should combine the Bayes factor with the prior odds to compute the probability that the treatment is better, which more comprehensively and accurately measures if the treatment is better than standard. After recalculating the probability by Bayes theorem, the author points out that there is a conflict with the result from p=0.5. However, when the group at the ASA symposium disagrees with the proposal, one of the reasons is that Bayes factor is too subject and researchers can make other assumptions that might change their conclusions.

Link: http://www.sfchronicle.com/news/article/A-statistical-fix-for-the-replication-crisis-in-12289045.php

Comments

Leave a Reply

Blogging Calendar

November 2017
M T W T F S S
 12345
6789101112
13141516171819
20212223242526
27282930  

Archives