Bayesian Statistics and Reasoning
A central concept in Bayesian statistics is that the probability you assign to an event’s occurrence depends intimately on what you know. The results of Bayesian reasoning range from obvious (consider that the probability of getting heads twice in two tosses of a coin is zero if the first toss gives tails) to surprisingly counterintuitive. The famous Monty Hall problem (in which a prize awaits behind one of three doors, and after you guess a door, one of the other two doors is revealed not to contain the prize, and you are then asked whether you should change your guess, which you should because there is a ⅓ chance the door you selected is the door containing the prize, and a ⅔ chance that it is the other that has not been opened), for instance, provides a result of Bayes’ theorem that rather defies one’s intuition. Bayesian principles, which draw “on deep philosophical debates about the nature of reality,” nevertheless provide a basis for powerful statistical methods with widespread application.
Common in fields like psychology is a more simple form of statistical analysis based solely on the frequency with which an event occurs. Statements about the probability that something is the case can be made on the basis of the outcomes of a series of experiments. Such statements are taken as conclusions that something is the case when it is highly unlikely that the result occurred simply by chance. Often, however, the threshold for drawing such conclusions, beyond which results are said to be ‘statistically significant,’ is set as high as 5%. Statistics professor Andrew Gelman of Columbia University points out that this means we expect one in twenty ‘statistically significant’ results to be a result of chance, rather than a result of the conclusion that is being drawn.
All of this goes hand-in-hand with what is being called the ‘replication crisis’ — that replications of studies in fields such as biology and psychology often fail to reach the conclusions of the original authors, thus calling the validity of their conclusions into question. Bayesian statistics have been helpful as a sanity check in revisiting studies that are being revisited. Psychology professor Uri Simonsohn of the University of Pennsylvania points out that, in addition to questionable statistical methods, inferences drawn from statistics are commonly based on dubious logic.
Unfortunately, the temptation to draw interesting and counterintuitive conclusions from data is made greater by the higher publishability of such conclusions. And in the field of psychology, where the movement to replicate studies has only recently begun, appealing results are often called conclusions as the result of a single study. Compare this to the field of physics, for instance, where results are often qualified to the point of seeming unimportant, and independent verifications are the standard. It of course would be unwise to make categorical statements without first verifying that your conclusions are indeed statements about the physical world, and not simply possible explanations for statistical fluctuations. So it is auspicious that attempts to replicate studies in psychology are now taking place.
In class, we have given some time to studying information cascades, an interesting result of Bayes’ theorem. Curiously, this effect, too, was quite counterintuitive. Likewise, answers to questions about conditional probabilities are often difficult to intuit, and must instead be worked out by application of the theorem. Perhaps this conflict between statistical and human reasoning would make for an interesting topic of psychological study (verified, of course, in independent trials).
Sources: http://nyti.ms/1mIlGiD http://gu.com/p/3qxtx/sbl