Bringing Game Theory to Statistics: Hypothesis Testing & Error
Statistical testing founds itself upon ideas of game theory. Every time one conducts a test for statistical significance, they consent to play a game of odds. The predetermination of null and alternative hypotheses necessitates the definition of a value alpha (α), which defines the chance of rejecting the null hypothesis by random chance. Type I errors can occur when the null hypothesis is falsely rejected (i.e. a false positive). As the α value increases, so does the likelihood of a Type I error occurring. Conversely, a Type II error occurs when one fails to reject a false null hypothesis, also called a false negative (Table 1).
Table 1: Error Conditions
The most commonly accepted and utilized value of alpha is .05. This number was arrived at through attempts to minimize the risk of a Type II error. It becomes apparent why the minimization of a Type II error might be preferable to the minimization of a Type I error by way of example. A patient has preliminary tests done for the presence of a particular disease. If an error is to be made, it would surely be better to return a false positive than a false negative. In the case of a false positive, more precise tests will inevitably be run that will conclude in a negative diagnosis. However, in the case of a false negative, the patient has no idea that they have the disease in question, and have no reason to suspect otherwise. By analyzing games in a worst-case scenario against nature (rather than against a decision-making player), a minimum Type II error for can be computed. To obtain the proper In the case of Bernoulli distributions, with a sample size of 25, it was found that “the maximal added type II error” (that is, the probability of a Type II error – the minimum Type II error) is .046, when alpha is equal to .05.
The world of statistics welcomes concepts of game theory with open arms, relying on ideas on inference and assumption to extrapolate information about a large number of objects from a smaller subset of those same objects. However, due to inherent randomness, most statistical conclusions are based on chance. Applications of game theory can help one to arrive at the most appropriate amount of chance to minimize the chance of error and maximize confidence in one’s answer.
Source: http://www.econ.upf.edu/docs/papers/downloads/1099.pdf