Using Prediction Markets to Forecast the Spread of Infectious Diseases
“Using Prediction Markets to Forecast Trends in Infectious Diseases” by Phillip Polgreen, Forest Nelson, and George Neumann: https://pdfs.semanticscholar.org/9137/6ffb25ec0317ba6c4dec0393cc817b6a496b.pdf
In this paper, Polgreen, Nelson, and Neumann discuss the potential of prediction markets to forecast the outbreak and spread of infectious diseases. They focus on the results of an experiment they performed to test the ability of a prediction market to forecast the spread of the influenza virus in Iowa. Although the “flu season” occurs every year at approximately the same time, the extent to which the flu spreads in a given region, the strain of influenza that causes it, as well as various other factors vary widely between different years and are difficult to predict in advance, such that there is sufficient uncertainty for a prediction market to be sensible. The authors performed an experiment during the 2004-2005 flu season in which they created a prediction market for the spread of the influenza virus in Iowa consisting of 62 traders that were various types of health care professionals. Each trader was given 100 virtual dollars that would be converted into grant money after the close of the market. Prediction contracts were created for the level of influenza activity in Iowa during 1-week periods occurring every other week during the flu season, and were offered 8 weeks in advance of the period for which they predicted. Every set of contracts for a 2-week period consisted of 5 options to describe the level of influenza activity ranging from “no activity” to “widespread,” in accordance with the 5-level classification system used by the Center of Disease Control (CDC), whose official classifications were used to determine which contracts would be deemed correct. This prediction market was able to predict the level of the influenza activity approximately 4 weeks in advance of a conventional statistical method based on historic data would have. Although this market was too small to draw statistically reliable conclusions, it provides substantial reason to believe that prediction markets can serve as a useful supplement to other tools in detecting or predicting the outbreak and spread of infectious diseases.
Polgreen, Nelson, and Neumann discuss various significant advantages prediction markets have over conventional methods of statistical data collection and analysis. Conventional statistical studies and data reports released by public health organizations that are typically currently used by health care professionals are based on old data. This information lag exists because conventional data collection methods require substantial effort and resources to conduct, and because the information collected is diverse and potentially subjective, is time consuming to combine for analysis. This lag, which is typically at least 1-2 weeks, substantially limits the utility of the information in preventing disease spread. In contrast, one of the most important assets of prediction markets in the context of preventing the spread of fast-spreading infectious diseases is that they collect, aggregate, and make publicly available data on a continuous basis. Further, prediction markets could be used to predict the strain of influenza present in an outbreak of the flu, to aid in the manufacture and distribution of vaccines. Another utility of prediction markets is that they provide a detailed record of when particular information likely became known to specific health care professionals, which can be used to improve the accuracy of conventional disease detection systems. Prediction markets also require less resources to operate than do surveys and other data-collection techniques, and experiments have shown that they require a much smaller number of participants than do surveys to yield results of similar accuracy.
The content of this paper directly relates to our discussion in class about prediction markets, which are very interesting because, as in the above discussed example, they can provide an accurate means of predicting an uncertain outcome without the need for a single entity to devote resources into performing a study involving data collection and analysis. The fundamental basis of a prediction market is that the prediction market aggregates a large number of independent indicators of some uncertain outcome that if observed individually would not be a strong indicator of the outcome, but which, if not subject to widespread bias, when averaged form a strong indicator of the outcome. The mathematical basis of the accuracy of a prediction market is largely conceptually similar to the mathematical basis of the law of large numbers that is critical to the field of statistics, which states that the average of a large number of identically distributed random variables converges to the actual mean of the quantity being measured as the number of averaged distributions become large. For example, in the case of the prediction market for the spread of the flu, the market odds were the average of many predictions based on independent data sources such as different doctors and hospital workers. A significant rise in symptoms of the flu seen by a single doctor taken alone is not a strong indicator that there is a rise in the prevalence of the flu in that region, because random variation in the frequency at which flue symptoms are observed are naturally expected to occur. However, a significant rise in flu symptoms seen by a large number of doctors is likely to be statistically significant. This rise would be reflected in the price of the relevant contract in the flu prediction market. As the number of traders participating in the market increases, the number of independent data points increases, so the probability distribution of the prediction market odds will become increasingly reliably centered on the true odds, and the prediction market is increasingly likely to be correct. One interesting variation of a conventional prediction market is a combinatorial prediction market. In a combinatorial prediction market market, participants bet on combinations of outcomes of different events. In deciding how to place their bets in such a market, betters should use the mathematics of conditional probability discussed in class to determine the probability of a combination of events given their beliefs regarding both the probability of individual outcomes and the probabilities of event outcomes given the outcome of other events.
It is very important to be aware of situations in which prediction markets may fail. The ability of a prediction market’s odds to converge to the true odds of an outcome is critically dependent on the contributing bets being based on independently developed beliefs. This requirement is violated if there exists some widespread bias that influences a large portion of the contributing beliefs in the same direction, in which case the market odds would converge to odds that are above or below the true odds. For example, many participants’ beliefs may be influenced by the same inaccurate media reports, or many participants’ evaluation of the likelihood of an outcome may suffer from similar fallacies in reasoning. One possible cause for the failure of a prediction market due to widespread bias is the rise of an infinite loop of self-reinforcing beliefs. In such a situation, traders base their beliefs largely on the current odds of the markets. This moves the market odds further in their current direction, which results in more traders betting in accordance with those odds, and this process continues. This process is conceptually similar to the process that gives rise to the information cascades discussed in lecture. In the model of information cascades developed in this course, participants, who have access to their own private information and the decisions of previous participants, but not the private information of previous participants, base their decisions largely on the decisions of previous participants. As a result, a sufficiently large initial group of incorrect decisions results in that the population of participants believing in that incorrect decision, much like how incorrect prediction market beliefs may reinforce themselves and become widespread.
In discussing the potential utility of prediction markets in predicting outcomes in various situations, it is important to note that their reliability is critically dependent on the extent to which participants’ behaviors can be accurately modeled in a way such that their bets reflect their true beliefs. If participants’ bets are not representative of their true beliefs, the basis of the prediction market as a tool for accurate prediction of outcomes collapses, because the market odds are no longer an average of many experts’ predictions. In economics, a market scoring rule (MSR), also called a “score function,” is the relationship between the payoff to an individual, or score, and the set of probabilities of different outcomes and their relationship to a particular quantity. In the context of prediction markets, the market scoring rule can be thought of as a relationship between the probabilities of winning various amounts of money, which would be determined by a participant’s placement of bets on various outcomes, and the expected payoff of that set of probabilities. For example, in lecture, the MSR was simplified such that expected payoff was a function solely of expected monetary gain, in that the payoff was proportional to the logarithm of expected monetary gain. A “proper scoring rule” is a scoring rule in which the set of bets that maximizes expected payoff is the set of bets that reflects a participant’s true beliefs of the probabilities of the possible outcomes. The example discussed in lecture, in which expected payoff is proportional to the logarithm of expected monetary gain, is a common type of proper scoring rule that has been observed to accurately model behavior in many situations. A noteworthy example of a MSR that is not proper is if the expected payoff is proportional to the expected monetary gain. In this case, a better should place all of its available wealth on a single outcome, that outcome being determined by the better’s belief in the probability it will win and the market odds for that outcome. In considering the use of prediction markets as accurate indicators of an outcome, we should make sure that its traders’ behavior can be well approximated by a proper scoring rule.
To understand why prediction markets have the potential to be very useful to aid in preventing the spread of infectious diseases, it is important to understand the nature of the spread of infectious diseases. Typically, initially only a small number of members of a population have a disease. Every time they interact with another member of the population in a way which is conducive to the transmission of that particular type of disease, they have some probability of transmitting the disease to the other member. At a rate dependent on the nature of the disease under consideration, the disease may spread to an increasingly large portion of the population. As the number of people which have the disease, and which are likely to get the disease if preventative measures are not taken, becomes larger, it is increasingly difficult to mitigate the spread of the disease. So, it is very important to know about the a disease outbreak, and to know as much information as possible about the nature of the disease, as early on as possible, so that available resources may be used to successfully prevent the disease’s spread. This is especially prominent in the example of the flu discussed in the paper, because there exists reliable vaccines suitable for the different possible strains of the flu that if distributed quickly enough and to the appropriate people, are effective in stopping the spread of the flu.