Skip to main content



Human Error in the Urn Game

http://onlinelibrary.wiley.com/store/10.1093/ei/39.4.609/asset/ei%25252f39.4.609.pdf?v=1&t=i28bxn97&s=8225525c7a67bc97b8953718478c6da01bff8d5e

In class, one of the applications for Bayes’ Rule we discussed was to calculate probabilities in the urn game. That is, say we have an urn with red and blue marbles inside, two of one color and one of the other color, with a 1/2 chance of either color being the majority. If we draw a red marble, it is not very obvious what the probability is that the urn is majority red GIVEN that we drew a red marble. However, it is clear what the probability is that the urn is majority red (1/2), what the probability is that we draw a red marble given a majority red urn (2/3), and what the probability is of drawing  a red marble at all (1/2). Bayes’ rule gives us a relationship between these 4 probabilities, so that we can find the conditional probability we want in terms of the other 3 probabilities (2/3 in this case). This method extends to multiple rounds of the urn game, where players draw marbles in turn and announce their guess of what the majority color of the urn is, so that each player knows the guesses of all players that went prior (but not necessarily the actual color of their draw).

However, there is one major assumption we made in our analysis of the urn game. In our calculations, we assumed all players were perfectly reasoning individuals who would always guess the more probable majority color from the information they were given (and would guess the color of their draw if they found themselves with an equal probability of either outcome). Unfortunately, the real world is not so ideal. People make mistakes in their guesses, which can alter the outcome of the game. In this study, the researchers played the urn game the same way that we describe it, but they used a slightly different formula in their probability calculations to account for human error. In particular, their formula had a precision parameter, λ, that indicated the amount of error in the players’ decisions. When λ=0, their probability expression for the likelihood of a player making the right choice simplifies to 1/2, which means that the player is essentially picking randomly – they have a 1/2 chance of picking the right (i.e. higher probability) answer and a 1/2 chance of picking the wrong answer. When λ=∞, their probability for the player making the right choice simplifies to 1, which means the player will always make the right choice. Within this framework that accounts for error, they then ran the test under different conditions – in one case, the subjects got a reward just for participating, while in the other cases, subjects got rewards only when they guessed correctly. What they found was that error in guesses was much higher in the participation-reward scenario. This makes some sense since people are more likely to make mistakes, guess randomly, or even guess the wrong answer when the payoff for guessing correctly is the same as if they guess incorrectly. Another interesting result they found was that cascade decisions were far less likely in the participation-reward scenario, about 40% compared to about 75% in the scenarios rewarding correct guesses. Again, this phenomenon can probably be explained by the fact that people are more likely to guess whatever they want when there is no incentive for guessing correctly, so that a cascade is less likely to happen since the participants may guess differently than the cascading answer (since they have nothing to lose). Meanwhile, in the case where there is a reward for correct guesses, cascades and cascade answers are much more likely, since people are more likely to join the cascade and guess the same as everyone else (under the assumption that they will be more likely to guess correctly if they do so).

The urn game is an interesting situation to examine when considering conditional probabilities and the occurrence of information cascades or herd mentality scenarios. However, it is important to remember that human nature dictates that we are not perfect and are liable to make mistakes or incorrect decisions. Therefore, while it is interesting to examine the outcomes of the urn game under various ideal conditions, one must also take into account human error when looking to model more realistic situations.

Comments

Leave a Reply

Blogging Calendar

November 2014
M T W T F S S
 12
3456789
10111213141516
17181920212223
24252627282930

Archives