Big Data: A self-fulfilling prophecy?
Something I found interesting about prediction markets was the idea of the self-fulfilling prophecy, and how it can be embedded into network behavior and the outcomes that manifest because of it. This seems very similar to the ideas behind machine learning–when we give a computer information inputs, it produces a certain output by weighting the inputs and taking their sum. Bets (on horses for example) are also weighted based on how an individual predicts an outcome, and to find the odds we aggregate this information. Turning back to machine learning algorithms like the basic perceptron, a computer can learn by the reinforcement of one of its processes over time. This leads to specific pathway of routes that causes an outcome to be chosen more often, and other nodes that make up different routes to be ignored.
One of the major issues with this, is that the inputs we give the computer are data lacking knowledge of why that data exists, similar to assumptions in betting. This means that the big data that we give the machine may manifest what we already believe to be true. In his Ted talk, Kenneth Cukier, gives the example that if we look at pie sales, we might assume that the favored pie in the U.S. is apple, however, if we look at pies made for individual consumption, apple is not the most purchased. This is how we know that apple pie is actually not favored by individuals, but it is a safe bet for large groups or families. Data can be very misleading, and when it comes to machines, we run into a problem when we accept outputs with proper knowledge or control over why we got it.
Just as the wisdom of the crowds can lead to false conceptions of intelligence, big data can lead to false conclusions.