Programs Playing Poker: Pluribus and Game Theory
https://science.sciencemag.org/content/365/6456/885
Game theory lends itself nicely to games such as rock-paper-scissors, simple attack-defense games, and certain problem set questions from Cornell University. Algorithms already exist that can guarantee convergence on Nash equilibrium for any two-player zero-sum game (CFR being the one that will be touched upon later), so one can easily see how applicable this facet of game theory can be in practical application. However, the significance and strategy involving hidden information in poker has made it an immensely difficult game for artificial intelligence to model. While poker has forms of many differing complexities, this article details Pluribus, an AI that specializes in six player no-limit Texas hold’em poker. While Nash equilibrium isn’t applicable at every step of the AI, game theory still proves essential to its success.
In games such as checkers, a zero-sum game, AI’s have achieved great success against human players by approximating Nash equilibria strategies. Of course, compared to computers, humans are limited computationally, so have to often defer to other strategies. A more human approach to a game is to evaluate the weaknesses of an opponent and, combined with judgement and a bit of luck, use this to succeed. It would thus make sense that a successful artificial AI for poker might both approximate Nash equilibria strategies and evaluate weaknesses of opponents. However, such a solution would involve algorithms that have proven to be elusive. Thankfully, AI’s have still managed to prevail.
Pluribus employs a fixed strategy so the tendencies of opponents are, interestingly enough, entirely ignored. Due to the difficulty of building an efficient algorithm for finding Nash equilibria, Pluribus builds its strategy by playing against copies against itself continually, iterating upon its own knowledge. With Pluribus, an algorithm known as counterfactual regret (CFR) minimization guarantees that a Nash equilibrium strategy is eventually converged upon as the average strategy after over time.
With these aforementioned strategies and components, Pluribus has managed to defeat professionals playing for monetary prizes of thousands of dollars. Pluribus demonstrates both the incredible power and the limitations of Nash equilibria. While Nash equilibria prove essential to the AI, there’s also much more that makes Pluribus successful. Programs playing poker are fine and all, but it’s the implications of the existence of Pluribus that suggest a bright, game theory fueled future for AI.