#5: No Limit Royal Hold’em | Reading
No Limit Royal Hold’em
This game is the same as regular No Limit Texas Hold’em, except it uses a deck size of 20 cards instead of 52 and antes instead of blinds. The deck consists of all cards Ten and higher (T, J, Q, K, A) and all suits.
The variation we will use has each player starting with 20 chips and blinds of 1 and 2. We will have 2 betting rounds: preflop and the flop.
Bet sizes that are legal are:
Min: \(\min(1, \text{previous bet or raise size})\)
Max: \(\text{current stack size}\)
Odds of Hands
Hand ranks are based on the probability of each type of hand occurring. Note that since we are making 5-card hands, we always use the 5-card probabilities. In Texas Hold’em, with 7 cards, you are more likely to have a pair (43.8%) than high-card (17.4%), but we use the 5-card probabilities of pair (42.2%) and high-card (50.1%) when determining hand ranks.
For Royal Hold’em, the total number of deals is: \({20 \choose 5} = \frac{20!}{15!*5!} = \frac{20*19*18*17*16}{5*4*3*2*1} = 15504\)
Now we compute the frequency of every hand ranking and divide by the total number of deals to get the probability. Note that there is only one kind of flush, which is the Royal Flush from Ten to Ace. Also that it is not possible to have only high-card.
| Hand | Frequency Calculation | Frequency | Probability | Cumulative Probability |
|---|---|---|---|---|
| Royal Flush | \({4 \choose 1}\) | \(4\) | \(0.000258\) | \(0.000258\) |
| Four of a Kind | \({5 \choose 1}{4 \choose 4}{4 \choose 1}{4 \choose 1}\) | \(80\) | \(0.00516\) | \(0.005418\) |
| Full House | \({5 \choose 1}{4 \choose 3}{4 \choose 1}{4 \choose 2}\) | \(480\) | \(0.03096\) | \(0.036378\) |
| Straight (excluding Royal Flushes) | \({4 \choose 1}^5 - {4 \choose 1}\) | \(1020\) | \(0.065789\) | \(0.102167\) |
| Three of a Kind | \({5 \choose 1}{4 \choose 3}{4 \choose 2}{4 \choose 1}^2\) | \(1920\) | \(0.123839\) | \(0.226006\) |
| Two Pair | \({5 \choose 2}{4 \choose 2}^2{3 \choose 1}{4 \choose 1}\) | \(4320\) | \(0.278638\) | \(0.504644\) |
| One Pair | \({5 \choose 1}{4 \choose 2}{4 \choose 3}{4 \choose 1}^3\) | \(7680\) | \(0.495356\) | \(1\) |
The rankings work out to be the same as in regular Texas Hold’em, except that there are no high-card hands and no regular flush hands.
Game Sizes
Sourced from Michael Johanson’s 2013 paper Measuring the Size of Large No-Limit Poker Games
The size of a game is a simple heuristic that can be used to describe its complexity and compare it to other games, and a game’s size can be measured in several ways.
The most commonly used measurement is to count the number of game states in a game: the number of possible sequences of actions by the players or by chance, as viewed by a third party that observes all of the players’ actions.
In the poker setting, this would include all of the ways that the players’ private and public cards can be dealt and all of the possible betting sequences.
This number allows us to compare a game against other games such as chess or backgammon, which have \(10^{47}\) and \(10^{20}\) distinct game states respectively (not including transpositions).
The card frequencies are the same for limit and no limit variations of Hold’em. These can be calculated separately from the betting sequences.
Limit Hold’em game sizes are easier to measure because there is a fixed maximum size of bets (usually 4 per round) that have no dependence on previous rounds/actions. The infosets and infoset-actions are the most relevant size values since each infoset and action has two tabular values, one for accumulated regret and one for accumulating average strategy. The game size for Limit Hold’em is \(3.589 \times 10^{13}\) infoset-actions.
However No Limit Hold’em game sizes are more complex and depend on three factors:
- Amount of money remaining
- Size of the bet facing
- If a check is legal
Within one betting round, any two decision points that are identical in these three factors will have the same legal actions and the same betting subtrees for the remainder of the game, regardless of other aspects of their history.
Second,each of these factors only increases or decreases during a round. A player’s stack size only decreases as they make bets or call an opponent’s bets. The bet being faced is zero at the start of a round (or if the opponent has checked), and can only remain the same or increase during a round.
Finally, a check is only allowed as the first action of a round.
These observations mean that we do not have to walk the entire game in order to count the decision points. Instead of considering each betting history independently, we will instead consider the relatively small number of possible configurations of round, stack-size, bet-faced, and check-allowed, and do so one round at a time, starting from the start of the game.
Royal No Limit Hold’em was suggested as a testbed game since it is small enough to be tractable on consumer-grade computers and resembles regular No Limit Texas Hold’em. The relatively small size means that unabstracted equilibrium and best-response computations would be tractable and therefore abstractions could be evaluated to motivate research that translates to larger domains.
Abstraction in Poker
Abstraction refers to shrinking the game in some way to make it more tractable to solve.
The main ways to do this in poker are through card bucketing and bet size restrictions.
Card Abstraction
Card abstraction entails putting sets of cards into buckets. For example, in 100 card Kuhn Poker, we could have a bucket of cards for every 10 card ranks. This means that cards 1-10, 11-20, …, 91-100 are all in a separate bucket. A bucket would be represented by an infoset and so you would play any hand within a bucket the same way – only if there is a showdown is the true hand and outcome revealed.
There are more complex ways to bucket by hand strength in Texas Hold’em. The simplest bucketing method is preflop lossless abstraction such that hands can be reduced from \({52 \choose 2} = 1326\) to \(169\) starting hands.
Of the \(169\), there are:
- \(13\) pairs, one for each rank
- \(78\) suited hands
- \(78\) offsuit hands
The suited and offsuit hands are \(78\) each because there are \(13*12 = 156\) total types of unpaired hands. The only thing that matters when playing them preflop is whether they are suited or offsuit. We split this into \(78\) suited and \(78\) offsuit hands.
Bet Abstraction
Bet abstraction requires allowing only a limited number of bet sizes when solving the game. A limited number of bet sizes is important to reduce game size since in no limit games, players can generally bet any amount up to their stack size.
The abstracted actions could include things like:
- Minimum bet
- \(25\%\) pot
- \(50\%\) pot
- \(75\%\) pot
- \(100\%\) pot
- \(200\%\) pot
- Allin
This works for training, but then a problem arises when playing when the opponent makes a bet that does not match the abstracted bet sizes that were learned during training.
The agent has to translate the bet to understand how to respond.
In 2013, Ganzfried and Sandholm proposed a mapping as follows:
\(f_{A,B}(x) = \frac{(B-x)(1+A)}{(B-A)(1+x)}\)
Where \(x\) is the opponent bet and it is an element of \([A, B]\), where \(A\) is the largest betting size in the abstraction that is \(\leq x\) and \(B\) is the smallest betting size in the abstraction that is \(\geq x\), assuming \(0 \leq A \leq B\).
The result is the probability that we map the action to \(A\) and \((1-f_{A,B}(x))\) is the probability that we map the action to \(B\).