Luck vs. skill in poker

The thread of our recent discussion of quantifying luck vs. skill in sports turned to poker, motivating the present post.

1. Can good poker players really “read” my cards and figure out what’s in my hand?

For a couple years in grad school a group of us had a regular Thursday-night poker game, nickel-dime-quarter with a maximum bet of \$2, I believe it was. I did ok, it wasn’t hard to be a steady winner by just laying low most of the time and raising when I had a good hand. Since then I’ve played only very rarely (one time was a memorable experience with some journalists and a foul-mouthed old-school politico—I got out of that one a couple hundred dollars up but with no real desire to return), but I did have a friend who was really good. I played a couple of times with him and some others, and it was like the kind of thing you hear about: he seemed to be able to tell what cards I was holding. Don’t get me wrong here, I’m not saying that he was cheating or that it was uncanny or anything, and it’s not like he was taking my money every hand. As always in a limit game, the outcomes had a lot of randomness. But from time to time, it big hands, it really did seem like he was figuring me out. I didn’t think to ask him how he was doing it but I was impressed.

Upon recent reflection, though (many years later), it seems to me that I was slightly missing the point. The key is that my friend didn’t need to “read” me or know what I had; all he needed to do was make the right bets (or, to be more precise, make betting decisions that would perform well on average). He could well have made some educated guesses about my holdings based on my betting patterns (or even my “tells”) and used that updated probability distribution to make more effective betting decisions. The point is that, in many many settings, he doesn’t need to guess my cards; he just needs a reasonable probability distribution (which might be implicit). For example, in some particular situation in a particular hand, perhaps it would be wise for him to fold if he the probability is more than 30% that a particular hole card of mine is an ace. With no information, he’d assess this event as having an (approximate) 2% probability. So do I have that ace? He just needs to judge whether the probability is greater or less than 30%, an assessment that he can do using lots of information available to him. But once he makes that call, if he does it right (as he will, often enough; that’s part of what it means to be a good poker player), it’ll seem to me like he was reading my hand.

2. Some references on luck vs. skill in poker

Louis Raes pointed to three papers:

Ben van der Genugten and Peter Borm wrote quite a bit on Poker and the extent to which skill or luck is important. This work is mainly geared towards Dutch regulation but interesting nonetheless.

See:

3. Rick Schoenberg’s definition

Rick Schoenberg sent along an excerpt from his book, Probability with Texas Holdem Applications. Rick writes:

Surprisingly, a lot of books on game theory do not define the words “luck” or “skill”, maybe because it is very hard to do so. . . . in poker I [Rick] define skill as equity gained during the betting rounds and luck as equity gained during the deal of the cards. I then go through a televised twenty-something hand battle between Howard Lederer and Dario Minieri, two players with about as opposite styles as you can get, and try to quantify how much of Lederer’s win was due to luck and how much was due to skill.

I’ll go through Rick’s material and intersperse some comments.

Here’s section 4.4 of Rick’s book:

The determination of whether Texas Hold’em is primarily a game of luck or skill has recently become the subject of intense legal debate. Complicating things is the fact that the terms luck and skill are extremely difficult to define. Surprisingly, rigorous definitions of these terms appear to have eluded most books and journal articles on game theory. A few articles have defined skill in terms of the variance in results among different players, with the idea that players should perform more similarly if a game is mostly based on luck, but their results might differ more substantially if the game is based on skill. Another definition of skill is the extent to which players can improve, and it has been shown that poker does indeed possess a significant amount of this feature (e.g. Dedonno and Detterman, 2008). Others have defined skill in terms of the variation in a given player’s results, since less variation would indicate that fewer repetitions are necessary in order to determine the statistical significance of one’s long-term edge in the game, and hence the sooner one can establish that one’s average profits or losses are primarily due to skill rather than short term luck.

These definitions above are obviously extremely problematic for  various reasons. One is that they rely on the game in  question being played repeatedly before even a crude assessment of  luck or skill could be made. More importantly, there are many contests  of skill wherein the differences between players are small, or where one’s results vary wildly. For instance, in Olympic trials of 100-meter sprints, the differences between finishers are typically quite  small, often just hundredths of a second. This hardly implies that the  results are based on luck. There are also sporting events where an  individual’s results might vary widely from one day to another, e.g. pitching in baseball, but that hardly means that luck plays a major  role.

Regarding the quantification of the amount of luck or skill in a particular game of poker, a possibility might be to define luck as equity  gained when cards are dealt by the dealer, and skill as equity gained  by one’s actions during the betting rounds. (Recall that equity was defined in Section 4.3 as the product cp.) That is, there are several  reasons you might gain equity during a hand:
* The cards dealt by the dealer (whether the players’ hole cards or  the flop, turn, or river) give you a greater chance of winning the  hand in a showdown,
* The size of the pot is increased while your chance to win the hand  in a showdown is better than that of your opponent(s).
* By betting, you get others to fold and thus win a pot that you might otherwise have lost.
Certainly, anyone would characterize the first case as luck, unless  perhaps one believes in ESP or time travel.

Uh oh. Daryl Bem’s getting off the bus right here, I think!

Thus [Rick continues], a possible way to  estimate the skill in poker can be obtained by looking at the  second and third cases above. That is, we may view one’s skill as  being comprised of the equity that one gains during the betting  rounds, whereas luck is equity gained by the deal of the cards. The  nice thing about this is that it is easily quantifiable, and one may  dissect a particular poker game and analyze how much equity each  player gained due to luck or skill.

There are obvious objections to this. First, why equity? One’s equity (which is sometimes called express equity) in a pot is defined as one’s expected  return from the pot given no future betting, and the assumption of no  future betting may seem absurdly simplistic and unrealistic. On the other  hand, unlike implied equity which would account for betting on future betting rounds, express equity is unambiguously defined and easy to compute.  Second, situations can occur where one would expect a terrible player  to gain equity during the betting rounds against even the greatest  player in the world, so to attribute such equity gains to skill  might be objectionable. For instance, in heads up Texas Hold’em, if the two players are dealt AA and KK, one would expect the player with KK to  put a great deal of chips in while way behind, and this situation  seems more like bad luck for the player with KK than any deficit in  skill. One possible response to this objection is that skill is difficult to define, and in fact most poker players, probably due to  their huge and fragile egos, tend to chalk up losses for virtually any  reason as being solely due to bad luck. In some sense, anything can be  attributed to luck if one has a general enough definition of the word.  Even if a player does an amazingly skillful poker play, such as  folding a very strong hand because of an observed tell or betting  pattern, one could argue that it was lucky that the player happened  to observe that tell, or even that the player was lucky to have been  born with the ability to discern the tell. On the other hand, situations like the AA versus KK example truly do seem like bad luck.  It is difficult to think of any remedy to this problem. It may be that  the word skill is too strong a word, and that while it may be of  interest to analyze hands in terms of equity, one should instead use  the term equity gained during the betting rounds rather than skill in what follows.

We should distinguish between the two concerns that Rick is noting here. First, the division between luck and skill is inherently arbitrary (in a similar way to the arbitrariness of the division between prior and likelihood in a hierarchical model). To take things to extremes, you could say that a skillful player is lucky to have been born with such skill. As Rick explains, some things are definitely luck, but nothing can really be defined as definitely being skill.

Rick’s second concern, which I share, is that in his example it does not seem like skill that, if you happen to get the pair of aces, your equity just keeps going up because the other player doesn’t know what you have. So I agree with him that his definition has real problems.

Rick continues:

Below is an extended example intended to illustrate the division of luck and skill in a given game of Texas Hold’em. The example involves the end of a tournament on Poker After Dark televised during the first week of October 2009. Dario Minieri and Howard Lederer were the final two players. Since this portion of the tournament involves only these two players, and since all hands (or virtually all) were televised, this example provides us with an opportunity to attempt to parse out how much of Lederer’s win was due to skill and how much was due to luck.

Technical side note: Before we begin, we need to clarify a couple of potential ambiguities. There is some ambiguity in the definition of equity before the flop,  since the small and big blind have put in different amounts of chips.  The definition used here is that preflop equity is the expected  profit (equity one would have in the pot after calling minus cost),  assuming the big blind and small blind call as well, or the equity one  would have by folding, whichever happens to be greater. For example,  in heads up Texas Hold’em with blinds of 800 and 1600, the preflop equity  for the big blind is 2bp – 1600, and max{2bp – 1600, -800} for the  small blind, where p is the probability of winning the pot in a  showdown, and b is the amount of the big blind. Define increases in  the size of the pot as relative to the big blind, i.e. increasing the  pot size by calling preflop does not count as skill. The probability p of winning the hand in a showdown was obtained using the odds calculator at cardplayer.com, and  the probability of a tie is divided equally among the two players in determining p.

Example 4.4.1. Below are summaries of all 27 hands shown on Poker After Dark in October 2009 between  Dario Minieri and Howard Lederer in the Heads Up portion of the  tournament, with each hand’s equity gains and losses broken down as  luck or skill. Each hand is analyzed from Minieri’s perspective, i.e. “skill -100” refers to 100 chips of equity  gained by Lederer during a betting round. The question we seek to address is: how much of Lederer’s win was due to skill, and how much of it was due to luck?

Based on the concerns discussed above, “skill” doesn’t seem to be the right word here. On the other hand, maybe this is no big deal because the “luck” aspects will ultimately average out.

For example [Rick continues], here is a detailed breakdown of hand 4, where the blinds were 800/1600, Minieri was dealt A♣ J♣, Lederer had A♥ 9♥, Minieri raised to 4300 and Lederer called. The flop was 6♣ 10♠ 10♣, Lederer checked, Minieri bet 6500, and Lederer folded.

a) Preflop dealing (luck): Minieri +642.08.  Minieri was dealt a 70.065% probability of winning the pot in a showdown, so Minieri’s  increase in equity is 70.065 x 3200 – 1600 = 642.08 chips.  Lederer was dealt a 29.935% probability to win the pot in a showdown, so his increase in equity is 29.935% * 3200  - 1600 = -642.08 chips.

b) Preflop betting (skill): Minieri +1083.51.  The pot was increased to 8600. 8600-3200=5400.  Minieri paid an additional 2700 but had 70.065% x 5400 = 3783.51  additional equity, so Minieri’s expected profit due to betting was 3783.51 –  2700 = 1083.51 chips.  Correspondingly, Lederer’s expected profit due to betting was -1083.51 chips, since  29.935% x 5400 – 2700 = -1083.51.

c) Flop dealing (luck): Minieri +1362.67.  After the flop was dealt, Minieri’s probability of winning the 8600 chip pot in a showdown went from 70.065% to 85.91%. So by luck, Minieri increased his equity by (85.91% –  70.065%) x 8600 = +1362.67 chips.

d) Flop betting (skill): Minieri +1211.74.  Because of the betting on the flop, Minieri’s equity went from 85.91%  of the 8600 pot to 100% of the pot, so Minieri increased his equity by  (100% – 85.91%)x8600 = 1211.74 chips.
So during the hand, by luck, Minieri increased his equity by 642.08 + 1362.67 = 2004.75 chips.  By skill, Minieri increased his equity by 1083.51 + 1211.74 = 2295.25 chips.  Notice that the total = 2004.75 + 2295.25 = 4300, which is the number  of chips Minieri won from Lederer in the hand.

Note that [Never use the expression “note that”! Also avoid “very” and “obviously” — ed.] before the heads-up battle began, the broadcast reported that Minieri had 72,000 chips, and  Lederer 48,000. Minieri must have won some chips in hands they did not televise, because the grand  total has Minieri losing about 74,500 chips.

(Blinds 800 and 1600.)

Hand 1. Lederer A♣ 7♠, Minieri 6♠ 6•. Lederer 43.535%, Minieri 56.465%.  Lederer raises to 4300. Minieri raises to 47800. Lederer folds.
Luck +206.88.  Skill +4093.12.

Hand 2. Minieri 4♠ 2•, Lederer K♠ 7♥. Minieri 34.36%, Lederer 65.64%.  Minieri raises to 4300, Lederer raises all in for 43500, Minieri folds.
Luck -500.48.  Skill -3799.52.

Hand 3. Lederer 6♥ 3•, Minieri A• 9♣. Lederer 34.965%, Minieri 65.035%.  Lederer folds in the small blind.
Luck +481.12.  Skill +318.88.

Hand 4. Minieri A♣ J♣, Lederer A♥ 9♥. Minieri 70.065%, Lederer 29.935%.  Minieri raises to 4300, Lederer calls 2700.  Flop 6♣ 10♠ 10♣. Minieri 85.91%, Lederer 14.09%.  Lederer checks, Minieri bets 6500, Lederer folds.
Luck +2004.75.  Skill +2295.25.

Hand 5. Lederer 5♠ 3♥, Minieri 7• 6♠. Lederer 35.765%, Minieri 64.235%.  Lederer folds in the small blind.
Luck +455.52.  Skill +344.48.

Hand 6. Minieri K♥ 10♣, Lederer 5¨ 2¨.  Minieri 61.41%, Lederer 38.59%  Minieri raises to 3200, Lederer raises to 9700, Minieri folds.
Luck +365.12.  Skill -3565.12

Hand 7. Minieri 10• 7♠, Lederer Q♣ 2♥.  Minieri 43.57%, Lederer 56.43%.  Minieri raises to 3200, Lederer calls 1600.  Flop 8♠ 2♠ Q♥.  Minieri 7.27%, Lederer 92.73%.  Lederer checks, Minieri bets 3200, Lederer calls.  Turn 4•.  Minieri 0%, Lederer 100%.  Lederer checks, Minieri bets 10,000, Lederer calls.  River A♥.  Lederer checks, Minieri checks.
Luck -205.76 – 2323.20 – 930.56 = -3459.52.
Skill -205.76 – 2734.72 – 10000 = -12940.48.

Hand 8. Lederer 7♣ 2•, Minieri 9♣ 4•.  Minieri 64.28%, Lederer 35.72%.  Lederer folds.
Luck +456.96.  Skill +343.04.

Hand 9. Minieri 4♠ 2♣, Lederer 8♥ 7•.  Minieri 34.345%, Lederer 65.655%.  Minieri raises to 3200, Lederer calls 1600.  Flop 3• 9♥ J♥.  Minieri 22.025%, Lederer 77.975%.  Lederer checks, Minieri bets 4800, Lederer folds.
Luck -500.96 – 788.48 = -1289.44.  Skill -500.96 + 4990.40 = +4489.44.

Hand 10. Lederer K♠ 5♠, Minieri K♥ 7♣.  Minieri 59.15%, Lederer 40.85%.  Lederer calls 800, Minieri raises to 6400, Lederer folds.
Luck +292.80.  Skill +1307.20.

Hand 11. Minieri A♥ 8♥, Lederer 6♥ 3♠.  Minieri 66.85%, Lederer 33.15%.  Minieri raises to 3200. Lederer folds.
Luck +539.20.  Skill +1060.80.

Hand 12. Lederer A• 4•. Minieri 7• 3♥.  Minieri 34.655%, Lederer 65.345%.  Lederer raises to 4300, Minieri raises to 11500, Lederer folds.
Luck -491.04.  Skill +4791.04.

Hand 13. Minieri 6♣ 3♣, Lederer K♠ 6♠.  Minieri 29.825%, Lederer 70.175%.  Minieri raises to 4800, Lederer calls 3200.  Flop 5♥ J♣ 5♣.  Minieri 47.425%, Lederer 52.575%.  Lederer checks, Minieri bets 6000, Lederer folds.
Luck -645.60 + 1689.60 = +1044.  Skill -1291.20 + 5047.20 = +3756.

Hand 14. Lederer 7• 5♠, Minieri 8• 5•.  Minieri 69.44%, Lederer 30.56%.  Lederer calls 800, Minieri checks.  Flop K♥ 10♠ 8♣.  Minieri 94.395%, Lederer 5.605%.  Minieri checks, Lederer bets 1800, Minieri calls.  Turn 7♠.  Minieri 95.45%, Lederer 4.55%.  Minieri checks, Lederer checks.  River 6♥.  Check, check.
Luck +622.08 + 798.56 + 71.74 + 309.40 = 1801.78.  Skill 0 + 1598.22 + 0 + 0 = 1598.22.
Blinds 1000/2000.

Hand 15. Minieri 9• 5♠, Lederer A♥ 5•.  Minieri 26.755%, Lederer 73.245%.  Minieri calls 1000, Lederer raises to 7000, Minieri raises to 14000,  Lederer calls 7000.  Flop 10♠ Q• 6♥.  Minieri 15.35%, Lederer 84.65%.  Lederer checks, Minieri bets 14000, Lederer folds.
Luck -929.80 – 3193.40 = -4123.20.  Skill -5578.80 + 23702 = 18123.20.

Hand 16. Lederer 5♠ 5♥, Minieri A♣ J•.  Minieri 46.085%, Lederer 53.915%.  Lederer calls 1000, Minieri raises to 26800, Lederer calls all in. The board is 3♠ 9♠ K♠ 10• 9•.
Luck -156.60 – 24701.56 = -24858.16.  Skill -1941.84.

Hand 17. Minieri K♣ 10♣, Lederer 7• 5•.  Minieri 62.22%, Lederer 37.78%.  Minieri raises to 5000, Lederer calls 3000.  Flop J♠ J• 4♠.  Minieri 69.90%, Lederer 30.10%.  Check check.  Turn 8♠.  Minieri 77.27%, Lederer 22.73%.  Lederer bets 6000, Minieri folds.
Luck +488.80 + 768 + 737 = 1993.80.  Skill +733.20 + 0 – 7727 = -6993.80.

Hand 18. Lederer 5♠ 5♣, Minieri 10♠ 6♥.  Minieri 46.12%, Lederer 53.88%.  Lederer calls 1000, Minieri checks.  Flop 7♣ 8♣ Q♥.  Minieri 38.235%, Lederer 61.765%.  Minieri checks, Lederer bets 2000, Minieri calls.  Turn J♥.  Minieri 22.73%, Lederer 77.27%.  Minieri bets 4000, Lederer folds.
Luck -155.20 – 315.40 – 1240.40 = -1711.  Skill 0 – 470.60 + 6181.60 = +5711.

Hand 19. Lederer K♥ 5♠, Minieri K♣ 10•.  Minieri 73.175%, Lederer 26.825%.  Lederer raises to 5000, Minieri calls 3000.  Flop J• 8♥ 10♥.  Minieri 92.575%, Lederer 7.425%.  Check, check.  Turn 5•.  Minieri 95.45%, Lederer 4.55%.  Minieri bets 6000, Lederer folds.
Luck +927 + 1940 + 287.50 = 3154.50.  Skill +1390.50 + 0 + 455 = 1845.50.

Hand 20. Minieri 7♣ 2♠, Lederer Q♠ 9♠.  Minieri 30.205%, Lederer 69.795%.  Minieri raises to 6000. Lederer calls 4000.  Flop A• A♠ Q•.  Minieri 1.165%, Lederer 98.835%.  Lederer checks, Minieri bets 6000, Lederer calls.  Turn J♣.  Minieri 0%, Lederer 100%.  Lederer checks, Minieri bets 14000, Lederer raises to 35800, Minieri  folds.  Luck -791.80 – 3484.80 – 279.60 = -4556.20.  Skill -1583.60 – 5860.20 – 14000 = -21443.80.

Hand 21. Minieri 10♥ 3•, Lederer Q♥ J♠.  Minieri 30.00%, Lederer 70.00%.  Minieri calls 1000, Lederer checks.  Flop 8♠ 4♥ J♣.  Minieri 4.34%, Lederer 95.66%.  Lederer checks, Minieri bets 2000, Lederer raises to 7500, Minieri  raises to 18500, Lederer raises all-in, Minieri folds.  Luck -800 – 1026.40 = -1826.40.  Skill 0 – 18673.60 = -18673.60.

Hand 22. Lederer A♠ 2•, Minieri 5♣ 3♥.  Minieri 42.345%, Lederer 57.655%.  Lederer calls 1000. Minieri checks.  Flop K♠ 10♣ 3♠.  Minieri 80.10%, Lederer 19.90%.  Check check.  Turn Q♠.  Minieri 65.91%, Lederer 34.09%.  Check, Lederer bets 2000, Minieri folds.
Luck -306.20 + 1510.20 – 567.60 = 636.40.  Skill 0 + 0 – 2636.40 = -2636.40.

(Blinds 1500/3000.)

Hand 23. Minieri 7♥ 7♣, Lederer 8• 3•.  Minieri 68.175%, Lederer 31.825%.  Minieri all-in for 21,700, Lederer folds.
Luck +1090.50.  Skill +1909.50.

Hand 24. Minieri Q♥ 5♥, Lederer 8• 5•.  Minieri 68.37%, Lederer 31.63%.  Minieri all-in for 26,200, Lederer folds.
Luck +1102.20.  Skill +1897.80.

Hand 25. Lederer 9♣ 3♣, Minieri 5• 2•.  Minieri 40.63%, Lederer 59.37%.  Lederer folds.
Luck -562.20.  Skill +2060.20.

Hand 26. Minieri 10♣ 2♠, Lederer 7♣ 7♥.  Minieri 29.04%, Lederer 70.96%.  Minieri folds.
Luck -1257.60.  Skill -242.40.

Hand 27. Lederer Q♣ 9♣, Minieri A♣ 5♠.  Minieri 55.37%, Lederer 44.63%.  Lederer all-in for 29,200. Minieri calls.  Board 7♣ 6♣ 10♠ Q♠ 6•.
Luck +322.20 – 32336.08 = -32013.88.  Skill +2813.88.

Grand Totals:  Luck -61023.59.  Skill -13478.41.

Overall, although Lederer’s gains were primarily (about 81.9%) due  to luck, Lederer also gained more equity due to skill than  Minieri. On the first 19 hands, Minieri actually gained 20,836.41 in equity due to skill,  and it appeared that Minieri was outplaying Lederer.  On hands 20 and 21, however, Minieri tried two huge unsuccessful bluffs, both  on hands (especially hand 20) where he should probably have strongly suspected  that Lederer would be likely to call, and on those two hands combined,  Minieri lost 40,117.40 in equity due to skill. Although Minieri played  very well on every other hand, all of Minieri’s good plays on other  hands could not overcome the huge loss of skill equity from just  those two hands.

It is important to note that the player who gains the most equity due to skill does not always win. In the first 19 hands of this example, for instance, Minieri gained 20836.41 in equity attributed to skill, but because of bad luck, Minieri actually lost a total of 2800 chips over these same 19 hands. The bad luck Minieri suffered on hand 16 negated most of his gains due to skillful play. A common misconception is that one’s luck will ultimately balance out, i.e. that one’s total good luck will eventually exactly equal one’s total bad luck, but this is not true. Assuming one plays the same game repeatedly and independently, and assuming the expected value of one’s equity due to luck is 0 which seems reasonable, then one’s average equity per hand gained by luck will ultimately converge to zero. This is the law of large numbers, and is discussed further in Section 7.4. It does not imply that one’s total equity gained by luck will converge to zero, however. Potential misconceptions about the laws of large numbers and arguments about possible overemphasis on equity are discussed in Section 7.4.

To conclude this Section, a nice [I’d avoid the word “nice” too! — AG] illustration of the potential pitfalls of analyzing a hand purely based on equity is a recent hand from Season 7 of High Stakes Poker. In this hand, with blinds of \$400 and \$800 plus \$100 antes from each of the 8 players, after Bill Klein straddled for \$1600, Phil Galfond raised to \$3500 with Q♠ 10♥, Robert Croak [now that’s a great poker name. — ed.] called in the big blind with A♣ J♣, Klein called with 10♠ 6♠, and the other players folded. The flop came J♠ 9♥ 2♠, giving Croak top pair, Klein a flush draw, and Galfond an open ended straight draw. Croak bet \$5500, Klein raised to \$17500, and Galfond and Croak called. At this point, it is tempting to compute Klein’s probability of winning the hand by computing the probability of exactly one more spade coming on the turn and river without making a full house for Croak, or the turn and river including two 6s, or a 10 and a 6. This would yield a probability of [(8 x 35 – 4 – 4) + C(3,2) + 2×3] ÷ C(43,2) = 281/903 ~ 31.12%, and Klein could also split the pot with a straight if the turn and river were KQ or Q8 or 78, without a spade, which has a probability of [3×3 + 3×3 + 3×3] ÷ C(43,2) = 27/903 ~ 2.99%. These seem to be the combinations Klein needs, and one would not expect Klein to win the pot with a random turn and river combination not on this list, and especially not if the turn and river contain a king and a jack with no spades. But look at what actually happened in the hand. The turn was the K♣, giving Galfond a straight, and Croak checked, Klein bet \$28000, Galfond raised to \$67000, Croak folded, and Klein called. The river was the J♥, Klein bluffed \$150000, and Galfond folded, giving Klein the \$348,200 pot!

P.S. Image above from Barney Townshend shows some playing cards that Schoenberg designed a few years earlier.

1. Jonathan (another one) says:

Full disclosure (which seems odd for someone not using his full name): I was hired by the gov’t to assess this question in a court case.

The real problem in luck vs. skill is that skill in a competition is clearly a relative concept. When I beat my seven year old grandson in poker time after time, that means I have a lot more skill than he does; the hands I lose are hands in which I was unlucky. By contrast, my skill level is unchanged if I ever enter the WSOP, but I’ll get killed.

This is why I disagree with 100 meter sprint example. I would say that if two identically trained sprinters meet, the winner will be determined by luck. That is true even though either one of them could run past everyone else on the planet. Luck is simply the sum of the factors we haven’t, or conceptually cannot, measure. In cards, the identity of hidden cards isn’t measurable, though better inferences about the distribution of those cards might be made from examination and analysis of betting behavior and tells. The inferences are skill, but they are only relative skill, and they nullify when faced with equally skillful players.

Thus, luck vs. skill in poker (or sprinting or golf, if the result you’re aiming for is winning, not scoring) is a function of who you’re playing with. So is online poker luck or skill (it doesn’t help that some of the legal cases revolve around which one “predominates,” which I point out also will depend on the length of the session. if I play one hand with the best poker player who ever lived, luck will predominate — I’ll go all in with whatever I have). In penny-stakes games, luck will predominate just because good poker players are unwilling to play at these stakes — the return is too low. In high stakes games, luck will predominate except among well-heeled players who don’t care much if they lose. Everyone wants to get into a game where they are one of the best two players in the game, a situation which obviously has no equilibrium.

This then leads to the final conundrum: is it a skill to figure out which game you should play in and stick with it? If it is,then poker is almost all skill, except at the very highest levels, where it is almost all luck. If picking your stakes/game is not a skill (but what is it? luck?) then almost everything is luck.

• Zach says:

Great points. (I completely agree about the sprinter example.)

• jonathan says:

Having sprinted, my “assertion” is that each sprinter is a bundle of attributes that work themselves out over the course of a race. At the end of 100 or 200 meters, all that matters is the time but you get there by starting, then the first section, then the second, then the third and each of these involves a series of actions that require focus and luck. You can evaluate how good this guy is by his average times not by how he does in any race. We see examples of this all the time (though commentators rarely talk about it): a guy who starts very fast may be tying up his muscles a bit more or coming up to vertical too early and so it looks like the other guy blows by when it’s more that the other guy was able in this particular race to get his body moving freely through the race stages. The fast starter may have had “better skill” at the moment of start and in the brief period after but then “less skill” at an important transition point in the race.

People think sprinting is about extreme effort but it isn’t at all; it’s about getting your body moving as fast as it can, which requires immense freedom of movement from every constituent part. You can feel each little off part as you move through the race stages. That’s immensely hard to repeat time after time. A guy like Bolt is simply faster than everyone else right now (on average) but that’s mostly mechanical advantage: his longer legs with his quick turnover eat ground. BTW, that kind of turnover in a guy that size used to be considered a dead giveaway for chemical enhancement, but …

I also want to add, somewhat unrelated to sprinting, that luck extends over long periods. Take BABIP, batting average for balls in play. Look at how it varies from season to season so this player seems to be such a terrific hitter but then his BABIP regresses and he’s seen as not living up to the potential shown in that year. Or take turnover ratios in football. It’s not uncommon for a team to suddenly be a contender and that very often ties to its extreme success in turnover ratio. That matters because NFL games are decided by less than a TD on average. Next year, regression and … The point is luck can last for a season, so there’s no way to know that a series of hands in a particular tournament “evened out”. A player can be lucky for an entire tournament.

2. Nick Menzies says:

You use the division between prior and likelihood as the statistical analogy. It seem to me that the natural analogue for skill is the systematic part of a regression equation, with luck being the error term. A lot of the insights seem to translate pretty directly.

In the first long except from Rick Schoenberg’s book, he seems to be confusing the problem of estimating the luck:skill ratio with the problem of definitions.

3. Fernando says:

I would say luck is having a string of very good hands dealt during a game. Like rolling a six in a die time and again.

Skill is the systematic component of winnings holding luck constant. Presumably you can etimate this by having players play against a computer or a scripted confederate where the order in which cards are dealt is known in advance, so they are all playing the same game.

But I know nothing about poker.

4. TexMex says:

An alternative approach would be to ask: how many games would it take to distinguish an advanced beginner from an expert? In the case of chess, obviously one will be enough. In the case of (heads-up) poker, you would need quite a few so that the expert attains, say, a minimum of 60% win rate. Even in the case of backgammon, which arguably has a higher skill component than poker, a novice with a simple strategy can win many games due to luck before the skill of the expert starts to dominate.
Similarly, how many games of poker are needed to establish who from a pair of expert players has the edge with a reasonable degree of confidence? Thousands?
As Jonathan notes, for players at the same level (stakes), poker turns into a game of luck.

• Jonathan (another one) says:

That’s one of the measures I proposed. When we look at the average online poker session, we find that the mean session is under an hour, with a standard deviation of about 40 minutes. By no measure is this large enough to overcome even fairly large skill differences. (Poker players online play a little bit faster than a hand a minute… 70 or so per hour.)

• jonathan says:

I remember Omar Sharif talking about his poker playing for living. I wish I remembered better what he said because he was eloquent on the subject. I think one point was that you had to treat it as a job, that you had to put in hours, and you had to lose because players don’t play if they don’t see an opportunity to win.

5. […] The eternal debate: luck vs. skill; poker, in this case…(Andrew Gelman) […]

6. Anonymous says:

The assignment of skill to a player (athlete, investor, or whoever) can be thought of as a prediction of their future ability to win. That’s the only reason anyone cares whether it’s skill or luck anyway.

7. Todd says:

In what way do you suppose poker is different than any other card game?

I would argue Bridge has many of the same features of poker — there is a vast skill difference between novices and experts, but over the course of a single game, a string of particularly good hands will obscure that fact.

However, bridge has a system for assessing skill (http://en.wikipedia.org/wiki/Duplicate_bridge) that makes it very clear, again and again, that the same players come out on top because they have greater skill.

Perhaps poker needs a duplicate system in order to convince courts it’s a game of skill?

8. Daniel Vainsencher says:

Never use: “Never use the expression X” without explanation!
Correcting behavior without motivation only helps the fakers. Giving the advice and the motivation is much harder, but expands a culture, instead. You may now argue with the reasons I’ve given, but that expands a culture too…

Aside from which, Rick’s book certainly seems to be designed to appeal to people interested in poker who happen to be willing to read equations and perform calculations, rather than to people interested in probability. In which case, being reassuring and relaxing to read (what word is nicer than “nice”?) is perhaps more valuable than… whatever you are actually advocating (you didn’t explain) when you propose not to use nice, note that, etc.

• B D McCullough says:

Rick’s book is not primarily for poker players. I taught out of Rick’s book last year, and will do so again this year. I reviewed it for TAS. From my review:

This book should be judged by its title, so let me concur with the author’s
asssertion in the preface that this is not a poker book. It is a probability text,
and most of the examples just happen to be based on hold’em poker—it would
not take the place of, say, Ross (2006). This text can be used as a low-level
probability text, of the sort that might be used in a class that satisfies a general
education requirement. However, unlike the class of Physics for Poets textbooks
by which liberal arts majors satisfy a science requirement without taking a real
science course, this book is sufficiently rigorous that it cannot be used for
innumerate students who want to satisfy a math requirement without taking a
real math course.

9. Charles says:

Luck versus skill: I had thought that the distinguishing feature is that a skill is something that you get better at through practice. A statement like “the player was lucky to have been  born with the ability to discern the tell” seems very strange and confused; I’ve never met a baby that plays poker.

“In his example it does not seem like skill that, if you happen to get the pair of aces, your equity just keeps going up because the other player doesn’t know what you have.” Surely it is skill to bet in such a way that (a) the other player loses the maximum amount of money in expectation, while (b) not realizing from your large bets that you have a much better hand than theirs?

I also found the following (perhaps intentional) irony amusing: “it wasn’t hard to be a steady winner by just laying low most of the time and raising when I had a good hand” and “I did have a friend who was really good … he seemed to be able to tell what cards I was holding.” Well, no kidding. Every time you raised, you had a good hand!

Not that I was ever any better at poker. But it has been my impression that an important part of poker skill is understanding the information that other players are attempting to obtain from your bets, and then manipulating this information to your benefit.

• Andrew says:

Charles:

You can be amused all you want but the game in which I was a steady winner was not the same as the game where I had the friend who could tell what cards I was holding. These were two different poker games with two different groups of people. In the first game I kept using that simple strategy because it worked. In the second game I was trying to play better but it didn’t seem to matter what I did. Of course I understood the principle that it’s necessary to mix things up so as to minimize the information conveyed to the other players. The difficulty was that this didn’t seem to work.

10. […] zanimiv blogerski zapis Andrewa Gelmana za vse spletne in reallife pokeraše: http://andrewgelman.com/2014/08/14/luck-vs-skill-poker/#more-23431. Tisti, ki vas zadeva zanima v bolj poglobljenih statistično-znanstvenih podrobnostih, pa najdete […]

11. AK says:

In addition to Schoenberg’s self-acknowledged limitations in his definition of “skill”, it seems like skill should somehow incorporate not only the betting decision a player makes, but also the player’s thought process behind making the decision. For example, take Hand 1. Minieri re-raises to take the pot and increase his stack by \$4,300, thereby “gaining \$4,093.12 in chips due to skill” by the methodology. Anyone could have made this decision to re-raise if put in that situation, from an amateur to a seasoned pro. However, if one player in that situation would re-raise with the blind hope of winning the pot, while another player would re-raise because he estimated that Lederer folds most of the time if re-raised in that situation (knows Lederer’s playing style/history, assumed he had a low pair, recognized a tell, etc.) have both players really won the pot due mainly to skill? Sure, in both cases the bet caused Lederer to fold, but were there equal amounts of skill behind each decision to re-raise? It seems like more of a won pot is due to skill when there is some sort of rationale behind a betting decision that wins a pot than when the betting decision is made with less reasoning behind it.

How to quantify this? I’m not sure how to do this really, but my first thought in this hand is what if after reviewing Lederer’s playing history, Minieri knows that when Lederer has pocket 6s (or more generally, a mid/low pair) he folds after being re-raised by 10 times his bet pre-flop 80% of the time? If Minieri knows this, assumes Lederer is holding a mid/low pair, then re-raises to take the pot, I’d say more of his chip gain is due to skill, compared to a less experienced player re-raising with little reasoning behind it. To take this a step further, if Minieri normally would fold or only call in this situation, but instead re-raises based on his knowledge of Lederer’s play, I’d argue EVEN MORE of the chip gain is due to skill, because Minieri went against his tendencies to make the correct betting decision. In short, is there a way to incorporate each player’s tendencies into the equation to better separate luck and skill?

12. Hi, thanks for the comments. I like the idea of skill being estimated via how long it would take to discern an expert from a beginner, based purely on results. That’s kind of interesting. I also just wanted to say I wrote my book specifically for me to teach probability courses with it, and have now used it for undergrad probability courses, both upper and lower division, about 5 or 6 times. Incidentally, the book did replace Ross (2006) in both instances. The students complain about a lot of things, but I’ve never had them complain about the book I don’t think. There are a lot of errors in the book, many of them annoyingly added by the copy editor. I located an estimated 1500 of them, but many still remain. It’s hard to find all 1500 of anything. For example, you can imagine in an intro probability book, both capital sigma and lower case sigma come up a lot, for sums and standard deviations, respectively. Well, the copy editor decided to change all the lower case sigmas to capital sigmas. I had to go through line by line and change them back, along with so many other errors, and you can’t just do “find and change” for something like that. It was pretty painful. For a list of the remaining errors see http://www.stat.ucla.edu/~frederic/errors.html .

On the subject of errors, one thing I tried to say as politely as I could in my book on p146 is that there’s a huge error in Chen and Ankenman’s “The Mathematics of Poker” which renders all the calculations and formulas in the whole last chapter wrong or meaningless or both. I’ve never received a single ounce of feedback about this though, probably because only like 2 people have ever read my whole book.

Yours,
Rick

• Jerrod Ankenman says:

Well, I haven’t read your book, but I’d be happy to know what you think is a “huge” error that invalidates “the whole last chapter” that no one has uncovered so far. (Also, the last chapter of our book contains no calculations–perhaps you meant the chapter preceding the error?). If you contacted one of us about it in the past, it’s possible that we overlooked your communication, although I do try to respond to criticism or possible errors when I can. I’m easy to reach; firstname.lastname@yale.edu will work for a couple more months.

13. Nameless says:

I suspect that, if you apply Rick’s model above to a sufficient number of games, you’ll find that the estimate of “skill” is going to be strongly correlated with the estimate of “luck”. The way it is defined, it is mainly the leak-through of luck, as alluded to by the AA vs KK example.

We need to look closer into what’s happening in the game. It’s not hard to see that “skill” has a few different aspects.

* First, there is the ability to make correct plays based purely on your cards and table cards. You can evaluate that using the approach similar to Rick’s above, but with two important changes. Instead of calculating expected equity using all cards of all players, calculate it for each player independently assuming a random hand in his opponent’s hands. Call it “blind equity”. We may need to define some sort of “aggressiveness curve” to reflect the fact that, say, a player still holding cards during river has a stronger-than-average hand, and incorporate it into move evaluations.

14. George says:

There’s a lot of statistical flair in all these assessments. But rather than trying to measure each players’ psyche and differing strategies for each hand, wouldn’t it be better — and much clearer — to just track players’ bankrolls over time? And the variations to those bankrolls?

\$\$\$ is the perfect index here to measure and settle this debate in poker — skill vs. luck — as a whole, no?

15. Hi!
I came here by accident and wanted to say that I liked your post.
Also, are the two Schoenbergs (Arnold & Rick) somehow related?

16. […] In comments, Rick Schoenberg wrote: […]

17. SanguineEmpiricist says:

Mr Gerlman, this ‘debate’ is largely solved. There is no reason to consult papers when we have probably over billions of hands played in online poker games, with consistent winners over a long set of time. If you have any further questions I can point you to all the relevant resources.