How good are the best poker players?

Last post, I mentioned a study by Steven Levitt and Thomas Miles that argued for skill in poker.  A couple of people later sent me ungated links to that study (.pdf) ... when I read it, I was surprised how small the differences seem to be between players.

Levitt and Miles (call them LM) found that more skilled players (as judged by various sets of poker writers) made a 30%+ return in tournaments, while non-ranked players lost money, to the tune of negative 15 percent.  

But, looking more closely at the study, that seems to be a very small difference. As it turns out, the skilled players were only between 4 and 29 percent more likely than the average player to make the final table.

That seems tiny.  In team sports, if you pick the best team at the beginning of the year, you'd think they've got to be at least twice as likely to make the finals, right?  In baseball, a random team has a 1 in 15 chance to make the World Series.  A *good* team, on the other hand ... well, you'd think it would be 1 in 5 or something, right?  Three times as high?

And in *individual* sports ... look at golf, say.  If there are 100 players in a typical PGA tournament, a random player has a 10 percent chance of finishing in the top ten.  In his career, Tiger Woods finished in the top ten around 63 percent of the time.  That's not 29 percent -- it's 6,200 percent.

(It's not that great an analogy, of course, but even if I tried to make the comparison more reasonable, the difference would still be huge.)

------

Also ... I'm not completely sure that the differences in the LM study are statistically significant.  In fact, the study doesn't even talk about statistical significance, which is kind of strange, for an academic study.

And it's not easy to figure out, because the authors don't give us enough information.  What we'd want to see is, the mean and SD of profit per tournament for the skilled players, and the man and SD of profit for the rest.  We don't get that.  Instead, we get the mean and SD *per player*, rather than per player-tournament.  That doesn't help, because some players enter ten times as many tournaments than others, so we can't separate talent from opportunity.

Also, we get the mean and SD for buy-in (entry fee) separately from the mean and SD for winnings, so we don't even know the mean and SD for profit or for return on investment!

I find that kind of strange.

-------

The authors' data show there was an average 1,152 players per each of the 56 tournaments (using Table 3, which leaves out the higher-stakes "Main Event").  Guessing that each final table is 9 players, that's 504 overall seats at the final tables combined.  Skilled players were 12.6 percent of the entries, so you'd expect them to account for about 64 of those seats.  

If they were 20 percent more likely to get there, that's only an extra 13 seats.

So, what LM found is that skilled poker players take a seat from less-skilled poker players one out of every four tournaments, roughly.

That doesn't seem like a lot, does it?

-------

Is there really that much luck?  In the comments to the last post, Brian Burke wrote,

"I think once a player has learned the basics, he's par with even the best players in terms of skill. Once all players are equal in terms of skill, luck dominates the outcome. Even if players differed significantly in terms of skill, it might take thousands of hands to determine who is the better player.

"I think of poker like a complex version of rock, paper, scissors. Strictly speaking, RPS a game of skill. The skill is this: don't be predictable. Once everyone realizes the basic skill, now it's effectively a game of pure luck."

The evidence so far supports that, at least a little bit.  It could certainly be that the small effects LM found are just the (equally-skilled) good players taking turns beating up on the crappy ones.

--------

Or, maybe the sample the authors used -- the 2010 World Series of Poker (WSOP) -- was an outlier, or too small a sample.  Here's some evidence, perhaps, that suggests that might be the case.

The "Main Event" of the WSOP is the biggest one.  There's a $10,000 buy-in, and, this year, there were 6,352 entries.

In July, they played enough poker to eliminate 6,343 of those entries, leaving only nine.  Those nine players will play their final table in November.

Right now, you can place a bet on which of those players will win.  If they were close to equal in skill, the odds would be almost the same.  Are they?

Well, it's hard to tell ... the players all enter the final round with different amounts of chips.  The higher bankrolls have better odds.  Here are the odds I found, with the bankrolls in brackets:

J.C. Tran (38,000,000)            9/5  
Amir Lehavot (29,700,000)         9/2  
Marc McLaughlin (26,525,000)      5/1  
Jay Farber (25,975,000)           7/1  
Ryan Riess (25,875,000)           6/1  
Sylvain Loosli (19,600,000)       8/1  
Michiel Brummelhuis (11,275,000) 13/1  
Mark Newhouse (7,350,000)        15/1  
David Benefield (6,375,000)      15/1  

I don't know how to convert chip counts into odds, so most of this data doesn't help me.  But there's one exception: Ryan Riess is favored over Jay Farber, 6/1 vs. 7/1, even though Farber has (a tiny bit) more chips.  

Does that suggest Riess is significantly better?  Maybe.  Or, it could be that the bookmaker expects more money on Riess.  Or, it could have something to do with seating position (I don't know much about poker).  But ... it's something.

If you look at the overall odds, the implied probabilties add up to about 128%, which means you'd have to bet $128 to win $100 (the difference is the bookmaker's take).  That's pretty high, so even if the Farber/Reiss difference isn't justified, it's still not an exploitable profit opportunity.

What if we look at the pre-tournament odds instead?  I found this page, which shows odds on the top 20 players.  Here are 1-3 and 18-20:

Daniel Negreanu (CAN)   40/1
Phil Ivey (USA)         50/1
Phil Hellmuth (USA)     50/1
...
Patrik Antonius (FIN)  100/1
Pius Heinz (GER)       100/1
Russell Thomas (USA)   100/1

Those odds are much, much higher than the LM study would suggest.  100/1 suggests these players are at least 6,000% more likely to win, rather than 20% as implied by the study.  (Actually, the 20% was for the final table.  Generously assuming that the top final-table player has twice the average chance of winning, that still only makes a 40% increased chance of being the overall champion.  Forty is still less than six thousand.)

------

Ah!  After I wrote that, I found something much better.  Here are a bookie's odds on Daniel Negreanu winning at least one of the WSOP's 57 tournaments:

Yes: 11/10
No:   8/13

If the average tournament has 1100 players, and Negreanu enters all 57 of them -- with only one buy-in for each -- he'd have roughly a 57/1100 chance of a win if he were average.  If his chance is 40 percent above average, that goes up to roughly 80/1100, or 13:1 against.

But, here, you can lay 13/8 instead of 13/1, which suggests that the best players are *eight times* more likely to win than average.

Did I do that right?

-------

So, what's happening with the original study?   It could be that the poker analysis industry isn't very good at figuring out who the best players are.  It could be that the VERY best poker players are much better than just the "best."  It could be that 2010 was an outlier. 

I'm thinking it's a combination of all three.  But, I don't really have any expertise here.  I know some of you reading this are serious poker players ... do you guys know what's going on?