How good are the best poker players?

Last post, I mentioned a study by Steven Levitt and Thomas Miles that argued for skill in poker.  A couple of people later sent me ungated links to that study (.pdf) ... when I read it, I was surprised how small the differences seem to be between players.

Levitt and Miles (call them LM) found that more skilled players (as judged by various sets of poker writers) made a 30%+ return in tournaments, while non-ranked players lost money, to the tune of negative 15 percent.  

But, looking more closely at the study, that seems to be a very small difference. As it turns out, the skilled players were only between 4 and 29 percent more likely than the average player to make the final table.

That seems tiny.  In team sports, if you pick the best team at the beginning of the year, you'd think they've got to be at least twice as likely to make the finals, right?  In baseball, a random team has a 1 in 15 chance to make the World Series.  A *good* team, on the other hand ... well, you'd think it would be 1 in 5 or something, right?  Three times as high?

And in *individual* sports ... look at golf, say.  If there are 100 players in a typical PGA tournament, a random player has a 10 percent chance of finishing in the top ten.  In his career, Tiger Woods finished in the top ten around 63 percent of the time.  That's not 29 percent -- it's 6,200 percent.

(It's not that great an analogy, of course, but even if I tried to make the comparison more reasonable, the difference would still be huge.)

------

Also ... I'm not completely sure that the differences in the LM study are statistically significant.  In fact, the study doesn't even talk about statistical significance, which is kind of strange, for an academic study.

And it's not easy to figure out, because the authors don't give us enough information.  What we'd want to see is, the mean and SD of profit per tournament for the skilled players, and the man and SD of profit for the rest.  We don't get that.  Instead, we get the mean and SD *per player*, rather than per player-tournament.  That doesn't help, because some players enter ten times as many tournaments than others, so we can't separate talent from opportunity.

Also, we get the mean and SD for buy-in (entry fee) separately from the mean and SD for winnings, so we don't even know the mean and SD for profit or for return on investment!

I find that kind of strange.

-------

The authors' data show there was an average 1,152 players per each of the 56 tournaments (using Table 3, which leaves out the higher-stakes "Main Event").  Guessing that each final table is 9 players, that's 504 overall seats at the final tables combined.  Skilled players were 12.6 percent of the entries, so you'd expect them to account for about 64 of those seats.  

If they were 20 percent more likely to get there, that's only an extra 13 seats.

So, what LM found is that skilled poker players take a seat from less-skilled poker players one out of every four tournaments, roughly.

That doesn't seem like a lot, does it?

-------

Is there really that much luck?  In the comments to the last post, Brian Burke wrote,

"I think once a player has learned the basics, he's par with even the best players in terms of skill. Once all players are equal in terms of skill, luck dominates the outcome. Even if players differed significantly in terms of skill, it might take thousands of hands to determine who is the better player.

"I think of poker like a complex version of rock, paper, scissors. Strictly speaking, RPS a game of skill. The skill is this: don't be predictable. Once everyone realizes the basic skill, now it's effectively a game of pure luck."

The evidence so far supports that, at least a little bit.  It could certainly be that the small effects LM found are just the (equally-skilled) good players taking turns beating up on the crappy ones.

--------

Or, maybe the sample the authors used -- the 2010 World Series of Poker (WSOP) -- was an outlier, or too small a sample.  Here's some evidence, perhaps, that suggests that might be the case.

The "Main Event" of the WSOP is the biggest one.  There's a $10,000 buy-in, and, this year, there were 6,352 entries.

In July, they played enough poker to eliminate 6,343 of those entries, leaving only nine.  Those nine players will play their final table in November.

Right now, you can place a bet on which of those players will win.  If they were close to equal in skill, the odds would be almost the same.  Are they?

Well, it's hard to tell ... the players all enter the final round with different amounts of chips.  The higher bankrolls have better odds.  Here are the odds I found, with the bankrolls in brackets:

J.C. Tran (38,000,000)            9/5  
Amir Lehavot (29,700,000)         9/2  
Marc McLaughlin (26,525,000)      5/1  
Jay Farber (25,975,000)           7/1  
Ryan Riess (25,875,000)           6/1  
Sylvain Loosli (19,600,000)       8/1  
Michiel Brummelhuis (11,275,000) 13/1  
Mark Newhouse (7,350,000)        15/1  
David Benefield (6,375,000)      15/1  

I don't know how to convert chip counts into odds, so most of this data doesn't help me.  But there's one exception: Ryan Riess is favored over Jay Farber, 6/1 vs. 7/1, even though Farber has (a tiny bit) more chips.  

Does that suggest Riess is significantly better?  Maybe.  Or, it could be that the bookmaker expects more money on Riess.  Or, it could have something to do with seating position (I don't know much about poker).  But ... it's something.

If you look at the overall odds, the implied probabilties add up to about 128%, which means you'd have to bet $128 to win $100 (the difference is the bookmaker's take).  That's pretty high, so even if the Farber/Reiss difference isn't justified, it's still not an exploitable profit opportunity.

What if we look at the pre-tournament odds instead?  I found this page, which shows odds on the top 20 players.  Here are 1-3 and 18-20:

Daniel Negreanu (CAN)   40/1
Phil Ivey (USA)         50/1
Phil Hellmuth (USA)     50/1
...
Patrik Antonius (FIN)  100/1
Pius Heinz (GER)       100/1
Russell Thomas (USA)   100/1

Those odds are much, much higher than the LM study would suggest.  100/1 suggests these players are at least 6,000% more likely to win, rather than 20% as implied by the study.  (Actually, the 20% was for the final table.  Generously assuming that the top final-table player has twice the average chance of winning, that still only makes a 40% increased chance of being the overall champion.  Forty is still less than six thousand.)

------

Ah!  After I wrote that, I found something much better.  Here are a bookie's odds on Daniel Negreanu winning at least one of the WSOP's 57 tournaments:

Yes: 11/10
No:   8/13

If the average tournament has 1100 players, and Negreanu enters all 57 of them -- with only one buy-in for each -- he'd have roughly a 57/1100 chance of a win if he were average.  If his chance is 40 percent above average, that goes up to roughly 80/1100, or 13:1 against.

But, here, you can lay 13/8 instead of 13/1, which suggests that the best players are *eight times* more likely to win than average.

Did I do that right?

-------

So, what's happening with the original study?   It could be that the poker analysis industry isn't very good at figuring out who the best players are.  It could be that the VERY best poker players are much better than just the "best."  It could be that 2010 was an outlier. 

I'm thinking it's a combination of all three.  But, I don't really have any expertise here.  I know some of you reading this are serious poker players ... do you guys know what's going on?

Recognizing poker as a game of skill

Apparently, some people think poker might not be a game of skill.  Even important people, like prosecutors and politicians.

In that light, academics Steve Levitt and Thomas Miles recently published a working paper that proves the role of skill.  In a nutshell, they noticed that the highest-ranked poker players (as determined before a tournament) win a lot of money, while everyone else loses a lot of money.  

I haven't actually read the paper, because as a non-academic non-journalist non-resident-of-a-developing-country, it would cost me $5.  But .. well, yeah, of course.

Why are people still debating this?  Is there any regular poker player who doubts it?  Why does it require an academic paper (instead of, say, a two-paragraph summary of the evidence)?  Why the resistance to understanding what should be obvious?

I think it's a combination of things.  Poker, involving gambling as it does, is seen to be a kind of "vice," a seedy lower-class pastime of the uneducated.  That prejudice conflicts with a view of the game as requiring talent and skill and serious cognitive abilities.  It's probably easier to assume poker is blind luck than to re-evaluate a long-held prejudice.

The unconscious syllogism goes something like this: Poker is "bad".  Games that require skill and intellect are "good".  Therefore, poker can't require skill and intellect!

Contrast poker to, say, bridge -- even non-duplicate bridge, where the cards are dealt randomly.  Would any judge reasonably consider that bridge is just a game of luck, that it all just depends on what cards you get?  Of course not.  Bridge is played by upper-class, educated people.  Since it doesn't carry the same aura of vice, there's no dissonance in assuming it requires brainpower.

-------

Coincidentally, I found an article published only a couple of days ago that details a very similar debate about pinball and luck.  

In 1976, New York City had a legal prohibition on pinball, which the industry was trying to overturn.  City Council refused to believe pinball was a game of skill, so the industry got Roger Sharpe, one of the best players in the country, to demonstrate his abilities.  (Sharpe later went on to become a pinball designer and consultant.)

Sharpe played expertly, but one stubborn council member argued that the machine might have been tampered with.  So, Sharpe played skilfully on a second machine.  The councilman was still skeptical.  

So, according to the article, Sharpe decided he needed to resort to desperate measures.  The machine he was playing, "Bank Shot," had five lanes at the top of the playfield.  Sharpe told the skeptic that he would pull the plunger in such a way that the ball would wind up in the center lane.  In effect, he "called his shot."  He succeeded.  The head of the council declared that he had seen enough, and pinball became legal.

The thing about this is ... there was a lot more luck in Sharpe's "called shot" than in his high-scoring games.  For one thing, there's probably, by my guess, a 10 to 15 percent chance that any given shot would wind up in that lane, just by luck.  (A first guess would be 20 percent, but, from my experience, those machines were usually designed so that the center lane is the hardest.)

Second, even the best player is going to miss that shot occasionally. 

A more reliable test of skill would be to actually compete.  If Sharpe had challenged 10 councilman to a best-of-seven skins match, he'd have had almost a 100 percent chance of going 10-0.

For an analogy ... imagine Babe Ruth having to prove that hitting a baseball requires skill.  So, he goes out, plays a major-league season, and hits .356 with more home runs than many other teams.  But people still don't believe it.  So the Babe says, "OK, stand there in center field and I'll hit a fly ball right to you."  And they do, and he does, and now people are convinced.

That's about what it's like.

-------

The thing is, there *is* a significant amount of luck in poker and pinball.  But, of course, that doesn't mean much -- there's also luck in golf, and chess.  

The difference, perhaps, is that in golf and chess, it's mostly "invisible" luck -- your brain and your body just happen to be a bit off, beyond your control, or, a rare situation arises, randomly, that you don't know how to deal with.  However, in poker, there's an obvious source of "external" luck -- the deal of the cards.  Similarly, in pinball, where there are bumpers and slingshots that propel the ball unpredictably.

In both games, there are times when you lose and it looks like it couldn't have been because of lack of skill.  Your opponent lucks into the nut flush, or you hit a good shot, but a bumper rockets the ball straight down between the flippers.

But in the longer run, the luck evens out.  Every player gets good hands and bad hands; the difference is that the skilled player will know how to maximize the value of the pots he wins on the good hands, how to know fold early on the bad hands, and, most importantly, to know when his "good" hand is actually not good enough.  

In pinball, the good player will compensate for the unplayable "house balls" by taking better advantage of the others -- and, by learning which particular shots lead to higher-probability bad outcomes. 

-------

Most importantly, for this debate: it doesn't make sense at all to ask whether a game is "predominantly" luck or "predominantly" skill.  It depends not just on the game itself, but on the details of the particular competition.

For instance, there's more luck in a single hand of poker than there is in an eight-hour elimination tournament.  There's more luck when players are similar in ability than when they're not.  In the PAPA pinball championships, there's more luck in the final rounds ("playoffs") than there is in qualifying ("regular season"), because of how the rules go.

Some games are all luck ("buy one lottery ticket at random").  Some games are all skill ("be taller than the other guy").  Most games are both -- and the proportion of how much is luck, and how much is skill, depends as much on the details as the rules of the actual game.

------

However, there is one universal.  Luck tends to even out.  So, the more you play, the less randomness matters, and the more skill matters.  In the long run, any game that has *any* skill component has a *large* skill component.

So, if politicians are truly worried about luck dominating online poker, they shouldn't discourage it.  Instead, they should mandate that everyone play more.

Basketball robot shoots worse than some humans

Today, at the Carnegie Science Museum in Pittsburgh, I watched their resident basketball robot shoot some free throws.  Here's a video of what it looks like.

How accurate do you think the robot is?  Take a guess before you read on.  (Or, just read on -- who am I to give you orders?)

The answer is ... not that accurate.  Well, at least, a lot less accurate than I thought.  When I was there, the robot's FT% was only 83 percent (405 for 488).

I was a bit shocked.  I expected close to perfect.  After all, it's the same throw under exactly the same circumstances, every time.  (Actually, it's two different shots: sometimes the robot throws underhand, and sometimes overhand from behind his back.  But, I witnessed the robot missing shots from both positions.)

It seems wrong, doesn't it, that a human can outperform an expensive robot at a repetitive physical task?  In his career, Rick Barry routinely shot over 90 percent (albeit underhanded).   So, the machine misses almost twice as many shots as Barry. 

What's going on?  I don't know.  

For what it's worth, here's my theory:

The robot lets the ball roll down the ramp that's his "hand" before actually doing the throw with his "arm".  Maybe the position it reaches varies randomly, based on random differences in friction.  Maybe if a dirty part of the ball contacts a dirty part of the arm, the ball doesn't quite reach the expected point, and the throw misses.

There could be other friction-related issues that cause variation, like, perhaps, the axis of rotation of the ball when it's released.  (The robot hits the backboard every time.)

Any physicists reading who can deliver a more informed hypothesis?

------

If a human can outperform a robot, then it must be that he does *something* better than the machine does.  What? 

My guess is: when the human shoots, he can notice if something's a little off, like the ball slips a bit.  In that case, he can adjust his motion on the spot to try to counter that.  The robot, of course, doesn't do that.

That's the only thing I can think of.  

-------

I'd love to hear other opinions, because I'm very, very surprised.  I would have bet good money that you could easily make a robot that shoots, say, 98 percent.  Could it really be that tiny differences caused by friction could make such a big difference in outcomes?