More luck in outcomes doesn't imply less skilled players

A couple of weeks ago, Michael Mauboussin posted about how there's so much luck in the NHL's shortened season. (Mauboussin is the author of "The Success Equation: Untangling Skill and Luck in Business, Sports, and Investing".) 

Most of the comments to the post didn't quite get it.  In particular, one commenter wrote, disbelievingly, 

"Did you watch the Rangers/Bruins game last night?  Luck?"

That, I think, is a pretty common response -- and not unreasonable.  NHL hockey players are among the best in the world.  hey've practiced thousands of hours, and any serious weakness in their game is instantly exploited by their opponents. How can we say that hockey is mostly luck?

Well, we don't. We're not actually arguing that hockey has more luck than skill. We're arguing something different: that the *outcome* represents luck more than skill. Or, more specifically, we're saying that the standings are more the product of *differences in luck between teams* than *differences in skill between teams*.

Which is what Mauboussin replied to the commenter:

"Here's the point: the article doesn't say that hockey players are not skillful (they are). It says that the skills of the players on opposing teams offset one another, leaving more to luck." 

The misunderstanding, it seems, stems from some readers not realizing we're talking about relative skill ... rather, they think we're talking about actual levels of player talent. We're not. If you found a peewee hockey league where the relative distribution of talent was the same as in the NHL, the luck/skill analysis would be exactly the same. The NHL season would be 60% luck, and the peewee season would also be 60% luck. The statement says nothing about how skilled the players are in a real-world sense, and it says nothing about how much skill is actually required to play the game.

Maybe we should making that more explicit. Instead of, "The standings are 60% luck," or "Hockey is 60% luck," we can say, "The standings are determined 60% by which teams are luckier than their opponents, and 40% by which teams are more talented than their opponents."

-------

Another part of the problem, perhaps, is that, in real life, we're not all that used to luck being a big factor. We rarely observe such small differences in skill as exist in most sports. In the NBA, even the worst team has a 10% chance to beat the best team. But I'm pretty sure that, at work, if you pick five random employees, and play them against the five best basketball players on staff, they'd win, like, zero percent of the time.  

In my recreational ball hockey league, my team is pretty bad. When we play the best team in the league, we *know* we're going to lose. No 10 percent, no 5 percent chance. We're going to get beat. Probably 19-3, or something. Maybe, if we're lucky, it'll be 13-5, or 14-7, or some such.  

In the real world, differences in talent are large. In professional sports, they're small.  

As Mauboussin says, in a related interview,

"This leads to one of the points that I think is most counter to intuition. As skill increases, it tends to become more uniform across the population. Provided that the contribution of luck remains stable, you get a case where increases in skill lead to luck being a bigger contributor to outcomes. That’s the paradox of skill."

We're not used to that. In our personal worlds, people are very different.  In school, the SAT-taking senior that's good at math is going to outscore the one who isn't, at least 99 percent of the time. The best programmer at work will finish faster than the worst programmer at work, 99.9 percent of the time.  And so on.  

In real life, when we see one person perform better than another, we can be fairly sure that person is actually better at the task.  

We can't do that as readily in professional sports. There, skill is much more uniform. We're not used to that, the idea that you can't tell the better team just by watching. We're not used to luck being such a big factor.