Aggregate career year luck as evidence of PED use

Back in 2005, I came up with a method to try to estimate how lucky a player was in a given season (see my article in BRJ 34, here). I compared his performance to a weighted average of his two previous seasons and his two subsequent seasons, and attributed the difference to luck.

I'm working on improving that method, as I've been promising Chris Jaffe I would (for the last eight years or something). One thing I changed was that now, I use a player's entire career as the comparison set, instead of just four seasons. One reason I did that is that I realized that, the old way, a player's overall career luck was based almost completely on how well he did at the beginning and end of his career.

The method I used was to weight the four surrounding seasons in a ratio of 1/2/2/1. If the player didn't play all four of those years, the missing seasons just get left out.

So, suppose a batter played from 1981 to 1989. The sum of his luck wouldn't be zero:

(81 luck) = (81)                     - 2/3(82) - 1/3(83) 
(82 luck) = (82) - 2/5(81)           - 2/5(83) - 1/5(84) 
(83 luck) = (83) - 2/6(82) - 1/6(81) - 2/6(84) - 1/6(85) 
(84 luck) = (84) - 2/6(83) - 1/6(82) - 2/6(85) - 1/6(86) 
(85 luck) = (85) - 2/6(84) - 1/6(83) - 2/6(86) - 1/6(87) 
(86 luck) = (86) - 2/6(85) - 1/6(84) - 2/6(87) - 1/6(88) 
(87 luck) = (87) - 2/6(86) - 1/6(85) - 2/6(88) - 1/6(89)
(88 luck) = (88) - 2/5(87) - 1/5(86) - 2/5(89) 
(89 luck) = (89) - 2/3(88) - 1/3(87) 
---------------------------------------------------------
total luck = 13/30(81) +1/6(82) - 7/30(83) - 1/30(84) - 1/30(86) - 7/30(87) - 1/6(88) + 13/30 (89)

(*Year numbers not followed by the word "luck" refer to player performance level that year).

(Sorry about the small font.)

If a player has a good first two years and last two years, he'll score lucky. If he has a good third and fourth year, or third last and fourth last year, he'll score unlucky. The years in the middle (in this case, 1985, but, for longer careers, any seasons other than the first four and last four) cancel out and don't affect the total.

Now, by comparing each year to the player's entire career, that problem is gone. Now, every player's luck will sum close to zero (before regressing to the mean).

It's not that big a deal, but it was still worth fixing.

--------

This meant I had to adjust for age. The old way, when a player was (say) 36, his estimate was based on his performance from age 34-38 ... reasonably close to 36. Although players decline from 34 to 38, I could probably assume that the decline from 34 to 36 was roughly equal to the decline from 36 to 38, so the age biases would cancel out.

But now, I'm comparing a 36-year-old player to his entire career ... say, from age 25 to 38. Now, we can't assume the 25-35 years, when the player was in his prime, cancel out the 37-38 years, when he's nowhere near the player he was.

---------

So ... I have to adjust for age. What adjustment should I use? I don't think there's an accepted aging scale. 

But ... I think I figured out how to calculate one.

Good luck should be exactly as prevalent as bad luck, by definition. That means that when I look at all players of any given age, the total luck should add up to zero.

So, I experimented with age adjustments until all ages had overall luck close to zero. It wasn't possible to get them to exactly zero, of course, but I got them close.

From age 20 to 36, for both batting and pitching, no single age was lucky or unlucky more than half a run per 500 PA. Outside of that range, there were sample size issues, but that's OK, because if the sample is small enough, you wouldn't expect them close to zero anyway.

---------

Anyway, it occurred to me: maybe this is an empirical way to figure out how players age! Even if my "luck" method isn't perfect, as long as it's imperfect roughly the same way for various ages, the differences should cancel out. 

As I said, I'm still fine-tuning the adjustments, but, for what it's worth, here's what I have for age adjustments for batting, from 1950 to 2016, denominated in Runs Created per 500 PA:

      age(1-17) = 0.7
        age(18) = 0.74
        age(19) = 0.75
        age(20) = 0.775
        age(21) = 0.81
        age(22) = 0.84
        age(23) = 0.86
        age(24) = 0.89
        age(25) = 0.9
        age(26) = 0.925
        age(27) = 0.925
        age(28) = 0.925
        age(29) = 0.925
        age(30) = 0.91
        age(31) = 0.8975
        age(32) = 0.8775
        age(33) = 0.8625
        age(34) = 0.8425
        age(35) = 0.8325
        age(36) = 0.8225
        age(37) = 0.8025
        age(38) = 0.7925
     age(39-42) = 0.7
       age(43+) = 0.65

These numbers only make sense relative to each other. For instance, players created 11 percent more runs per PA at age 24 than they did at age 37 (.89 divided by .8025 equals 1.11).

(*Except ... there might be an issue with that. It's kind of subtle, but here goes.

The "24" number is based on players at age 24 compared to the rest of their careers. The "37" number is based on players at age 37 compared to the rest of their careers. It doesn't necessarily follow that the ratio is the same for those players who were active both at 24 and 37. 

If you don't see why: imagine that every active player had to retire at age 27, and was replaced by a 28-year-old who never played MLB before. Then, the 17-27 groups and the 28-43 groups would have no players in common, and the two sets of aging numbers would be mutually exclusive. (You could, for instance, triple all the numbers in one group, and everything would still work.)

In real life, there's definitely an overlap, but only a minority of players straddle both groups. So, you could have somewhat of the same situation here, I think.

I checked batters who were active at both 24 and 37, and had at least 1000 PA combined for those two seasons. On average, they showed lucky by +0.2 runs per 500 PA. 

That's fine ... but from 750 to 999 PA, there were 73 players, and they showed unlucky by -3.7 runs per 500 PA. 

You'd expect those players with fewer PA to have been unlucky, since if they were lucky, they'd have been given more playing time. (And players with more PA to have been lucky.)  But is 3.7 runs too big to be a natural effect? (And is the +0.2 runs too small?)

My gut says: maybe, by a run or two. Still, if this aging chart works for this selective sample within a couple of runs in 500 PA, that's still pretty good.

Anyway, I'm still thinking about this, and other issues.)

---------

In the process of experimenting with age adjustments, I found that aging patterns weren't constant over that 67-year period. 

For instance: for batters from 1960 to 1970, the peak ages from 27 to 31 all came out unlucky (by the standard of 1950-2015), while 22-26 and 32-34 were all lucky. That means the peak was lower that decade, which means more gentle aging. 

Still: the bias was around +1/-1 run of luck per 500 PA -- still pretty good, and maybe not enough to worry about.

---------

If the data lets us see different aging patterns for different eras, we should be able to use it to see the effects of PEDs, if any.

Here's luck per 500 PA by age group for hitters, 1995 to 2004 inclusive:

-1.75   age 17-22
-0.74   age 23-27
+0.61   age 28-32
+0.99   age 33-37
+0.45   age 38-42

That seems like it's in the range we'd expect given what we know, or think we know, about the prevalence of PEDs during that period. It's maybe 2/3 of a run better than normal for ages 28 to 42. If, say 20 percent of hitters in that group were using PEDs, that would be around 3 runs each. Is that plausible? 

Here's pitchers:

-1.22   age 17-22
-0.51   age 23-27
+1.36   age 28-32 
+1.42   age 33-37 
+1.07   age 38-42 

Now, that's pretty big (and statistically significant), all the way from 28 to 42: for a starter who faces 800 batters, it's about 2 runs. if 20 percent of pitchers are on PEDs, that's 10 runs each.

By checking the post-steroid era, we can check the opposing argument that it's not PEDs, it's just better conditioning, or some such. Here's pitchers again, but this time 2007-2013:

-0.06   age 17-22
+1.01   age 23-27
+0.30   age 28-32
-1.67   age 33-37
+0.59   age 38-42

Now, from 28 to 42, pitchers were *unlucky* on average, overall.

I'd say this is pretty good support for the idea that pitchers were aging better due to PEDs ... especially given actual knowledge and evidence that PED use was happening.

Fun with splits

This was Frank Thomas in 1993, a year in which he was American League MVP with an OPS of 1.033.

                 PA   H 2B 3B HR  BB  K   BA   OPS 
--------------------------------------------------   
'93 F. Thomas   676 174 36  0 41 112 54 .317 1.033  

Most of Thomas's hitting splits were fairly normal:

Home/Road:              1.113/0.950
First vs. Second Half:  0.970/1.114
Vs. RHP/LHP:            1.019/1.068
Outs in inning:         1.023/1.134/0.948
Team ahead/behind/tied: 1.016/0.988/1.096
Early/mid/late innings: 1.166/0.950/0.946
Night/day:              1.071/0.939

But I found one split that was surprisingly large:

              PA   H 2B 3B HR BB  K   BA   OPS  RC/G 
----------------------------------------------------
Thomas 1     352 108 22  0 33 58 34 .367 1.251 14.81 
Thomas 2     309  66 14  0  8 54 20 .259 0.796  5.45 

"Thomas 1" was an order of magnitude better than "Thomas 2," to the extent that you wouldn't recognize them as the same player. 

This is a real split ... it's not a selective-sampling trick, like "team wins vs. losses," where "team wins" were retroactively more likely to have been games in which Thomas hit better. (For the record, that particular split was 1.172/.828 -- this one is wider.)

So what is this split? The answer is ... 

.
.
.

The first line is games on odd-numbered days of the month. The second line is even-numbered days.

In other words, this split is random.

In terms of OPS difference -- 455 points -- it's the biggest odd/even split I found for any player in any season from 1950 to 2016 with at least 251 AB PA each half. 

If we go down to a 150 AB minimum, the biggest is Ken Phelps in 1987:

1987 Phelps   PA   H 2B 3B HR BB  K  BA   OPS   RC/G 
----------------------------------------------------
odd          204  31  3  0  8 39 33 .188 0.695  3.79 
even         208  55 10  1 19 41 42 .329 1.204 13.03 

And if we go down to 100 AB, it's Mike Stanley, again in 1987, but on the opposite days to Phelps:

1987 Stanley  PA   H 2B 3B HR BB  K  BA   OPS   RC/G 
----------------------------------------------------
odd          134  42  6  1  6 18 23 .362 1.034 10.49 
even         113  17  2  0  0 13 25 .170 0.455  1.55 

But, from here on, I'll stick to the 251 AB standard.

That 1993 Frank Thomas split was also the biggest gap in home runs, with a 25 HR difference between odd and even (33 vs. 8). Here's another I found interesting -- Dmitri Young in 2001:

2001 D Young  PA   H 2B 3B HR BB  K  BA   OPS   RC/G 
----------------------------------------------------
Odd          285  68 12  2  2 18 40 .255 0.639  3.48 
Even         292  95 16  1 19 19 37 .348 1.013  9.51 

Only two of Young's 21 home runs came on odd-numbered days. The binomial probability of that happening randomly (19-2/2-19 or better) is about 1 in 4520.*  And, coincidentally, there were exactly 4516 players in the sample!

(* Actually, it must be more likely than 1 in 4520. The binomial probability assumes each opportunity is independent, and equally likely to occur on an even day as an odd day. But, PA tend to happen in daily clusters of 3 to 5. Since PAs are more likely to cluster, so are HR. 

To see that more easily, imagine extreme clustering, where there are only two games a year (instead of 162), with 250 PA each game. Half of all players would have either all odd PA or all even PA, and you'd see lots of extreme splits.)

For K/BB ratio, check out Derek Jeter's 2004:  

2004 Jeter   PA   H 2B 3B HR BB  K  BA   OPS   RC/G 
---------------------------------------------------
odd         362 113 27  1 15 14 63 .325 0.888  7.12 
even        327  75 17  0  8 32 36 .254 0.720  4.40 

There were bigger differences, but I found Jeter's the most interesting. 

In 1978, all 10 of Rod Carew's triples came on even-numbered days:

1978 Carew   PA   H 2B 3B HR BB  K  BA   OPS   RC/G 
---------------------------------------------------
odd         333  92 10  0  0 45 34 .319 0.766  5.46 
even        309  96 16 10  5 33 28 .348 0.950  8.69 

A 10-0 split is a 1-in-512 shot. I'd say again that it's actually a bit more likely than that because of PA clustering, but ... Carew actually had *fewer* PA in that situation! 

Oh, and Carew also hit all five of his HR on even days. Combining them into 15-0 is binomial odds of 16383 to 1, if you want to do that.

Strikeouts and walks aren't quite as impressive. It's Justin Upton 2013 for strikeouts:

2003 Upton     PA   H 2B 3B HR BB   K   BA  OPS  RC/G 
-----------------------------------------------------
odd           330  71 14  1 16 31 102 .237 0.761 4.67 
even          303  76 13  1 11 44  59 .293 0.875 6.84 

And Mike Greenwell 1988 for walks:

88 Greenwell   PA   H 2B 3B HR BB   K  BA   OPS  RC/G 
-----------------------------------------------------
odd           357  91 15  3 10 62  18 .308 0.910 7.61 
even          320 101 24  5 12 25  20 .342 0.973 8.85 

Interestingly, Greenwell was actually more productive on the even-numbered days where he took less than half as many walks.

Finally, here's batting average, Grady Sizemore in 2005:

2005 Sizemore  PA   H 2B 3B HR BB   K  BA   OPS  RC/G 
-----------------------------------------------------
odd           344  69  9  4 12 26  79 .217 0.660 3.45 
even          348 116 28  7 10 26  53 .360 0.992 9.50 

Another anomaly -- Sizemore hit more home runs on his .217 days than on his .360 days.

-------

Anyway, what's the point of all this? Fun, mostly. But, for me, it did give me a better idea of what kinds of splits can happen just by chance. If it's possible to have a split of 33 odd homers and 8 even homers, just by luck, then it's possible to have 33 first-half homers and 8 second-half homers, just by luck. 

Of course, you should just expect that size of effect once every 40 years or so. It might more intuitive to go from a 40-year standard to a single-season standard, to get a better idea of what we can expect each year. 

To do that, I looked at 1977 to 2016 -- 39 seasons plus 1994. Averaging the top 39 should roughly give us the average for the year. Instead of the average, I figured I'd just (unscientifically) take the 25th biggest ... that's probably going to be close to the median MLB-leading split for the year, taking into account that some years have more than one of the top 39.

For HR, the 25th ranked is Fred McGriff's 2002. It's an impressive 22/8 split:

02 McGriff   PA   H 2B 3B HR BB   K  BA   OPS   RC/G 
----------------------------------------------------
odd         297  70 11  1 22 42  47 .275 0.961  7.74 
even        289  73 16  1  8 21  52 .272 0.754  4.89 

For OPS, it's Scott Hatteberg in 2004:

04 Hatteberg PA   H 2B 3B HR BB   K  BA   OPS   RC/G 
----------------------------------------------------
odd         312  92 19  0 10 37  23 .335 0.926  8.12 
even        310  64 11  0  5 35  25 .233 0.647  3.47

For strikeouts, it's Felipe Lopez, 2005. Not that huge a deal ... only 27 K difference.

05 F. Lopez  PA   H 2B 3B HR BB   K  BA   OPS   RC/G 
----------------------------------------------------
odd         316  78 15  2 12 19  69 .263 0.755  4.75 
even        321  91 19  3 11 38  42 .322 0.928  7.95 

For walks, it's Darryl Strawberry's 1987. The difference is only 23 BB, but to me it looks more impressive than the 27 strikeouts:

87 Strwb'ry  PA   H 2B 3B HR BB   K  BA   OPS   RC/G 
----------------------------------------------------
odd         315  77 15  2 19 37  55 .277 0.912  7.02 
even        314  74 17  3 20 60  67 .291 1.045  9.49 


For batting average, number 25 is Orestes Infante, 2011, but I'll show you the 24th ranked, which is Rickey Henderson in his rookie card year. (Both players round to a .103 difference.)

1980 Rickey  PA   H 2B 3B HR BB   K  BA   OPS   RC/G 
----------------------------------------------------
odd         340 100 13  1  2 60  21 .357 0.903  8.07 
even        368  79  9  3  7 57  33 .254 0.739  4.67 

-------

I'm going to think of this as, every year, the league-leading random split is going to look like those. Some years it'll be higher, some lower, but these will be fairly typical.

That's the league-leading split for *each category*. There'll be a random home/road split of this magnitude (in addition to actual home/road effect). There'll be a random early/late split of this magnitude (in addition to any fatigue/weather effects). There'll be a random lefty/righty split of this magnitude (in addition to actual platoon effects). And so on.

Another way I might use this is to get an intuitive grip on how much I should trust a potentially meaningful split. For instance, if a certain player hits substantially worse in the second half of the season than in the first half, how much should you worry? To figure that out, I'd list a season's biggest even/odd splits alongside the season's biggest early/late splits. If the 20th biggest real split is as big as the 10th biggest random split, then, knowing nothing else, you can start with a guess that there's a 50 percent chance the decline is real.

Sure, you could do it mathematically, by figuring out the SD of the various stats. But that's harder to appreciate. And it's not nearly as much fun as being able to say that, in 1987, Rod Carew hit every one of his 10 triples and 5 homers on even-numbered days. Especially when anyone can go to Baseball Reference and verify it.

Does the NHL's "loser point" help weaker teams?

Back when I calculated that it took 73 NHL games for skill to catch up with luck in the standings, I was surprised it was so high. That's almost a whole season. In MLB, it was less than half a season, and in the NBA, Tango found it was only 14 games, less than one-fifth of the full schedule.

Seventy-three games seemed like that was a lot of luck. Why so much? As it turns out, it was an anomaly -- the NHL was just having an era where differences in team talent were small. Now, it's back under 40 games.

But I didn't know that at the time, so I had a different explanation: it must be the extra point the NHL started giving out for overtime losses. The "loser point," I reasoned, was reducing the importance of team talent, by giving the worse teams more of a chance to catch up to the better teams.

My line of thinking was something like this: 

1. Loser points go disproportionately to worse teams. For team-seasons, there's a correlation of around .4 between negative goal differential (a proxy for team quality) and OTL. So, the loser point helps the worse teams gain ground on the better teams.

2. Adding loser points adds more randomness. When you lose by one goal, whether that goal comes early in the game, or after the third period, is largely a matter of random chance. That adds "when the goals were" luck to the "how many goals there were" luck, which should help mix up the standings more. In fact, as I write this, the Los Angeles Kings have two more wins and three fewer losses than the Chicago Blackhawks. But, because Chicago has five OTL to the Kings' one, they're actually tied in the standings.

But ... now I realize that argument is wrong. And, the conclusion is wrong. It turns out the loser point actually does NOT help competitive balance in the NHL. 

So, what's the flaw in my old argument? 

-------

I think the answer is: the loser point does affect how compressed the standings get in terms of actual points, but it doesn't have much effect on the *order* of teams. The bottom teams wind up still at the bottom, but (for instance) instead of having only half as many points as the top teams, they have two-thirds as many points.

Here's one way to see that. 

Suppose there's no loser point, so the winner always gets two points and the loser always gets none (even if it was an overtime or shootout loss). 

Now, make a change so the losing team gets a point, but *every time*. In that case, the difference between any two teams gets cut in half, in terms of points -- but the order of teams stays exactly the same. 

The old way, if you won W games, your point total was 2W. Now, it's W+82. Either way, the order of standings stays the same -- it's just that the differences between teams are cut in half, numerically.

It's still true that the "loser point" goes disproportionately to the worse teams -- the 50-32 team gets only 32 loser points, while the 32-50 team gets 50 of them. But that doesn't matter, because those points are never enough to catch up to any other team. 

If you ran the luck vs. skill numbers for the new system compared to the old system, it would work out exactly the same.

-------

In real life, of course, the losing team doesn't get a point every time: only when it loses in overtime. Last season, that happened in about 11.6 percent of games, league-wide, or about 23.3 percent of losses.

If the loser point happened in *exactly* 23.3 percent of losses, for every team, with no variation, the situation would be the same as before -- the standings would get compressed, but the order wouldn't change. It would be as if, every loss, the loser got an extra 0.233 points. No team could pass any other team, since for every two points it was behind, it only gets 0.233 points to catch up. 

But: what if you assume that it's completely random which losses become overtime losses?  Now, the order can change. A 40-42 team can catch up to a 41-41 team if its losses had randomly included two more overtime losses than its rival. The chance of that happening is helped by the fact that the 40-42 team has one extra loss to try to randomly convert. It needs two random points to catch up, but it starts with a positive expectation of an 0.233 point head start.

If losses became overtime losses in a random way, then, yes, the OTL would make luck more important, and my argument would be correct. But they don't. It turns out that better teams turn losses into OTL much more frequently than worse teams, on a loss-for-loss basis.

Which makes sense. Worse teams' losses are more likely to be blowouts, which means they're less likely to be close losses. That means fewer one-goal losses, proportionately. 

In other words: 

(a) bad teams have more losses, but 
(b) those losses are less likely to result in an OTL. 

Those two forces work in opposite directions. Which is stronger?

Let's run the numbers from last year to find out.

If we just gave two points for a win, and zero for a loss, we'd have: 

SD(observed)=16.47
SD(luck)    = 9.06
SD(talent)  =13.76

But in real life, which includes the OTL, the numbers are

SD(observed)=15.44
SD(luck)    = 8.48
SD(talent)  =12.90

Converting so we can compare luck to talent:

35.5 games until talent=luck (no OTL point)
35.4 games until talent=luck (with OTL point)

It turns out, the two factors almost exactly cancel out! Bad teams have more chances for an OTL point because they lose more -- but those losses are less likely to be OTL almost in exact proportion.

And that's why I was wrong -- why the OTL point doesn't increase competitive balance, or make the standings less predictable. It just makes the NHL *look* more competitive, by making the point differences smaller.

2007-12 was an era of competitive balance in the NHL

Five years ago, I calculated that in the NHL, it took 73 games until talent was as important as luck in determining the standings. But in a previous study, Tango found that it took only 36 games. 

Why the difference?

I think it's because the years for which I ran the study -- 2006-07 to 2011-12 -- were seasons in which the NHL was much more balanced than usual. 

For each of those six seasons, I went to hockey-reference to find the SD of team standings points:

SD(observed)
--------------
2006-07  16.14
2007-08  10.43
2008-09  13.82
2009-10  12.95
2010-11  13.27
2011-12  11.73
--------------
average  13.18  (root mean square)

Tango's study was written in August, 2006. The previous season had a higher spread:

SD(observed)
--------------
2005-06  16.52

So, I think that's the answer. It just happened that the seasons I looked at had less competitive balance that the season or seasons Tango looked at.

But what's the right answer for today's NHL? Well, it looks like the standings spread in recent seasons has moved back closer to Tango's numbers:

SD(observed)
--------------
2013-14  14.26
2014-15  15.91
2015-16  12.86
2016-17  15.14
2017-18  15.44
--------------
average  14.76

What does that mean for the "number of games" estimate? I'll do the calculation for last season, 2017-18.

From the chart, SD(observed) is 15.44 points. SD(luck) is roughly the same for all years of the shootout era (although it varies very slightly with the number of overtime losses), so I'll use the old study's number of 8.44 points. 

As usual, 

SD(talent)^2 = SD(observed)^2 - SD(luck)^2
SD(talent)^2 = 15.44^2 - 8.44^2
SD(talent)   = 12.93

So last year, SD(talent) was 12.93. For the six seasons I looked at, it was 8.95. 

SD(talent)
--------------
2016-12   8.95
2017-18  12.93

Now, let's convert to games.*  

*Specifically, "luck as important as talent" means SD(luck)=SD(talent). Formula: using the numbers for a full season, divide SD(luck) by SD(talent), square it, and multiply by the number of games (82).

When SD(talent) is 8.95, like the seasons I looked at, it takes 77 games for luck and talent to even out. When SD(talent) = 12.93, like it was last year, it takes only ... 36 games.

Coincidentally, 36 games is exactly what Tango found in his own sample.

talent=luck, after
------------------
2016-12  77 games
2017-18  36 games

Two things we can conclude from this:

1. Actual competitive balance (in terms of talent) does seem to change over time in non-random ways. The NHL from 2006-12 does actually seem to have been a more competitive league than from 2013-18. 

2. The "number of games" way of expressing the luck/talent balance is very sensitive to moderate changes in the observed range of the standings.

--------

To expand a bit on #2 ... 

There must be significant random fluctuations in observed league balance.  We mention that sometimes in passing, but I think we don't fully appreciate how big those random fluctuations can be.

Here, again, is the SD(observed) for the seasons 2014-17:

SD(observed)
--------------
2014-15  15.91
2015-16  12.86
2016-17  15.14

It seems unlikely that 2015-16 really had that much tighter a talent distribution than the surrounding seasons. What probably happened, in 2015-16, was just a fluke -- the lucky teams happened to be lower-talent, and the unlucky teams happened to be higher-talent. 

In other words, the difference was probably mostly luck. 

A different kind of luck, though -- luck in how each individual team's "regular" luck correlated, league-wide, with their talent. When the better teams (in talent) are luckier than the worse teams , the standings spread goes up. When the worse teams are luckier, the standings get compressed.

Anyway ... the drop in the chart from from 15.91 to 12.86 doesn't seem that big. But it winds up looking bigger once you subtract out luck to get to talent:

SD(talent)
--------------
2014-15  13.49
2015-16   9.70
2016-17  12.57

The difference is more pronounced now. But, check out what happens when we convert to how many games it takes for luck and talent to even out:

Talent=luck, after
------------------
2014-15  32 games
2015-16  62 games
2016-17  37 games

Now, the differences are too large to ignore. From 2014-15 to 2015-16, SD(observed) went down only 19 percent, but the "number of games" figure nearly doubled.

And that's what I mean by #2 -- the "number of games" estimate is very sensitive to what seem like mild changes in standings variation. 

-------

Just for fun, let's compare 2006-07, one of the most unbalanced seasons, to 2007-08, one of the most balanced. Just looking at the standings, there's already a big difference:

SD(observed)
--------------
2006-07  16.14
2007-08  10.43

But it becomes *huge* when when you express it in games: 

Talent=luck, after
------------------
2006-07   31 games
2007-08  156 games

In one year, our best estimate of how many games it takes for talent to exceed luck changed by a factor of *five times*. And, I think, almost all that difference is itself just random luck.

How much of success in life is luck instead of skill?

How much of MLB teams' success is due to skill, and how much due to luck? We have a pretty good idea of the answer to that. But what about success in life, in general? If a person is particularly successful in their chosen field, how much of that success is due to luck?

That's the question Robert Frank asks in his 2016 book, "Success and Luck."  He believes that luck is a substantial contributor to success, as evidenced by his subtitle: "Good Fortune and the Myth of Meritocracy."

On the basic question, I agree with him that luck is a huge factor in how someone's life turns out. There is a near-infinite number of alternative paths our lives could have taken. If a butterfly had flapped its wings differently in China decades ago, I might not even exist now, never mind be sitting here typing this blog post.

In his preface, Frank favorably quotes Nicholas Kristof:

"America's successful people['s] ... big break came when they were conceived in middle-class American families how loved them, read them stories, and nurtured them .... They were programmed for success by the time they were zygotes."

But ... that's not a very practical observation, is it? Sure, I am phenomenally lucky that my parents decided to have sex that particular moment that they did, and that the winning sperm cell turned out to be me. In that light, you could say that luck explains almost 100 percent of my success. 

So, maybe a better question is: suppose I was born as me, but in random circumstances, in a random place and time. How much more or less successful would I be, on average?

As Frank writes:

"I often think of Birkhaman Rai, the young hill tribesman from Bhutan who was my cook long ago when I was a Peace Corps volunteer in a small village in Nepal. To this day, he remains perhaps the most enterprising and talented person I've ever met....

"... Even so, the meager salary I was able to pay him was almost certainly the high point of his life's earnings trajectory. If he'd grown up in the United States or some other rich country, he would have been far more prosperous, perhaps even spectacularly successful."

Agreed. Those of us who are alive in a wealthy society in 2017 are pretty much the luckiest people, in terms of external circumstances, of anyone in the history of the world.  For all of us, almost all of our success is due to having been born at the right time in the right place. 

But, again, that's not a very useful answer, is it? Even the most talented, hardest-working person would have nothing if he had been born in the wrong place and time, so you have to conclude that every successful person has been overwhelmingly lucky.

I think we have to hold our personal characteristics as a given, too. Because, almost everyone who is successful in a given field has far-above-average talent or interest in that field. I was lucky to have been born with a brain that likes math. Wilt Chamberlain was lucky to have been born with a genetic makeup that made him grow tall. Bach was born with the brain of a musical genius.

It gets even worse if you consider not just innate talent for a particular field, but other mental characteristics that we usually consider character rather than luck. Suppose you have an ability to work hard, or to persevere under adversity. Those likely have at least some genetic -- which is to say, random -- basis. So when someone with only average musical talent becomes a great composer by hard work, we can say, "well, sure, but he was lucky to have been born with that kind of drive to succeed."

Frank says:

"I hope we can agree that success is much more likely for people with talents that are highly valued by others, and also for those with the ability and inclination to focus intently and work tirelessly. But where do those personal qualities come from? We don't know, precisely, other than to say that they come from some combination of genes and the environment. ...

"In some unknown proportion, genetic and environmental factors largely explain whether someone gets up in the morning feeling eager to begin work. If you're such a person, which most of the time I am not, you're fortunate."

So, even if you got to where you are by working hard, Frank says, that's still luck! Because, you're lucky to have the kind of personality that sees the value of hard work.

I don't disagree with Frank that the kind of person you are, in terms of morals and virtues, is partly determined by luck. But, in that case, what *isn't* luck?

------

That's the problem with Frank's argument. Drill down deep enough, and everything is luck. You don't even need a book for that; I can do it in one paragraph, like this:

There are seven billion people in the world right now. Which one I am, out of those seven billion, is random, as far as I'm concerned; I had no say in which person I would be born as. Therefore, if I wind up being Bill Gates, the richest man in the world, I hit a 6,999,999,999 to 1 shot, and I am very, very lucky!

What Frank never explicitly addresses is: what kind of success does he consider NOT caused by luck? I don't think that anywhere, in his 200-page book, he even gives one example. 

We can kind of figure it out, though. At various points in the book, Frank illustrates his own personal lucky moments. There was the time he got his professor job at Cornell by the skin of his teeth (he was the last professor hired, in a year where Cornell hired more economics professors than ever before). Then, there was the time he almost drowned while windsurfing, but just in time managed to free himself from under his submerged sail. "Survival is somtimes just a matter of pure dumb luck, and I was clearly luck's beneficiary that day."

Frank's instances of luck are those that occurred on his path while he was already himself. He doesn't say how he was almost born in Nepal and destined for a life of poverty, or he was lucky that one of his cells didn't mutate while in the womb to make him intellectually disabled. 

I'll presume, then, that the luck Frank is talking about is the normal kind of career and life luck that most of us think about, and that the "your success is mostly luck because you were born smart" is just a rhetorical flourish.

--------

We don't have a definition problem in our usual analysis of baseball luck, because we are careful to talk about what we consider luck and what we don't. For a team's W-L record, we specify that the "luck" we're talking about is the difference between the team's talent and the team's outcome. So, if a team is good enough to finish with an average of 88 wins, but it actually wins 95 games, we say it was lucky by 7 games.

We specifically ignore certain types of luck, such as injuries and weather and bad calls by the umpire. And, we specifically exclude certain types of luck, like how an ace pitcher randomly happened to meet and marry a woman from Seattle, which led him to sign at a discount with the Mariners, which meant that they wound up more talented than they would have otherwise.

By specifically defining what's luck and what's not, we can come up with a specific answer to the specific question. We know the difference between talent (as we define it) and luck (as we define it) can be measured by the binomial approximation to the normal distribution. So, we can calculate that the effect of luck is a standard deviation of about 6.4 games per season, and the variation in talent is about 9 games per season.

From that, we can calculate a bunch of other things. Such as: on average, a team that finishes with a 96-66 record is most likely a 91-71 team that got lucky. In other words, if the season were replayed again, like in an APBA simulation, that team would be more likely to finish with 91 wins than with 96.

I think that's the question Frank really wants to answer -- that if you took Bill Gates, and made him play his life over, he wouldn't come close to being the richest man in the world. He just had a couple of very lucky breaks, breaks that probably wouldn't have come is way if God rolled the dice again in his celestial APBA simulation of humanity.

-----

Another reason to think that's what Frank means is that, when he gets down to mathematical business, that seems to be the definition he uses. There, he talks about luck as distinct from "skill" and "effort". 

When Frank does that, his view of success and luck is a lot like the sabermetrician's view of success and luck. We assume a person (or team) has a certain level of talent, and the observed level of success might be higher or lower than expectations depending on whether good luck or bad luck dominates.

In his Chapter 4, and its appendix, Frank tries to work that out mathematically.

Suppose everyone has a skill level distributed uniformly between 0 and 100, and a level of luck distributed uniformly between 0 and 100 (where 50 is average). And, suppose that the level of success is determined 95 percent by skill and 5 percent by luck.

Even though luck creates only 5 percent of the outcome, it's enough to almost ensure that the most skilled person winds up NOT the most successful. With 1,000 participants, the most skilled will "win" about 55 percent of the time. With 100,000 participants, the most skilled will win less than 13 percent of the time.

Frank gives an excellent explanation of why that happens:

"The most skilled competitor in a field of 1,000 would have an expected skill level of 99.9, but an expected luck level of only 50.
"It follows that the expected performance level of the most skillful of 1,000 contestants is P=0.95 * 99.9 + 0.05 * 50 = 97.4 ... but with 999 other contestants, that score usually won't be good enough to win.

"With 1,000 contestants, we expect that 10 will have skill levels of 99 or higher. Among those 10, the highest expected luck level is ... 90.9. The highest expected peformance socre among 1,000 contestants must therefore be at least P = 0.95 * 99 + 0.05 * 90.9 = 98.6, which is 1.2 points higher than the expected performance score of the most skillful contestant. 

"... The upshot is that even when luck counts only for a tiny fraction of total performance, the winner of a large contest will seldom be the most skillful contestant but will usually be one of the luckiest."*

(* I feel like I should point out that this sentence, while true, is maybe misleading. Frank is comparing the chance of being the *very highest* in skill with the chance of being *one of the highest* in luck. When skill is more important than luck (it's 19 times as important in Frank's example), it's also true (perhaps "19 times as true") that "the winner of a large contest will seldom be the luckiest contestant but will usually be one of the most skillful."  And, it's also true that "the winner of a large contest will seldom be the most skillful contestant, but even more seldom be the most lucky.")

--------

So, the most skilled of 10,000 competitors will wind up the winner only 55 percent of the time. Doesn't that prove that success is largely due to luck?

It depends what you mean by "largely due to luck."  Frank's experiment does show that, often, the luckier competitor wins over the more skillful competitor. Whether that alone constitutes "largely" is up to you, I guess. 

You could argue otherwise. As it turns out, the competitor with the most skill is still the one most likely to win the tournament, with a 55 percent chance. The person with the most luck is much less likely to win. Indeed, in Frank's simulation, perfect luck is only a 2.5 point bonus over average luck. So if the luckiest competitor isn't in the top 5 percent of skill, he or she CANNOT win.

It's true that the most successful competitors were likely to have been very lucky. But it's not true that the luckiest competitors were also the most successful.

Having said that ... I agree that in Frank's simulation, luck was indeed important, and the winner of the competition should realize that he or she was probably lucky -- especially in the 100,000 case, where the best player wins only 13 percent of the time. But Frank doesn't just talk about winners -- he talks about "successful" people. And you can be successful without finishing first. More on that later.

------

A big problem with Frank's simulation is that the results wind up enormously overinflated on the side of luck. That's because he uses uniform distributions for both luck and skill, rather than a bell-shaped (normal) distribution. This has the effect of artificially increasing competition at the top, which makes skill look much less important than it actually is. 

Out of 100,000 people in Frank's uniform distribution, more than 28,000 are within 1 SD of the highest-skilled competitor. But in a normal distribution, that number would be ... 70. So Frank inflates the relevant competition by a factor of 400 times.

To correct that, I created a version of Frank's simulation that used normal distributions instead of uniform. 

What happened? Instead of the top-skilled player winning only 13 percent of the time, that figure jumped to 88 percent.

Still ... Frank's use of the uniform distribution doesn't actually ruin his basic argument. That's because he assumed only 5 percent luck, and 95 percent skill. This, I think, vastly understates the amount of luck inherent in everyday life. 

It's easy to see that luck is important. The important question is: *how* important? I don't know how to find the answer to that, and when I discovered Frank's book, I was hoping he'd at least have taken a stab at it.

But, since we don't know, I'm just going to pick an arbitrary amount of luck and see where that leads. The arbitrary amount I'm going to pick is: 40 percent luck, and 60 percent skill. Why those numbers? Because that's roughly the breakdown of an MLB team's season record. Most readers of this blog have an intuitive idea of how much luck there is in a season, how often a team surprises the oddsmakers and its fans.

In effect, we're asking: suppose there are 100,000 teams in MLB, with only one division. How often does the most talented team finish at the top of the standings?

The answer to that question appears to be: about 11 percent of the time. 

(That's pretty close to the 13 percent that Frank gave, but it's coincidence that his uniform distribution with a 5/95 luck/talent split is close to my normal distribution with a 40/60 split.)

Here's something that surprised me. Suppose now, instead of 100,000 competitors, you make the competition ten times as big, so there's 1,000,000. How often does the best competitor win now?

I would have expected it to drop significantly lower than 11 percent. It doesn't. It actually rises a bit, to 14 percent. (Both these numbers are from simulations, so I'm not sure they're "statistically significantly" different.)

Why does this happen? I think it's because of the way the normal distribution works. The larger the population, the farther the highest value pulls away from the pack. 

On average, the most talented one-millionth of the population are more than around 4.75 SD from the mean. Suppose the average of those is 4.9 SD. So, we'll say the best competitor out of a million is around 4.9 SD from the mean.

If "catching distance" is 0.7 SD, you need to be 4.2 SD from the mean, which means your main competition consists of 13 competitors (out of a million).

But if there are only 100,000 in the pool, the most talented player is only around 4.4 SD from the mean, and "catching distance" only 3.7 SD. How many competitors are there above 3.7 SD? About 11 (out of 100,000).

The more competitors, the farther out a lead the best one has, which means the fewer competitors there are with a decent chance to catch him.

------

I decided to use the larger simulation, with a million competitors. A couple of results:

On average, the top performer in the simulation was the 442nd overall in talent. At first that may sound like merit doesn't matter much, but 442nd out of one million is still the top one-fiftieth of one percent -- the 99.95 percentile.

Going the other way, if you searched for the top player by talent, how did he or she perform? About 99th overall, or the 99.99 percentile. 

-------

We know (from Tango and others) that to get from observed performance to talent, we regress to the mean by this amount:

1 - (SD(talent)/SD(observed))^2

Assume SD(talent) = 60, and SD(luck) = 40. That means that SD(observed) = 72.1, which is the square root of 60 squared plus 40 squared.

So, we regress to the mean by 1-(60/72.1)^2, which is about 31 percent. 

If our top performer is at 4.9 SD observed, that's 72.1*4.9 = 353.29 units above average. Regressing 31 percent gives us an estimate of 243.77 units of talent. Since talent has an SD of 60, that's the equivalent 4.06 SD of talent.

That means if the top performer comes in at 4.9 SD above zero, his or her likeliest talent is 4.06 SD. That's about 27th out of a million, or some such.

In other words, the player with top performance should be around 27th in talent.

(Why, then, did the simulation come up with 442nd instead of 27th? I think it's because converting SDs to rankings isn't symmetrical when you can vary a lot.

For instance: suppose you wind up with two winners, one at 3.06 SD and one at 5.06 SD. The average of the SDs is 4.06, like we said. But, the 5.06 ranks first, while the 3.06 ranks 1000th or something. The average of the ranks doesn't wind up at 27 -- it's about 500.)

--------

The book is called "Success and Luck," but it really could be called "Money and Luck," because when Frank talks about "success," he really means "high income."  The point about luck is to support his idea of a consumption tax on the rich.

Frank's argument is that successful people should be willing to put up with higher taxes. His case, paraphrased, goes like this: "Look, the ultra-rich got that way because they were very lucky. So, they shouldn't mind paying more, especially once they understand how much their success depended on luck, and not their own actions."

About half the book is devoted to Frank discussing his proposal to change the tax system to get the ultra-rich to pay more. That plan comes from his 1999 book, "Luxury Fever." There and here, Frank argues that the ultra-rich don't actually value luxuries for their intrinsic value, but, rather, for their ability to flaunt their success. If we tax high consumption at a high rate, Frank argues, the wealthiest person will buy a $100K watch instead of a $700K watch (since the $100K watch will still cost $700K after tax) -- but he or she will still be as happy, since his or her social competitors will also downgrade the price of their watch, and the wealthiest person will still have the most expensive watch, which was his or her primary goal in the first place.

So, the rich still get the status of their expensive purchases, but the government has an extra $600K to spend on infrastructure, and that benefits everyone, including the rich. 

There are only a few pictures in the book, but one of them is a cartoon showing a $150,000 Porsche on a smooth road, as compared to a $333,000 Ferrari on a potholed road. Wouldn't the rich prefer to spend the extra $183,000 on taxes, Frank asks, so that the government can pave the roads properly and they can have a better driving experience overall? 

Almost every chapter of the book mentions that consumption tax ... especially Chapter 7, which is completely devoted to Frank's earlier proposal.

------

Since money is really the topic here, it would be nice to translate luck into dollars, instead of just standard deviations. Especially if we want to make sure Frank's consumption tax burden is fair, when compared to estimates of luck.

If money were linear with talent, it would be easy: we just regress 31 percent to the mean, and we're done. But, it's not. Income accelerates all the way up the percentile scale: slowly at the bottom, but increasingly as you get to the top. 

If you look at the bottom 97% of income tax filers, their income goes from zero to about a million dollars. If income were linear, the top 3% would go from $1 million to $1.03 million, right? But it doesn't: it explodes. In fact, the top 3% go from $1 million to maybe $500 million or more. 

(Income numbers come from IRS Table 1.1 here, for 2015, and articles discussing the 400 highest-income Americans.)

That means plain old regression to the mean won't work. So, I ran another simulation.

Well, it's actually the same simulation, but I added one thing. I assigned each performance rank an income, based on the IRS table, in order down, as the actual value of "talent". I assumed the most talented person "deserved" $500 million, and that's what he or she would earn if there were no luck involved. I assigned the second most talented person $300 million, and the third $200 million. Then, I used the IRS table to assign incomes all the way down the list of the 1 million people in the simulation. I rescaled the table to a million people, of course, and I assumed income was linear within an IRS category.

(BTW, if you disagree with the idea that even the most talented individuals deserve the high incomes seen in the IRS chart, that's fine. But that's a separate issue that has nothing to do with luck, and isn't discussed in the book.)

With the IRS table, I was able to calculate, for all performance ranks, how much they "should have" earned if their luck was actually zero.

The best performer earned $500 million. How much would he or she have earned based on talent alone, and no luck? A lot less: $129 million. The second-place finisher earned $300 million but deserved only $78 million. The third-place finisher earned $100 million instead of $48 million.

So, the top three finishers were lucky by $371 million, $222 million, and $52 million, respectively.

The 4-10 finishers were lucky by an average of $62 million. 

The 11 to 100 finishers were lucky by less, only $40 million.

The 101 to 500 finishers were lucky by a bit more, $42 million each.

At this point, we're only at the first 500 competitors out of a million. You'd expect that the trend to continue, that the next few thousand high-earners would also have been lucky, right? I mean, we're still in the multi-million-dollar range.

But, no.

At around 500, luck turns *negative*. Starting there, the participants actually made *less* than their skill was worth.

Those who finished 501-1000 are still in the income stratosphere -- they're the top 0.05% to the top 0.1%, earning between $10 million and $2.3 million. But, on average, their incomes were $460,000 less than what each would have earned based on skill alone.

It continues unlucky from there. The next 8000 people -- that is, the top 0.2 to 0.9 percent -- lost significant income to luck, more than $250,000 each. It's not just random noise in the simulation, either, because (a) every group shows unlucky, (b) there's a fairly smooth trend, and (c) I ran multiple simulations and they all came out roughly equivalent.

Here's a chart of all the ranges, dollar figures in thousands:

     1-10   +$61107
   11-100   +$39906
  101-500   +$ 4227
 501-1000   -$  460
1001-2000   -$  503
2001-3000   -$  401
3001-4000   -$  320
4001-5000   -$  265
5001-6000   -$  135
6001-7000   -$  224
7001-8000   -$  178
8001-9000   -$  201

(My chart stops at 9,000, because 9000 was about all I could keep track of with the software I was using. I believe the results would soon swing from unlucky back to lucky, and stay lucky until the average income of around $68,000.)

If we believe the data, we find that it's true that the ultra, ultra rich benefitted from good luck, at least the top 0.05% of the population. The "only" ultra-rich, the 0.05 to 0.9 percentile, the vast majority of the "one percenters" -- those people actually *lost* income due to *bad* luck.

This surprised me, but then I thought about it, and it makes sense. It's a consequence of the fact that income rises so dramatically at the top, where the top 0.01 percent earn ten times as much as the next 0.99 percent.

Suppose you finish 3,000th in performance, earning $1 million. If you're 2500th in talent, you should have had $2 million. If you were 3500th in talent but lucky, you should have earned maybe $900,000.

If you were lucky, you gained $100,000. If you were unlucky, you lost $1 million. 

So if those two have equal probabilities (which they almost do, in this case), the unlucky lose much more than the lucky gain. And that's why the "great but not really great" finishers were, on average, unlucky in terms of income.

-------

Here's a baseball analogy. 

Normally, we think of team luck in MLB in terms of wins. But, instead, think of it in terms of pennants. 

The team that wins the pennant was clearly lucky, winning 100% of a pennant instead of (say) the true 40% probability given its talent. The other teams must have all been unlucky.

Which teams were the *most* unlucky? Clearly not the second division, which wouldn't have come close to winning the pennant even if the winning team hadn't gotten hot. The most unlucky, obviously, must be the teams that came close. Those are that teams where, if the winning team had had worse luck, they would have been able to take advantage and finish first instead.

In our income simulation, the top 100 is like a pennant, since it's worth so much more than the rankings farther below. So, when a participant gets lucky and finishes in the top 100, where did the offsetting bad luck fall? On the participants who actually had a good chance, but didn't make it.

Suppose only the top 1 percent in skill have an appreciable chance to make the top 100 in income. That means that if the top 0.01 had good luck and made more than they were worth, it must have been the next 0.99 percent who had bad luck and made less than they were worth, since they were the only ones whose failure to make the top 100 was due to luck at all.

-------

Frank does seem to understand that it's the very top of the scale that's benefitted disproportionately from luck. In 1995, he co-wrote a book called "The Winner-Take-All Society", which argues that, over time, the rewards from being the best rise much faster than the rewards from being the second best or third best.

Recapping that previous book, Frank writes:

"[Co-author Philip] Cook and I argued that what's been changing is that new technologies and market institutions have been providing growing leverage for the talents of the ablest individuals. The best option available to patients suffering from a rare illness was once to consult with the most knowledgeable local practitioner. But now that medical records can be sent anywhere with a single click, today's patients can receive advice from the world's leading authority on that illness.

"Such changes didn't begin yesterday. Alfred Marshall, the great nineteenth-century British economist, described how advances in transportation enabled the best producers in almost every domain to extend their reach. Piano manufacturing, for example, was once widely dispersed, simply because pianos were so costly to transport ...

"But with each extension of the highway, rail, and canal systems, shipping costs fell sharply, and at each stop production became more concentrated. Worldwide, only a handful of the best piano producers now survive. It's of course a good thing that their superior offerings are now available to more people. but an inevitable side effect has been that producers with even a slight edge over their rivals went on to capture most of the industry's income.

"Therein lies a hint about why chance events have grown more important even as markets have become more competitive ..."

In other words: these days, the best doctor nationally has taken business away from the best doctor locally. But, the best doctor is the best doctor in part because of luck. So, luck rewards the best doctor nationally, but hurts the best doctor locally. And the best doctor locally is still pretty successful, maybe one of the richest people in town.

Which is what we see here, that the "ultra rich" gained from luck, and the merely "very rich" were actually hurt by it. Frank writes about the first part, but ignores the second part.

-------

Frank's implicit argument is that if people's success is more due to luck, it's more appropriate to tax them at a higher rate. I say "implicit" because I don't think he actually says it outright. I can't say for sure without rereading the book, but I think Frank's explicit argument is that if the rich are made to realize that they got where they were substantially because of good luck, they would be less resistant to his proposed high-rate consumption tax.

But if Frank *does* believe the lucky should pay tax at a higher rate, it follows logically that he has to also believe that the unlucky should pay tax at a lower rate. If Joe has been taxed more than Mary (at an identical income) because he was luckier, then Mary must have been taxed less than Joe because she was unluckier.

By Frank's own logic (but my simulation), that would mean that those who earned between $3,000,000 and $300,000 last year were unlucky, and deserve to pay less tax. I bet that's not what Frank had in mind.

-------

Of course, the model and numbers are debatable. In fact, they're almost certainly wrong. The biggest problem is probably the assumption that luck is normally distributed. There must be thousands of cases where a bit of luck turns a skilled performer, maybe someone normally in the $100K range, into a multi-million-dollar CEO or something. 

But who knows who those people are? They must be the minority, if we continue to assume that talent matters more than luck. But how small a minority, and how can we identify them to tax them more?

Anyway, regardless of what model you use, it does seem to me that the "second tier" of success, whoever those are, must be unlucky overall. 

In most cases, when you look at whoever is at the top of their category, they were probably lucky. If they hadn't been, who would be at the top instead? Probably the second or third in the category. Steve Wozniak instead of Bill Gates. Betamax (Sony) instead of VHS (JVC). Al Gore instead of George W. Bush. 

It seems pretty obvious to me that Wozniak, Betamax, and Al Gore have been very, very successful -- but not nearly as successful as they could have been, in large part because of bad luck. 

The main point of "The Winner-Take-All Society" is that the lucky (rich) winner winds up with a bigger share of the pie compared to the unlucky (but still rich) second-best, the unlucky (but still pretty rich) third best, and so on. In other words, the more "winner take all" there is, the bigger the difference between first and second place. 

The same forces that make the winner's income that much more a matter of good luck, must make the second-place finisher's income that much more a matter of bad luck. In a "Winner-Take-All Society," where only pennants pay off ... that's where luck becomes less important to the second division, not more.